My favourite Python snippets

I’ve coded Python full time for half a decade now, and I have found some snippets of code that are a godsend. I thought I would share them here, and if anyone has any amazing ones, please share in the comments!

The magic automatic, error-catching breakpoint:

This is hands down my single favourite piece of Python code, and the one that I implement in basically all of my projects.

import traceback, ipdb
import sys


def ipdb_sys_excepthook():
    """
    When called this function will set up the system exception hook.
    This hook throws one into an ipdb breakpoint if and where a system
    exception occurs in one's run.

    Example usage:
    >>> ipdb_sys_excepthook()
    """

    def info(type, value, tb):
        """
        System excepthook that includes an ipdb breakpoint.
        """
        if hasattr(sys, "ps1") or not sys.stderr.isatty():
            # we are in interactive mode or we don't have a tty-like
            # device, so we call the default hook
            sys.__excepthook__(type, value, tb)
        else:
            # we are NOT in interactive mode, print the exception...
            traceback.print_exception(type, value, tb)
            print
            # ...then start the debugger in post-mortem mode.
            # pdb.pm() # deprecated
            ipdb.post_mortem(tb)  # more "modern"

    sys.excepthook = info



def main():
  print("Everything is going swimmingly")
  raise NotImplementedError("Oh no what happened?")

if __name__ == "__main__":
  ipdb_sys_excepthook()
  main()

Just run ipdb_sys_excepthook() at the top of your script, or wherever your program starts to execute. It is that simple.

It will catch any exception (including KeyBoard Interrupts), print the full traceback, and then throw you into an ipdb breakpoint where the exception occured.

I cannot put into words how useful this is. It has probably saved me multiple personal relationships and dozens of debugging hours. It does not matter if the exception/error is in your code, or in some 3rd party package. You can interact with the locally scoped variables wherever the Exception occured, and move up and down the stacktrace with the usual ipdb commands such as u and d. It is incredible. Imagine you are running a long program that takes days, and suddenly there’s an error and your program crashes. That would normally be a disaster. However, with ipdb_sys_excepthook(), you have automatically been thrown into a breakpoint exactly where the problem occured, and can debug and see exactly what the issue was.

The magic breakpoint in action. `u` means “go up” in the stack trace.

The only failure I’ve ever seen for ipdb is that it doesn’t play nicely with list comprehensions, or any(...) as a boolean check. For some reason it doesn’t recognize the entities inside the list comprehension / any. However, fortunately it does play nicely with IPython, so you can always import IPython from inside your ipdb session and spin it up if you want! That’ll do away with the problems, and also allow you to run multiple lines of commands at once.

I cannot express how satisfying it is to catch a bug where it is, solve it, re-run the script, and then automatically stop at the next bug without having to manually place breakpoints. Having to re-run scripts to debug errors is no longer something I ever worry about.

In other words, zero-shot placement of the breakpoint. I love this code.

The *args to dict converter:

This is one I came across relatively recently, but it’s super cool, enough that I felt justified in writing this blog post because I suddenly had 2 cool snippets.

This function takes in an arbitrary list of arguments, and transforms it into a dict. The magic is where the dict key names come from: they are the names the variables were given in code. E.g., running args_to_dict(income, expenses) will return a dict with keys income and expenses, and the values will be the values of those variables.

import inspect 
import re

def args_to_dict(*args) -> Dict[str, Any]:
    """
    Capture the variable names of the arguments passed to the function.

    Example usage:
    a = 5
    b = "hello"
    >>> args_as_dict = args_to_dict(a, b)
    >>> print(args_as_dict)
    {'a': 5, 'b': 'hello'}

    Inputs:
    *args: The arguments passed to the function.

    Outputs:
    output (Dict[str, Any]): A dictionary containing the variable names of the arguments passed to the function.
    """
    frame = inspect.currentframe()
    outer_frame = frame.f_back
    calling_code = inspect.getframeinfo(outer_frame).code_context[0].strip()

    # Extract the argument string
    match = re.search(r"\((.*?)\)$", calling_code)
    if match:
        arg_string = match.group(1)

        # Split the argument string, handling potential commas within function calls or data structures
        arg_names = []
        paren_count = 0
        current_arg = ""
        for char in arg_string:
            if char == "(" or char == "[" or char == "{":
                paren_count += 1
            elif char == ")" or char == "]" or char == "}":
                paren_count -= 1

            if char == "," and paren_count == 0:
                arg_names.append(current_arg.strip())
                current_arg = ""
            else:
                current_arg += char

        if current_arg:
            arg_names.append(current_arg.strip())

        # Print the argument names and their values
        output = {}
        for name, value in zip(arg_names, args):
            output[name] = value
    else:
        raise ValueError("Couldn't parse the function call.")
    return output

It uses some regex to accomplish this, but considering that it’s working on Python syntax which is unlikely to change any time soon (Guido says 4.0 may never happen), I think it is forgiveable.

I find this particularly useful for all sorts of data applications, where I often want to convert lists of arguments into a dictionary. That process has historically been a bit messy to iterate on. E.g., adding a new entry into a dict can involve updating a mapping function to take in a new variable and add a new key to the dict.

For example, does this look familiar?

def extract_features_from_file(file: str) -> tuple:  
  ...
  ...
  return costs, income, nb_employees, nb_customers, contract_length

def process_features(feat_1, feat_2, feat_3, feat_4, feat_5):
    # Do something

def main():
    c, i, nb_e, nb_c, cl = extract_features_from_file("file.csv")
    process_features(c, i, nb_e, nb_c, cl)

That is a bit of a pain, since if you ever add a new feature, you need to update it in multiple places. What seems to happen 100% of the time is that you end up calling the same entity by different names in different parts of the code for no reason, and before you know it your code is a bit of a mess.

The following is a bit better:

def extract_features_from_file(file: str) -> dict:  
  ...
  ...
  return {
    "costs": costs,
    "income": income,
    "nb_employees": nb_employees,
    "nb_customers": nb_customers,
    "contract_length": contract_length,
  }

def process_features(features: dict):
    costs = features["costs"]
    # Do something

def main():
    features: dict = extract_features_from_file("file.csv")
    process_features(features)

Now, we have a single dict that we return, and it packages up everything neatly into a single object we can pass around.

args_to_dict takes it a step further:

def extract_features_from_file(file: str) -> dict:  
  ...
  ...
  return args_to_dict(costs, income, nb_employees, nb_customers, contract_length)

def process_features(features: dict):
    costs = features["costs"]
    # Do something

def main():
    features: dict = extract_features_from_file("file.csv")
    process_features(features)

Now:

• I don’t have to think basically at all about consistency between key names and variable names in the code.
• Spend energy thinking about how to route new variables through my pipeline. I just feed in an extra variable into args_to_dict, it deals with it flexibly, and I can access it from wherever that dict object is used.
• It’s just neat.

That is it for my favourite pieces of Python code!

As a bonus for the Neovim-ers out there, I have a few keybindings that make my Python experience really pleasant!

• leader+lb takes the current line I’m on and inserts a line break at 100 characters from the left margin.
• leader+ip inserts this code import ipdb; ipdb.set_trace() on a newline, automatically indented to the appropriate level.
• leader+o formats my code with black.
• leader+in indents the line I’m on by 4 spaces.
• leader+cp toggles my Co-pilot on/off.

Below are the Lua snippets from my neovim config if they’re useful (I use LazyVim).

-- Insert a Python ipdb breakpoint with auto-indentation
vim.api.nvim_set_keymap('n', '<leader>ip', [[<cmd>lua require('util.keymaps_py').insert_ipdb_breakpoint()<CR>]], { noremap = true, silent = true })

-- Insert linebreak 100 characters from start of line (doesn't split last word)
vim.api.nvim_set_keymap('n', '<leader>lb', '0100lBi<CR><ESC>', { noremap = true, silent = true })

-- Insert a standard Python indentation (4 spaces) at the start of the line
vim.api.nvim_set_keymap('n', '<leader>in', '0i<space><space><space><space><ESC>', { noremap = true, silent = true })

-- Auto-format
vim.keymap.set("n", "<leader>o", function()
    local efm = vim.lsp.get_active_clients({ name = "efm" })

    if vim.tbl_isempty(efm) then
        return
    end

    vim.lsp.buf.format({ name = "efm", async = true })
end, { noremap = true, silent = true, desc = "Auto-format file" })


-- Copilot toggle
api.nvim_create_user_command("CopilotToggle", function()
  vim.g.copilot_enabled = not vim.g.copilot_enabled
  if vim.g.copilot_enabled then
    vim.cmd("Copilot enable")
    print("Copilot ON")
  else
    vim.cmd("Copilot disable")
    print("Copilot OFF")
  end
end, { nargs = 0 })

vim.keymap.set("n", "<leader>cp", ":CopilotToggle<CR>", { noremap = true, silent = true })

Thanks for reading!

Do you have any favourite Python snippets (or any other language)? If so please share!