Dictionary-Tricks in Python

What can Python Dictionaries store?#

Dictionaries are datastructures containing key-value pairs (like a Map in Java), as you may know already. Besides that, dictionaries in Python are really special compared to similar data structures in other programming languages. They benefit from the fact that Python has dynamic typing, so you can use keys and values of different types in the same dictionary without any problems. You can use any object as value and every hashable object as key, which includes functions, classes, and even datatypes! You can find out whether an object is hashable just by passing it into the built-in hash() function.


def my_function():
    pass  # do something here

class MyClass:
    pass  # here happens some cool shit

example_dict = {
    my_function: True,
    False: 4j,
    int: 'you can use even types as key',
    MyClass: lambda x: pow(x, 2),
}

As you can see, dictionaries are really powerful and enable you to write creative and smart code with them. However, there are some limitations to the key values. For instance, mutable objects like lists are not hashable, so you can't use them as a key.

Looping over Dictionaries#

In Python, it is widespread to loop over lists, tuples, or other simple iterables, but what about dictionaries? In fact, you can loop over dictionaries the same way you loop over lists. But you may notice that you only get the keys when you try it out.

Normally you also want the values when iterating over a dictionary. The easiest approach to do this would be the following:


dictionary = {'key_1': 'value_1', 'key_2': 'value_2', 'key_3': 'value_3'}

for key in dictionary:
    value = dictionary[key]
    print(key, value)

It works, even though it's not really beautiful and pythonic. Luckily, there is a better solution that makes the code a bit prettier. Every dictionary has the items() method that returns an iterator providing a tuple containing key and value pairs.


dictionary = {'key_1': 'value_1', 'key_2': 'value_2', 'key_3': 'value_3'}

for key, value in dictionary.items():
    print(key, value)

Additionally, if you just want the dictionary values, you can use the values() method:


dictionary = {'key_1': 'value_1', 'key_2': 'value_2', 'key_3': 'value_3'}

for value in dictionary.values():
    print(value)

Unpacking Operator#

In my opinion, the unpacking operator is probably the smartest trick when it comes to writing cool and simple code with dictionaries in Python. In short, the unpacking operator ** unpacks dictionaries to keyword arguments. Another unpacking operator * which does nearly the same thing for other iterables like tuples or lists.

You also may have seen the double stars in some function parameters, where they do exactly the opposite: they wrap all (not defined) keyword arguments into a dictionary!

However, let's see what you can do with the unpacking operator:


import requests


request_arguments = {
    'url': 'any url',
    'params': {'param': 'value'},
    'data': {'some': 'payload data'},
    'auth': 'API-KEY',
    'allow_redirects': False,
    'timeout': 30,
    'headers': {'content-type': 'application/json'}
}

response = requests.post(**request_arguments)

Instead of writing all the keyword arguments directly in the function call, we can simply store them in a dictionary and unpack them in the function call. This makes the code more flexible and readable. Furthermore, we could go one step further and wrap this code in another function that can take any arguments and pass them to the post function! This technique can be particularly useful in larger projects where you want to generalize most of your functionality and write a lot of wrapper functions.

Besides that, there are many other cases to use the unpacking operator. Try it out and share your use cases in the comments!

Dictionary Comprehension#

You may have heard of list comprehensions before, but this trick works for dictionaries too!

The syntax is almost the same as for list comprehensions, the only difference is that you need two values on the left side: the key and the value, separated by a colon. The middle part represents the current value of the iterable which is on the right, exactly like list comprehensions or a classic for loop.


squared_values = {n: n**2 for n in range(5)}

assert squared_values == {
  0: 0, 
  1: 1, 
  2: 4, 
  3: 9, 
  4: 16,
}

This example shows a dictionary that stores the numbers from 0 to 99 as keys and the corresponding squared numbers as values. Sure, you could create much more complex dictionaries with this approach. However, always remember that others are reading this code too. To make your code more readable, it is often best to write a classic for loop. For this reason, you should only use dictionary comprehension in simple cases where it is clear what is happening.

Sorting a Dictionary#

In general, the dictionary is not meant to be a sorted data structure. However, dictionaries keep the insertion order by default since Python version 3.6. Although dictionaries have an order based on the insertions, Python has no built-in way to sort them (like lists, for instance).

Nevertheless, we can sort a dictionary by using the feature that you can create dictionaries with a list of tuples that contain key and value pairs. So we can get those key-value tuples with the items() method, sort them and create a new dict with the sorted tuples.


some_http_codes = {
    404: 'Not Found',
    401: 'Unauthorized',
    400: 'Bad Request',
    403: 'Forbidden',
    402: 'Payment Required',
    405: 'Method Not Allowed',
}

sorted_http_codes = dict(sorted(some_http_codes.items(), 
                                key=lambda kv: kv[0], 
                                reverse=False))

assert sorted_http_codes == {
    400: 'Bad Request',
    401: 'Unauthorized',
    402: 'Payment Required',
    403: 'Forbidden',
    404: 'Not Found',
    405: 'Method Not Allowed',
}

The sorted() function takes the key-value tuples and sorts them. To sort them correctly, it is important to set the sorting key. This is just a function that takes the key-value tuples and returns the value by which the pairs should get sorted. In this case, we want to sort by the HTTP code (the key), which is always stored on the 0th index in each tuple. Therefore the lambda expression we use for the sorting key returns this value. If you want to sort the dictionary by the values, just return the value at index 1.

Default Values#

Whenever you want to get values from a dictionary it is possible that the given key doesn't exist. In this case, Python raises a KeyError that you have to catch, otherwise the program exits. The problem with covering unknown keys is the amount of boilerplate code it produces. It makes the code less readable and more clunky.

Another way to avoid KeyErrors is to check if the key exists before accessing it:


player_points = {
    "Bob": 1337,
    "Alice": 42,
}

def get_points_for_player(player_name: str) -> int:
    if player_name in player_points:
        return player_points[player_name]
    return 0

get_points_for_player("Bob")  # returns 1337
get_points_for_player("Kevin")  # returns 0

But this approach doesn't really solve the problem and produces some boilerplate code, although it's better than the previous one. Luckily, there are two ways to get a default value when a key doesn't exist!

The easiest way that works for any dictionary is to use the get() method when retrieving data from it. The first argument for this method is the key you are looking for. The interesting part is the second argument, where you can optionally pass a default value, which will be returned when the key was not found.


player_points = {
    "Bob": 1337,
    "Alice": 42,
}


def get_points_for_player(player_name: str) -> int:
    return player_points.get(player_name, 0)


assert get_points_for_player("Bob") == 1337
assert get_points_for_player("Kevin") == 0

Another way to get a default value when the key doesn't exist is to use the collections module's defaultdict data structure. The defaultdict works like a normal dictionary with the additional feature to define a default value, which is always returned when retrieving data from an unknown key. The first argument in the constructor is a callable whose return value is used as the default argument. Every argument after that will work with the normal dictionary constructor.


from collections import defaultdict
from datetime import datetime

last_login = defaultdict(lambda: datetime.now(), {
    "Bob": datetime(year=2020, month=1, day=1),
    "Alice": datetime(year=2020, month=1, day=2),
})

assert last_login["Bob"] == datetime(year=2020, month=1, day=1)
assert isinstance(last_login["Kevin"], datetime)

This code snippet shows the same example as the previous one, but is more powerful because you don't have to specify a fixed value and can create the value dynamically on the fly. Everybody using the dictionary doesn't need to care about setting defaults or catching a KeyError. However, notice every time accessing the dictionary with a non-existing key, a new entry with this key is added!

Creating a Dictionary from two Lists#

If you ever encounter the situation where two lists should create a dictionary where one list contains the keys and the other one the values, there is a simple solution!


def dict_from_two_lists(keys: list, values: list) -> dict:
    if not len(keys) == len(values):
        raise ValueError("List of keys must have the same length as the list of values!")
        
    return dict(zip(keys, values))

names = ["Bob", "Alice", "Kevin"]
ages = [21, 42, 69]

ages_of_persons = dict_from_two_lists(keys=names, values=ages)

assert ages_of_persons == {
    "Bob": 21,
    "Alice": 42,
    "Kevin": 69,
}

We use the feature to create a dictionary by passing a list of tuples to the constructor, where every tuple is a key-value pair. The zip function creates such a list of tuples with two elements. The alternative approach would be a for loop with a counter variable, which adds every item to the dict, but the zip-approach is clearly more beautiful. In general, you can do really cool things with the zip function combined with dictionaries. Make sure to keep this option in mind before writing ugly code!

Deleting Items#

Adding items to a dictionary is quite simple, but what about deleting items? Fair enough, deleting an item from a dictionary is really simple. However, there are a few things that you should have in mind when you want to delete items from a Python dictionary.

First of all, there are two methods how you can delete items from a dictionary:

use the del keyword
use the pop() method

The main difference between those possibilities is that the pop method returns the removed value, whereas del just deletes the item.


sample_dict = {'value_one': 1, 'value_two': 2}

del sample_dict['value_one']
assert 'value_one' not in sample_dict

removed_value = sample_dict.pop('value_two')
assert 'value_two' not in sample_dict
assert removed_value == 2

It really becomes interesting when you want to delete items while iterating over the same dict. Python will throw a RuntimeError when you try to do this because you change the dictionary's size. Unfortunately, I can't present you a really cool solution for this case, but here are two different workarounds. If you know a better solution that solves the problem in a better way, please let me know in the comments!

Create a copy of the dictionary just for the loop


def remove_even_numbers_1(dictionary: dict):
    for key, value in dictionary.copy().items():
        if key % 2 == 0:
            del dictionary[key]

Delete entries directly from previously calculated keys


def remove_even_numbers_2(dictionary: dict, items_to_delete: list):
    numbers_to_delete = set(key for key, value in dictionary.items() if key % 2 == 0)
    
    for number in numbers_to_delete:
        del dictionary[number]

I personally don't favor any of these approaches; I think it depends on the context which one fits better. Either way, neither of the two variants is really beautiful.

Merging Dictionaries#

If you want to merge dictionaries, you have 3 options:

Use the update() method

Every dictionary has this method, which can take a dictionary as argument. The items of the passed dictionary get updated if they already exist or added if not. Additionally you can create a new dict by copying one of the dictionaries you want to merge before doing the update. This has the advantage that you don't change the original dicts when you merge them.


def merge_two_dicts(first: dict, second: dict) -> dict:
    merged_dicts = first.copy()
    merged_dicts.update(second)
    return merged_dicts

Merge with the unpacking operator

A much simpler approach to merge dicts is to simply create a new dictionary with the items from both dicts. You can use the previously explained unpacking operator for this.


def merge_two_dicts(first: dict, second: dict) -> dict:
    return dict(**first, **second)

Use the merge operator (since Python 3.9)

Python 3.9 introduced the single pipe | as the merge operator for dictionaries. If you run your project with Python version 3.9 or newer, this might be the best approach.


first = {'a': 1, 'b': 2}
second = {'c': 3, 'd': 4}

merged = first | second

assert merged == {'a': 1, 'b': 2, 'c': 3, 'd': 4}

Making copies of a Dictionary#

With the final dictionary trick, I want to raise awareness of the things to keep in mind when copying dictionaries. Dictionaries are mutable data structures, which means that every time you pass a dictionary as an argument to a function or assign it to a new variable, the dictionary gets passed by reference. So everything you do with the dictionary in this function will affect the original one too! In comparison, when you do the same thing with a string (which is immutable), the function's operations do not affect the original string because it gets passed by value.

This makes it harder to create copies of dictionaries. Therefore every dictionary has a copy() method, which creates a shallow copy of the dictionary. You can check this with the id() function that returns the internal id of an object.


sample_dict = {'key': 'value'}

sample_dict_too = sample_dict  # doesn't create a copy
assert id(sample_dict) == id(sample_dict_too)  # same ids

sample_dict_copy = sample_dict.copy()
assert id(sample_dict) != id(sample_dict_copy)  # not the same ids
assert sample_dict == sample_dict_copy  # content is the same

Another important fact to keep in mind is that the copy() function only creates a shallow copy of the dictionary. This means that the items are inserted as references into the copied dictionary. If you have immutable objects as values, they will be the same in the original and the copy! To avoid this, you can use the copy module's deepcopy() function, which will insert copies of the dictionary items.


import copy


sample_dict = {'key': ['sample', 'list']}

shallow_copy_dict = sample_dict.copy()
assert id(sample_dict['key']) == id(shallow_copy_dict['key'])

deep_copy_dict = copy.deepcopy(sample_dict)
assert id(sample_dict['key']) != id(deep_copy_dict['key'])

Summary#

This post presented you 10 dictionary-tricks in Python. I hope you enjoyed them and learned new pythonic approaches to apply to your projects! However, I guess there are many more cool things and tricks you can do with dictionaries. If you have one that can't be missing here, leave a comment below!