Handling form submissions is (still) a time-consuming and disappointing task.
Sometimes all you want is to create a form that triggers an email with the
provided details. The sad truth is, there is no such solution available for free.

But there is a solution that comes with nearly zero cost and effort: Utilizing
[AWS Lambda](https://aws.amazon.com/lambda/) and
[AWS Simple Email Service (SES)](https://aws.amazon.com/ses/).

![Form data to email architecture proposal](/assets/blog/posts/form_aws_lambda/aws-gateway-lambda-ses-form-data-architecture.svg)

If you never worked with AWS, no worries,
this requires absolutely no in-depth knowledge. All you need to know, is how to
[navigate AWS](https://www.youtube.com/watch?v=A43m4TDFCUM). However, I recommend you read
about the
[Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/)
and to create a
[budget alert](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-create.html)
when you're starting to use AWS to avoid costly mistakes.

## Components

Even though it might seem trivial, I want to lose a few words to the components
in use. Feel free to skip this section if you're already familiar with the AWS
API Gateway, AWS Lambda, and AWS SES Services.

### API Gateway

The API Gateway is a fully managed service that allows exposing business logic
via API endpoints. The API Gateway supports containerized, serverless workloads
as well as web applications. It is the entry point to your application logic.

We utilize its feature to invoke Lambdas via a public endpoint in our case.

### Lambda

Lambda is based on the serverless concept. Instead of ever-running EC2 instances
(the AWS solution for virtual machines), lambdas only consume resources when
they are invoked. The actual costs are calculated in Gigabyteseconds (GBS). This unit
is calculated by multiplying the required main memory times the execution time. For
example, if you have a memory consumption of 80MB and your Lambda takes 560ms to
compute, this equals ~0.044GBS

<math>
  GBS(80MB,560ms) = \frac{'{'}80{'}'}
  {'{'}1024{'}'}GB*0.56s = 0.04375 GBS
</math>

1GBS is currently billed at _$0.00001667_ which means your request effectively
costs _$0.0000007293125_ or in other words, you can make about 1.37
million requests until you have to pay a single dollar.

The savings in resources are hilariously high. While virtual machines usually
cost at least a few cents an hour at any given cloud provider; you can make
thousands of serverless requests until you have to pay the same.

### Simple Email Service (SES)

The AWS SES is a service to send and receive emails. It supports various nice
features such as DKIM. This will almost certainly ensure your mails won't get
tagged as spam as long as your content does not behave like phishing.

To use SES in this scenario, you will have to confirm the email or domain you're
going to use to send mails. The SES wizard is clear and concise about the
instructions; you should not run into any issues here.

## Why use AWS

When it comes to architecture, in 9 of 10 cases, the definitive answer to your
question is: _„it depends“_. So maybe let's take one step back and talk about
why this solution might fit your use case.

The two factors that led to my decision were **Availability** and **Costs**.
Neither is a solution with low availability nor high costs tolerable for
submitting mere forms.

### Availability

The availability of this solution is at least **99,9%**. Which equals about 8.7
hours of downtime per year. The AWS API Gateway and the AWS Lambda Service have
an availability of at least 99,95%, according to their service level agreements (SLA).

<math>Availability(API Gateway, Lambda)=0.9995 * 0.9995=~0.999 = 99.9\%</math>

This is enterprise level availability and most likely much higher than the
availability of your self-hosted backend. Also, if necessary, it would be pretty
easy to increase the availability by utilizing availability zones, for instance.

### Costs

The costs are staggeringly low. All of this easily fits in the free-tier
offering of AWS and thereby your costs for the first 12 months would be $0.

After that, your costs are that low; they won't even sum up to a total of $1 per
year. But let's calculate that.

Let's assume:

- your website is frequently visited, and 50 people submit a form each day
- the lambda takes 75MB of main memory and 80ms to complete (my average)

That would be _18250 submissions_ (50\*365) and _~107GBS_
(75/1024GB\*0,08s\*18250 submissions) per year.

- 18250 requests via the API Gateway are billed with $0.0219 ($1.20 per million)
- 107 GBS via Lambda are billed at $0.00178 ($0.00001667 per GBS)
- 18250 emails via SES are billed with $0 (the first 62 000 each month are free)

This sums up to a _total of $0.02 per year_. I don't think there is any other
managed solution that can match these costs. But feel free to prove me wrong
in the comments.

## Lambda Function

First, we create a new lambda. Navigate to **Lambda** on the AWS Cloud web
interface. Make sure your region is set correctly (at the top bar on the
right side). Click on _Create function_, add your desired function name and
select NodeJS as runtime, and click on _Create function_ again (at the bottom).

After the creation succeeded, you should now be in the integrated code editor
with some default code:

```javascript
exports.handler = async event => {
  // TODO implement
  const response = {
    statusCode: 200,
    body: JSON.stringify('Hello from Lambda!')
  }
  return response
}
```

As you can see here, a lambda function is exported by `exports.handler` and is
an asynchronous function with a single argument `event`.

Now let's paste in our code to process the form submission:

```javascript
const AWS = require('aws-sdk') // Import the aws sdk
const ses = new AWS.SES({ region: 'eu-central-1' }) // Create an SES client bound to eu-central-1

const RECEIVER = 'info@code-specialist.com' // The recipient
const SENDER = 'no-reply@code-specialist.com' // The sender (Must be configured with SES)

const response = {
  // The response we're going to send to the requester
  statusCode: 200,
  body: JSON.stringify('Success')
}

exports.handler = async function (event) {
  // The actual lambda
  console.log('Received event:', event) // To log the event, make sure not to violate applicable data protection directives!
  const message = await decodeAndTransformContent(event.body) // Decode and transform the form submission
  await sendEmail(message) // Send the email
  return response // Return the response
}

async function decodeAndTransformContent (eventBody) {
  const buffer = Buffer.from(eventBody, 'base64') // Create a buffer
  const content = buffer.toString('ascii') // Decode the base64 encoded form
  const tuples = content.split('&') // "a=b&c=d" => ["a=b", "c=d"]
  const kv_pairs = tuples.map(pairs => pairs.split('=')) // ["a=b", "c=d"] => [["a", "b"], ["c", "d"]]
  const formatted = kv_pairs.map(pairs => `<b>${pairs[0]}</b>: ${pairs[1]}`) // [["a", "b"], ["c", "d"]] => ["<b>a</b>: c", ...]
  return formatted.join('<br/>') // ["<b>a</b>: c", ...] => "<b>a</b>: b <br/>..."
}

async function sendEmail (message) {
  const params = {
    Destination: {
      ToAddresses: [RECEIVER] // A list of recipients
    },
    Message: {
      Body: {
        Html: {
          Data: message // The actual content
        }
      },
      Subject: {
        Data: 'Contact Form Submission' // Your subject
      }
    },
    Source: SENDER // The sender of the mail (Must be configured with SES)
  }
  return ses.sendEmail(params).promise() // Return the sendEmail promise
}
```

All of this code should be self-explanatory. If not, the comments should clarify
what's happening. The serverless function will take any given form event and create a HTML
message body for all the key-value pairs provided.

### Configure the Policy

When you created the lambda, a basic policy has been created automatically to
execute the lambda. We now have to extend this policy, so it also allows us to
send emails via SES.

Navigate to the **IAM** service and then to _Policies_. You should
find something named similar to
`AWSLambdaBasicExecutionRole-5749fdbf-a74e-49a6-a18e-e8dc2d5e7c98`. Click on it
and then press _Edit policy_. By default, this will open a visual editor.
Click on the _JSON_ tab instead.

You should now see the active policy statements in a JSON format:

```JSON
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:eu-central-1:1234567891011:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:eu-central-1:1234567891011:log-group:/aws/lambda/formFunction:*"
            ]
        }
    ]
}
```

Add another statement:

```JSON
...
    {
        "Effect": "Allow",
        "Action": [
            "ses:SendEmail",
            "ses:SendRawEmail"
        ],
        "Resource": "*"
    }
...
```

Finally, click on _Review policy_ and afterward on _Save changes_.

### Create an API Gateway

Last but not least, we need to create an API Gateway to expose the Lambda we
defined via a `POST` action. To do so, navigate to the **API Gateway** service.

Click on **Create API** and select **HTTP API**. You can now click on **Add
integration** and select your Lambda. Next, choose your API's name and click on
_Next_. You should now see an UI to configure your routes Select the method
`POST` for the lambda and choose a resource path for it. The resource path will
be the relative address of your endpoint. Leave everything else as it is and
navigate via _Next_ until you can click on _Create_.

You should now see an **Invoke URL** like
`https://code-specialistxyz.execute-api.eu-central-1.amazonaws.com`, which is the
base URL of your API. To call your Lambda you can now execute a post to
`https://code-specialistxyz.execute-api.eu-central-1.amazonaws.com/contactForm`
assuming your resource path is `contactForm`.

## Testing the function

To test your endpoint, you could use this minimal HTML5 snippet:

```html
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>Test Form</title>
  </head>
  <body>
    
    <form
      action="https://code-specialistxyz.execute-api.eu-central-1.amazonaws.com/contactForm"
      method="post"
    >
      <span>Name</span>
      <input name="name" /><br />
      <span>Email</span>
      <input name="email" /><br />
      <span>Message</span>
      <input name="message" /><br />
      <button type="submit">Submit</button>
    </form>
  </body>
</html>
```

And that's basically it. You're ready to go! 🚀

If anything didn't work as expected or you're in need of an explanation
somewhere, don't hesitate to leave us a comment 😊

Form Endpoint to send Emails with AWS Lambda and AWS SES


Multi-factor authentication is all around us nowadays. And for a good reason,
single layers of authentication are easily compromised by leaked credentials,
exposed keys or even third party security leaks.

However, there is no reason to get paranoid. Two or at most three
are enough when it comes to layers of authentication. The risk of
locking out yourself increases with each layer and restoring gets trickier too.

Also, various layers of authentication do not necessarily mean your server is
more secure each layer may be a vulnerability itself if not choosen wisely.

I usually use two layers:

- SSH-Key
- Time-based one-time Password (TOTP)

In the following three sections we're going through basic security measures and
how to configure your server to apply SSH-Key and TOTP (**Google
Authenticator**) based authentication. But first of all we start with some basic
security measures.

<InfoBox>
  In this example we're going to assume you use <b>Ubuntu 20.04</b>, but most
  concepts are easy to adjust to various operating systems and require only
  slight to none changes using another ubuntu version. <br />
  <br />
  This is a practical how-to, we assume you know about the basics.{' '}
  <b>Be careful when changeing security related settings</b>. You may easily
  lock yourself out or even create vulnerabilities if you lack proper knowledge
  of what you're doing. <b>Do not apply anything blindly.</b>
  <br />
  <br />
  To lessen the chance of a full lock out, you might want to keep a second ssh session
  connected at all time while applying the steps below.
</InfoBox>

## Basic Security

Your auth layers are basically useless if they're exorbitantly vulnerable. The least you
should do, is to make sure everything is up to date, disable the root login and
configure a basic firewall.

### Update

First thing you should do when configuring a new Linux server, is to update the packages

1. Update package lists

   ```shell
   sudo apt-get update
   ```

   ---

2. Upgrade the packages

   ```shell
   sudo apt-get upgrade -y
   ```

### Deactivate Root Login

One thing you really should do as your second action when configuring a new
server, is to disable the root user or atleast prohibit login with it. The root
is simply to mighty and a leak of its credentials is your servers armageddon.

1. Connect via SSH

   ```shell
   ssh root@111.11.11.11
   root@111.11.11.11 password:
   ```

   ---

2. Create a new user

   ```shell
   sudo adduser codespecialist
   ```

   **Shell output**

   ```shell
   Adding user `codespecialist' ...
   Adding new group `codespecialist' (1000) ...
   Adding new user `codespecialist' (1000) with group `codespecialist' ...
   Creating home directory `/home/codespecialist' ...
   Copying files from `/etc/skel' ...
   New password:
   Retype new password:
   passwd: password updated successfully
   Changing the user information for codespecialist
   Enter the new value, or press ENTER for the default
         Full Name []:
         Room Number []:
         Work Phone []:
         Home Phone []:
         Other []:
   Is the information correct? [Y/n]
   ```

   ---

3. Add the user to sudoers

   ```shell
   usermod -aG sudo codespecialist
   ```

   ***

4. Switch and check permissions

   ```shell
   su codespecialist
   sudo ls
   ```

   **Shell output**

   ```shell
   codespecialist@ubuntu:~$ sudo ls
   [sudo] password for codespecialist:
   codespecialist@ubuntu:~$
   ```

   If the sudo command succeeds, then you have been successfully aded to the
   sudoers.

   ***

5. Test logging in

   ```shell
   ssh codespecialist@111.11.11.11
   ```

   You should be able to login with the credentials you just created.

   ***

6. Disable the root login

   Open the `sshd_config`

   ```shell
   sudo nano /etc/ssh/sshd_config
   ```

   find the `PermitRootLogin` entry and change it to `no`

   ```config{numberLines: true}
   # Authentication:

   ...

   PermitRootLogin no

   ...
   ```

   ***

7. Restart the SSHD service

   ```shell
   sudo systemctl restart sshd
   ```

   ***

8. Test it

   Open a second terminal and try to connect as root

   ```
   ssh root@111.11.11.11
   ```

   **Shell output**

   ```shell
   root@111.11.11.11's password:
   Permission denied, please try again
   ```

### Set up a Firewall

To restrict access from and to ports, you might want to add a basic firewall.
One firewall widely used is the [Uncomplicated Firewall
(UFW)](https://en.wikipedia.org/wiki/Uncomplicated_Firewall)

1. Install UFW

   ```shell
   sudo apt-get install ufw
   ```

   ***

2. Add SSH firewall rule

   Allow ssh connections, either by applying `sudo ufw allow ssh` or `sudo ufw allow 22`.

   ```shell
   sudo ufw allow ssh
   ```

   **Shell output**

   ```shell
   [sudo] password for codespecialist:
   Rules updated
   Rules updated (v6)
   ```

   Make sure this step proceeds by checking the output. Else, you might lock out
   yourself and exstinguish any chance to restore the server as well.

   ***

3. Enable the firewall

   ```shell
   sudo ufw enable
   ```

   When you use the `ufw enable` command, UFW will be started, and also
   registered as a service started on system startup.

## Authenticate using a SSH-Key

Using a SSH-Key is an easy way to improve security without increasing the number
of manually provided credentials. All you have to do is configure your local ssh-agent
and provide your public key to the machine your want to secure. You need to
apply the following steps on your local machine.

<InfoBox>
   This section requires you to have installed <b>openshh-client</b>. This
   should be preinstalled on most operating systems. But you might have to find
   a work-arround when using Windows.
</InfoBox>

1. [Optional] Create a SSH-Key

   If you do not have a ssh key yet, you can create a new one. It's possibly a
   good choice to add a comment with `-C` to allocate it correctly later on.
   Follow the instructions given by the CLI. Also, make sure not to loose the
   private key created in this step. Its also recommended you use a passphrase
   in order for the key to be encrypted with AES-CBC instead of lying arround as
   plain text files

   ```shell
   ssh-keygen -t ed25519 -C "codespecialist@111.11.11.11"
   ```

   **Shell output**

   ```shell
   ssh-keygen -t ed25519 -C "codespecialist@111.11.11.11"
   Generating public/private ed25519 key pair.
   Enter file in which to save the key (/Users/codespecialist/.ssh/id_ed25519): /Users/codespecialist/.ssh/codespecialist
   Enter passphrase (empty for no passphrase):
   Enter same passphrase again:
   Your identification has been saved in /Users/codespecialist/.ssh/codespecialist
   Your public key has been saved in /Users/codespecialist/.ssh/codespecialist.pub
   The key fingerprint is:
   SHA256:ggejZWoZgOYJL4epj/IkvL34z0T3Var+GBS6K93kiTo codespecialist@111.11.11.11
   The key's randomart image is:
   +--[ED25519 256]--+
   |o                |
   |oo               |
   |+=. =    .   .   |
   |=.oB +  . . +    |
   |.o= ..ooS. o     |
   |o.  ....+.o      |
   |.+.  ...=+.      |
   |oo= oE oo+o      |
   |.+o+o++. o..     |
   +----[SHA256]-----+
   ```
   
   ---

2. Add the key to the server

   ```shell
   ssh-copy-id -i ~/.ssh/codespecialist codespecialist@111.11.11.11
   ```

   **Shell output**

   ```shell
   ssh-copy-id -i /Users/codespecialist/.ssh/codespecialist codespecialist@111.11.11.11
   /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/codespecialist/.ssh/codespecialist.pub"
   /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
   /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
   codespecialist@111.11.11.11's password:

   Number of key(s) added:        1

   Now try logging into the machine, with:   "ssh 'codespecialist@111.11.11.11'"
   and check to make sure that only the key(s) you wanted were added.
   ```

   ---

3. Check if the key was added correctly on the remote host

   ```shell
   cat ~/.ssh/authorized_keys
   ```

   **Shell output**

   ```shell
   ssh-ed25519 AAAAC3NzaC2lZDI1NTE5AAAAIMdM36AX6tY/+vnPSQ4JLB20SP2ANU7js8uBnIzlQHFf codespecialist@111.11.11.11
   ```

   ---

4. Disable password authentication

   Edit your `sshd_config` and change the `PasswordAuthentication` from `yes` to `no`

   ```shell
   sudo nano /etc/ssh/sshd_config
   ```

   ```
   ...
   PasswordAuthentication no
   ...
   ```

   ---

5. Restart the SSHD service

   ```shell
   sudo systemctl restart sshd
   ```

   ---

6. Login using the SSH-Key

   Logging in without password should now be possible, but at the same time,
   only your SSH-Key will be accepted from now on.

   ```shell
   ssh -i ~/.ssh/codespecialist codespecialist@111.11.11.11
   ```

## Authenticate using the Google-Authenticator

Now that we have a decent authentication mechanism and some basic security
features, lets add a second layer to the authentication.

<InfoBox>
Make sure you are logged in to the remote host with the user you want to use in
future, as the google authenticator is only configured for this specific user.
<br/><br/>

Also, please open a second SSH connection to the server and keep it open until
you're sure, the google autenticator is working properly. Otherwise you might
lock out yourself with a single misconfiguration.
</InfoBox>

1. Install the google authenticator 
   
   ```shell   
   sudo apt-get install libpam-google-authenticator
   ```

   ---

2. Run the google authenticator script

   Once you followed the steps of the google authenticator, a QR-code will be
   generated in the shell, that you can scan with an authenticator app on your
   smartphone, tablet or whatever. Just make sure, its not your local machine, as
   the authentication layer is horribly ineffective against attackers compormising
   your local machine.

   ```shell
   google-authenticator
   ```

   **Shell output** 

   ```shell

   Do you want authentication tokens to be time-based (y/n) y
   Warning: pasting the following URL into your browser exposes the OTP secret to Google:

   Your new secret key is: G2URAC442DYGBIJDCDHY2HTFNM
   Your verification code is 221804
   Your emergency scratch codes are:
   25690887
   90233439
   52851954
   71051205
   30624510

   Do you want me to update your "/home/codespecialist/.google_authenticator" file? (y/n) y 

   Do you want to disallow multiple uses of the same authentication
   token? This restricts you to one login about every 30s, but it increases
   your chances to notice or even prevent man-in-the-middle attacks (y/n) y

   By default, a new token is generated every 30 seconds by the mobile app.
   In order to compensate for possible time-skew between the client and the server,
   we allow an extra token before and after the current time. This allows for a
   time skew of up to 30 seconds between authentication server and client. If you
   experience problems with poor time synchronization, you can increase the window
   from its default size of 3 permitted codes (one previous code, the current
   code, the next code) to 17 permitted codes (the 8 previous codes, the current
   code, and the 8 next codes). This will permit for a time skew of up to 4 minutes
   between client and server.
   Do you want to do so? (y/n) y

   If the computer that you are logging into isn't hardened against brute-force
   login attempts, you can enable rate-limiting for the authentication module.
   By default, this limits attackers to no more than 3 login attempts every 30s.
   Do you want to enable rate-limiting? (y/n) y
   ```

   Make sure to write down these scratch codes, as they are the only thing that
   may help you, if you loose or damage your device with the TOTP-App.

   ---

3. Add the google authenticator as a auth factor

   ```shell
   sudo nano /etc/pam.d/sshd
   ```

   Search for the `@include common-auth` line which should be at the very
   beginning and comment it out.

   ```config
   #@include common-auth
   ```
   
   Add the authenticator module at the bottom of the file

   ```config
   auth required pam_google_authenticator.so
   ```

   ---

4. Adjust the SSHD config

   There are several things in the config you have to change or assure this
   time:
   
   - `ChallengeResponseAuthentication` is set to  `yes`
   - `UsePAM` is set to  `yes`
   - `AuthenticationMethods` is set to  `publickey,keyboard-interactive`

   ```shell
   ...
   ChallengeResponseAuthentication yes
   ...
   UsePAM yes
   ...
   AuthenticationMethods publickey,keyboard-interactive
   ```

   ---

5. Restart the SSHD service

   ```shell
   sudo systemctl restart sshd
   ```

   ---

6. Test it

   When try you open a new ssh session with the server, it is going to ask for a
   verification code, which should accept the one your TOTP app displays.

   ```shell
   ssh -i .ssh/codespecialist codespecialist@111.11.11.11
   ```

   **Shell output**

   ```
   (codespecialist@111.11.11.11) Verification code: 
   ```

## Congratulations!

Your remote host is now a bit more secure, and the likelyhood of you being a
victim of bruteforce attacks significantly lower. If you are interested in
further ways to improve security, system hardening is the topic to look
into next. 

If you encounter any issues please comment below, so we can provide a solution
and improve this article.

Linux Multi-Factor Authentication


Hyrum Wright is a Staff Software Engineer at Google, according to his [LinkedIn profile](https://www.linkedin.com/in/hyrum-wright-0905427/). He made an observation nowadays
called "**Hyrum's Law**". It regards interfaces and API design and should be considered and memorized by anyone employed in the **software engineering** sector.

## Hyrum's Law

> With a sufficient number of users of an API, it does no matter what you promise in the contract: all observable behaviors of your system will be dependend on by somebody.
>
> Hyrum Wright on [https://www.hyrumslaw.com/](https://www.hyrumslaw.com/) - popularized by Google Employees and Titus Winters

## Definitions

To fully understand the impact of this sentence, we have to define a few things:

- **_API_** \- Any occurrence of an interface
- _**Interface** \-_ Not the concrete code pattern such as in Java, for example, but the abstract concept that defines the exposure of any consumable functions

## Significance and Meaning

Hyrum himself called this observation "**The Law of Implicit Dependencies**". But people at Google spread the name "Hyrum's **Law**" for the apparent reason of briefness. However,
that's also why nobody not knowing about it will understand what's meant.

A colloquial interpretation of this law could be: "Every change breaks someone's workflow", as ridiculously shown in a comic on [https://xkcd.com/1172/](https://xkcd.com/1172/).
But to understand that, we have to take a step back and look at the big picture: Anyone exposing his code as (re)usable functions inside a project, or for example, as a RESTful
APIs outside the project, usually does that for the sake of people to explicitly use the code. However, by the laws of statistics, all the observable behaviors your code supplies
will eventually be used by someone, thereby implicitly assuming things.

## Example

To [grasp](https://code-specialist.com/code-principles/grasp/) what these abstract sentences actually mean, take this example: You build an API that publicly delivers cat facts:

```json
{
  "catfacts": [
    "fact"
    :
    {
      "number": 1,
      "text": "Cats have 230 bones while humans only have 206."
    },
    "fact"
    :
    {
      "number": 2,
      "text": "Cats live longer if they live indoor."
    }
  ]
}
```

Given this information, people will implicitly assume that this interface serves cat facts in ascending order regarding their number. Meaning if the API delivers these facts in a
random order, it will break someone's system at some point if they, for example, always ship the first fact as the latest one. Or even less obvious: the API orders the items
concerning their first digit of the number.

```json
{
  "catfacts": [
    "fact"
    :
    {
      "number": 1,
      "text": "Cats have 230 bones while humans only have 206."
    },
    "fact"
    :
    {
      "number": 10,
      "text": "Abraham Lincoln had 4 cats that lived in the white house with him"
    },
    "fact"
    :
    {
      "number": 2,
      "text": "Cats live longer if they live indoor."
    },
    ...
  ]
}
```

This will also lead to a system or workflow breaking at some point. And it may be unfathomable to its maintainers why that is. The same applies to explicit changes, such as
introducing a new ORM on the backend side that leads to the items being delivered in the inverted order. Funny things can happen now if people try to fix this naively.

## Consequences of Hyrum's Law

The problem should be apparent by now. But what are the implications for software engineers, teams, and consumers?

The most straightforward answer could be: "Don't change anything". But that's ridiculous again. Anyone working in the computer science industry will confirm that "nothing is as
constant as change", and change is, in fact, a good thing, as it's the foundation of any improvement and innovation.

The uncomfortable answer is: we have to take this into account and mind as a software engineer and a consumer of these APIs.

- As the developer, we should **provide as much information explicitly** as possible, most likely in the documentation. This will reduce the number of people that make implicit
  assumptions about the API
- As the consumer, we should **assume as little as possible about APIs** we consume. Of course, there will still be assumptions made, but if you keep in mind that they are
  assumptions, you may save hours debugging your code.
- As a software architect, we should **plan APIs thoroughly to minimize behavioural changes**. New behaviour should belong to new endpoints.

## TLDR;

- **Hyrum's law** basically means: Whatever you deliver, someone is going to use it in a way you never imagined
- Change is necessary and will eventually break someone's system
- Give more explicit information on your interface to reduce the number of implicit assumptions
- Make fewer assumptions about code and, if applicable, test its behavior


Hyrum's Law of Implicit Dependencies


## What does SOLID stand for?

**SOLID is a mnemonic acronym** of 5 acronyms themselves: **SRP** - Single Responsibility Principle, **OCP** - Open-Closed Principle, **LSP** - Liskov Substitution Principle, **
ISP** - Interface Segregation Principle, **DIP** - Dependency Inversion Principle.

These ideas were first mentioned by Robert C. Martin (Uncle Bob) in his
paper _"[Design Principles and Design Patterns](https://fi.ort.edu.uy/innovaportal/file/2032/1/design_principles.pdf)"_. Later, Michael C. Feathers coined the SOLID acronym, which
was then re-used by Uncle Bob in the chapter "_Design Principles_" in his book "Clean Architecture".

## Single Responsibility Principle (SRP)

> There should never be more than one reason for a class to change
>
> SRP: The Single Responsibility Principle - Robert C. Martin

The **single responsibility principle** was coined by Robert C. Martin himself but he also pays credit to Tom DeMarco's concept of cohesion, which he described
in "Structured Analysis and System Specification". It is the first of the SOLID principles and encourages to give classes only **a
single** and definite **reason to change**. If you can think of more than a single reason for the class to change, the class has more than one responsibility. SRP is often confused
with the ["Do one thing"-rule](https://code-specialist.com/blog/do-one-thing/) At a first glance, this concept seems rather abstract and useless, but the example that is
provided in the initial paper is kind of helpful:

![How to violate the Single Responsibility Principle. The Rectangle class has two reasons to change.](https://raw.githubusercontent.com/yannicschroeer/diagrams/3d65e422d6253dc0860b2809d13eb5cbac182247/srp_violation.svg)

Imagine a rectangle class that has the two public methods draw and area. The draw method should return coordinates or any graphical representation. The area method returns the area
of the current rectangle instance. The rectangle class thereby has two responsibilities. It's responsible for calculating the area and responsible to draw itself.

```python
class Rectangle(object):
 
  def __init__(self, height: float, width: float):
    self.height = height
    self.width = width
  
  def draw(self) -> VisualRepresentation:
    return visual_representation(self)
  
  def area()-> float:
    return self.height*self.width
```

If you are confused by the types behind the arguments in the constructor, you should have a look at [Python type hints](https://code-specialist.com/blog/type-hints/). However,
changes in the rectangle class would **affect** the GUI as well as the geometric calculation app. Changes in the rectangle class could potentially break the whole system or lead to
unpredictable bugs. The solution Martin proposes in this specific case is to split the class into two separate ones:

![Proposed solution to comply with the Single Responsibility Principle. A second class GeometricRectangle, implementing the math related methods](https://raw.githubusercontent.com/yannicschroeer/diagrams/46134d246c47b5564fd70f2f4d710a44f411888f/srp_solution.svg)

This separation leads to a **single responsibility** for each the geometric rectangle and the rectangle class. Changes in the draw method now can no longer affect the way they are
calculated.
```python
class GeometricRectangle(object):
  
  def __init__(self, height: float, width: float):
    self.height = height
    self.width = width
    
  def area()-> float:
    return height*width
  

class Rectangle(GeometricRectangle):
  
  def draw(self) -> VisualRepresentation:
    return visual_representation(self)
```

The attentive readers might have noticed that this concept's **strict and mindless usage will also lead to poor software design**. It would help if you never used the single
responsibility theory without proper support of other constructs like modules or facades. Otherwise, the structure proposed by the SRP will cause the code to break into a thousand
confusing pieces

## Open-Closed Principle (OCP)

> Software entities (classes, modules, functions, etc.) should be open for extension but closed for modification.
>
> The Open Closed Principle - Robert C. Martin

Whatever your personal definition of stable code is, this SOLID principle should be part of it. A good software architecture will almost certainly always fulfill the open-closed
rule. Originally, Bertrand Mayer coined this concept in his book "Object-Oriented Software Construction". OCP applies to all kinds of
objects in programming you can imagine. This philosophy's main focus is to allow you to scale your classes and modules without caring about legacy code. To simplify this, we are
just going to look at applying it to classes.

The key message of the **open-closed principle**, in this case, is that your classes should be open for extension but closed for modification. Meaning, once you created your class,
it **shouldn't change** any more. However, it could change by simply creating a child class that thereby **extends its behavior**. Imagine you had a user class that holds a name
and an age.

```python
class User:
  
  def __init__(self, username: str, age: int):
    self.username = username
    self.age = age
  
  def __repr__(self):
    return f"User: {self.username}, {self.age} years old"
```
Now imagine you want to extend this class by an attribute that saves the user's favorite game. A naive solution to this problem would be to simply add an attribute "favorite\_game"
to the user class:

```python
class User:

  def __init__(self, username: str, age: int, favorite_game: Game):
      self.username = username
      self.age = age
      self.favorite_game = favorite_game

  def __repr__(self):
      return f"User: {self.username}, {self.age} years old, favorite game: {self.favorite_game}"    
```


This might work if your system is small or in development. But if you want to change this in a productive system, things are going to break. Not only did the signature change
because the constructor now expects a favorite\_game, but also the _\_\_repr\_\__ method changed and might break things further. This violates Meyer's postulate. A possible
solution to this again could be inheritance:

```python
class User:

  def __init__(self, username: str, age: int):
      self.username = username
      self.age = age

  def __repr__(self):
      return f"User: {self.username}, {self.age} years old"


class Gamer(User):

  def __init__(self, username: str, age: int, favorite_game: Game):
      super().__init__(username, age)
      self.favorite_game = favorite_game
      
  def __repr__(self):
    return f"User: {self.username}, {self.age} years old, favorite game: {self.favorite_game}"     
```

By using this, your **functionality is extended**, but you don't apply any changes to the original class. _Please note: Inheritance isn't always a good solution or a solution at
all, but it is the easiest example to make._ _SOLID is not directly related to inheritance or polymorphism._

## Liskov Substitution Principle (LSP)

> _Subtype Requirement_: Let _**φ(x)**_ be a property provable about objects **_x_** of type _**T**_. Then **_φ(y)_** should be true for objects **_y_** of type **_S_** where _**S**_ is a subtype of _**T**_.
>
> A behavioral notion of subtyping - Barbara Liskov and Jeanette Wing

When you are most familiar with Python or hate maths, the **Liskov substitution principle** might be slightly confusing. This is because it originally was designed for **statically
typed programming languages** like Java, C, or FORTRAN. [Barbara Liskov](http://www.pmg.csail.mit.edu/~liskov/) introduced it in 1987. It is often considered the hardest of the
SOLID principles to understand. Visualizing Liskov's definition leads to the following class diagram:

![Liskov Substistion Principle definition as an UML class diagram](https://raw.githubusercontent.com/yannicschroeer/diagrams/64e744c8f9f3b7f04c9cf5f2cd81c0cd754816d8/liskov.svg)

We have a type **_T_** and a subtype _**S**_, and objects of _**x**_ that are of type **_T_** and objects _**y**_ of type _**S**_. Also, all the elements posses an attribute **_φ_**.
A representation in Python code could look like this:

```python
class T:
  
  def __init__(self, phi: list):
    self.phi = phi
    
class S(T):
  pass

if __name__ == "__main__":
  x = T(phi=["a", "b"])
  y = S(phi=["c", "d"])
```

And this indeed fulfills the Liskov substitution principle. Any object of type _**S**_ could replace its parent class. How could you possibly violate this?

```python
class T:
  
  def __init__(self, phi: list):
    self.phi = phi
    
class S(T):
  
  def __init__(self, phi: str):
    self.phi = phi

if __name__ == "__main__":
  x = T(phi=["a", "b"])
  y = S(phi="c, d")
```

If you look closely, you will see that the subtype S implements its own class attribute phi of type string instead of list. This violates Liskov's theory. You can no longer
replace _**T**_ with objects of type _**S**_. One becomes aware of this concept's meaningfulness when you think about implementing a method "_print\_phis_" that should solely print
all elements in the class attribute phi. Either instances of class _**T**_ or _**S**_ would run into a runtime error or lead to a higher degree of complexity in the "_print\_phis_"
method due to additional conditionals.

## Interface Segregation Principle (ISP)

> Clients should not be forced to depend upon interfaces that they do not use.
>
> The Interface Segregation Principle - Robert C. Martin

ISP or **interface segregation principle** is the SOLID equivalent of **high cohesion**
in [GRASP (General Responsibility Assignment Software Principles)](/grasp/). Thereby it naturally supports loose coupling and
maintainability. The main message behind ISP is that large Interfaces should be split into multiple ones. Functions and classes should not depend on methods they don't use. Look at
the following example for a "fat" interface:

```python
class GeometricInterface(ABC):
  
  @abstractmethod
  def get_area() -> float:
    raise NotImplementedError
    
  @abstractmethod
  def get_diameter() -> float:
    raise NotImplementedError
    
class Square(GeometricInterface):
  
  def __init__(self, height: float, width: float):
    self.height = height
    self.width = width
  
  def get_area():
    return self.height*self.width
  
  def get_diameter() -> float:
    raise NotImplementedError
  
class Circle(GeometricInterface):
  
  def __init__(self, radius: float):
    self.radius = radius
  
  def get_area():
    return self.radius * PI **2
  
  def get_diameter() -> float:
    return self.radius*2
```

Please excuse the far-fetched example, nothing I'm proud of. However, as you can see, the GeometricInterface has two abstract methods of which `get_diameter()` is not used by
its subtype square. Thereby the interface segregation rule is violated. There are many possible solutions to this. We could, for instance, segregate the GeometricInterface into
three Interfaces:

```python
class GeometricInterface(ABC):
  
  @abstractmethod
  def get_area() -> float:
    raise NotImplementedError

class ElipseInterface(GeometricInterface, ABC):
  
  @abstractmethod
  def get_diameter() -> float:
    raise NotImplementedError

class RectangleInterface(GeometricInterface, ABC):
  pass


class Square(Rectangle):

    def __init__(self, height: float, width: float):
        self.height = height
        self.width = width

    def get_area(self):
        return self.height * self.width

class Circle(Elipse):

    def __init__(self, radius: float):
        self.radius = radius

    def get_area(self):
        return self.radius * PI ** 2

    def get_diameter(self) -> float:
        return self.radius * 2
```

## Dependency Inversion Principle (DIP)

> A: High level modules should not depend upon low level Modules. Both should depend upon abstractions.
>
> B: Abstractions should not depend upon details. Details should depend upon abstractions.
>
> The Dependency Inversion Principle - Robert C. Martin

In my opinion, the **dependency inversion principle** is the easiest one to understand. However, it can be the hardest to implement. In essence, the DIP states that modules should
not rely on modules that belong to a subordinate concept and should not rely on generalizations. A quick piece of code that is often referenced:

```python
class LightBulb:
  
  def __init__(self, initial_state: bool=False):
    self.power = initial_state
    
  def turn_on(self):
    self.power = True
    
  def turn_off(self):
    self.power = False

class Switch:
  
  def __init__(self, light_bulb: LightBulb, pressed: bool=False):
    self.light_bulb = light_bulb
    self.pressed = pressed
    
  def toggle(self):
    self.pressed = not self.pressed # Toggle
    if self.pressed:
      self.light_bulb.turn_on()
    else:
      self.light_bulb.turn_off()
```

The **DIP violation** here is that a switch is a concept that is logically in a layer above the light bulb, and the switch relies on it. This will lead to poor extensibility or
even circular imports that prevent the program from being interpreted or compiled.

![Violating the Dependency Inversion Principle. The controller module relies on the device module instead vice versa](https://raw.githubusercontent.com/yannicschroeer/diagrams/fe1c96e84d8ef0701c95399a796805d6c938cf46/dip_violation.svg)

Instead of the light bulb telling the switch how the bulb should be handled, the switch should tell the light bulb how to implement it. The naive approach would be to define an
interface that tells the light bulb how it should behave to be used with a switch.

```python
class Device(ABC):
  power: boolean
    
  def __init__(self, initial_state: bool=False):
    self.power = initial_state
    
  def turn_on(self):
    raise NotImplementedError
  
  def turn_off(self):
    raise NotImplementedError
    
class Switch:
  
  def __init__(self, device: Device, pressed: bool=False):
    self.device = device
    self.pressed = pressed
    
  def toggle(self):
    self.pressed = not self.pressed # Toggle
    if self.pressed:
      self.device.turn_on()
    else:
      self.device.turn_off()  

class LightBulb(Device):
  
  def turn_on(self):
    self.power = True
    
  def turn_off(self):
    self.power = False
```

Visualized as a class diagram, this source code would lead to the object-oriented design:

![Dependency Inversion Principle in an UML class diagram. A Device Interface as the solution for the switch not to rely on the light bulb](https://raw.githubusercontent.com/yannicschroeer/diagrams/14b34110edd15e8d5325a5e28c8e61ca91df3670/dip_solution.svg)

**The dependency has been inverted**. Instead of the switch relying on the light bulb, the light bulb now relies on an interface in a higher module. Also, both rely on
abstractions, as required by the **DIP**. Last but not least, we also fulfilled the requirement "Abstractions should not depend upon details. Details should depend upon
abstractions" - The details of how the device behaves rely on the abstraction (Device interface).

## Conclusion

As with all guidelines, it's not recommended to follow the SOLID principles blindly either. Principles, patterns, and guidelines can easily turn into anti-patterns and lead to new
problems. Sometimes it not preventable to violation some of these. Our job as a developer is to create code and programs that
are [easy to read and thereby easy to maintain and extend](https://code-specialist.com/code-principles/kiss-principle/). Robustness and correctness are more or less nice side
effects of this approach. These are great guidelines, but that's what they are, guidelines. Specific problems need specific solutions. All principles and patterns can offer are
suggestions and best practices.

If you support my statement or are of a completely different opinion I would love to hear about it in the comments section.

## References

- [SRP: The Single Responsibility Principle - objectmentor.com, 02.02.2015](https://web.archive.org/web/20150202200348/http://www.objectmentor.com/resources/articles/srp.pdf)
- [The Open-Closed Principle - objectmentor.com, 02.02.2015](https://web.archive.org/web/20150212141727/http://www.objectmentor.com:80/resources/articles/ocp.pdf)
- [The Liskov Substitution Principle - objectmentor.com 11.02.2015](https://web.archive.org/web/20150211000751/http://www.objectmentor.com/resources/articles/lsp.pdf)
- [The Interface Segregation Principle - objectmentor.com 10.02.2015](https://web.archive.org/web/20150210224002/http://www.objectmentor.com/resources/articles/isp.pdf)
- [The Dependency Inversion Principle: objectmentor.com 10.02.2015](https://web.archive.org/web/20150210213530/http://www.objectmentor.com/resources/articles/dip.pdf)
- [Design Principles and Design Patterns](https://cds.cern.ch/record/1419478/files/0135974445_TOC.pdf)
- [Barbara Liskov, Professor at MIT](http://www.pmg.csail.mit.edu/~liskov/)
- [GRASP: General Responsibility Assignment Software Principles, code-specialist.com](https://code-specialist.com/blog/grasp/)
- [KISS Principle code-specialist.com](https://code-specialist.com/blog/kiss-principle/)
- [Do one things, code-specialist.com](https://code-specialist.com/blog/do-one-thing/)
- [Python type hints, code-specialist.com](https://code-specialist.com/python-type-hints/)

SOLID Principles with Python Code Examples


This error is often encountered by Linux users trying to install Python packages
from the requirements.txt. This issue occurs only while using a virtual
environment, specifically the virtualenv module of Python. When someone used
`pip freeze > requirements.txt` to write the pip freeze's output requirements to
a file, a bug of that library leads to this error. It writes the requirement
`pkg-resources==0.0.0` to the file, this will lead to the described error. There
are multiple ways to fix this, depending on whether you are the maintainer of
the requirements or merely a user. As the maintainer of the repository, we
strongly suggest you switch from virtualenv to venv. Since Python 3.3, the venv
module is a built-in module. To change your virtual environment, you may follow
the steps below.

<howto>
  <what>How to fix pkg-resources==0.0.0</what>
  <minutes>5</minutes>
  <description>
    No matching distribution found for pkg-resources==0.0.0
  </description>
  <tool>Python: pip</tool>
  <tool>Python: venv</tool>
  <tool>UNIX based OS like MacOS or Linux</tool>
  <step>
    <name>Clear the dependencies</name>
    <substep>
      Write your dependencies and filter out the pkg-resources `pip freeze |
      grep --invert-match pkg-resources > requirements.txt`
    </substep>
    <substep>
      Remove the virtualenv, Assuming it is stored in a folder called “venv” `rm
      -rf venv`
    </substep>
  </step>
  <step>
    <name>Create a new virtual environment</name>
    <substep>Create a clean venv `python3 -m venv venv`</substep>
    <substep>Activate it `source venv/bin/activate`</substep>
    <substep>Reinstall requirements `pip install -r requirements.txt`</substep>
  </step>
</howto>

**Done!** The error should no longer occur.

If you are not the maintainer of the repository we suggest you recommend this
post to the maintainer. While the maintainer is working on fixing this, you can
install the dependencies similarly to the freeze command above, by filtering the
packages for it:

```shell
cat requirements.txt | grep --invert-match pkg-resources | xargs -n 1 pip install
```

### Explanation

cat will output the content of the requirements, grep will filter
it for pkg-resources==0.0.0, and xargs -n 1 will execute pip install for each
line returned by the preceding commands.

We hope this solves your issue. If you encounter any problems, please don't
hesitate to leave a comment. We will try to help you as soon as possible!

## What is pkg-resources?

> The pkg_resources module distributed with setuptools provides an API for
> Python libraries to access their resource files, and for extensible
> applications and frameworks to automatically discover plugins. It also
> provides runtime support for using C extensions that are inside zipfile-format
> eggs, support for merging packages that have separately-distributed modules or
> subpackages, and APIs for managing Python's current “working set” of active
> packages.
> 
> Python Setuptools, https://setuptools.pypa.io/en/latest/pkg_resources.html

In essence: pkg-resources is a development dependency of packages which use
setuptools. Usually it should not be part of any actual packages. This
is clearly a bug in `virtualenv`.


No matching distribution found for pkg-resources==0.0.0


## The Decorator Principle

First of all, I will give you an introduction to the **Decorator Principle**.
This principle is not limited to Python, you can use it in every other language
as well. However, Python has some built-in syntax for decorators and the
decorator principle fits the language design as well. Understanding this
principle will make it much easier to familiarize yourself with decorators and
understand their functionality.

To explain the decorator principle, I will take the **menu of a restaurant** as
an example. We assume that every dish on the menu has a corresponding function
or class in our code. Whether you use classes of functions with decorators may
vary based on the programming language and architecture you are using, but the
core concept of the pattern stays the same. To keep it simple, we will only use
functions for this example.

At first, the menu only offers a burger and a steak. So we just have two
functions handling both dishes:

![Basic menu-functions for the restaurant example](/assets/blog/posts/decorators/decorators-burger-steak.svg)

But the restaurant has a problem: there are customers who want custom versions
of the dishes. Maybe someone wants another sauce or extra ingredients. To cover
all these cases, we need functions for every single version of the dishes. As
you can imagine, this is an absolutely terrible idea and design as it is
inflexible and violates a lot of code principles like
[KISS](https://code-specialist.com/code-principles/kiss-principle/) or
[DRY](https://code-specialist.com/code-principles/dry-principle/).

![Bad and messy design to implement custom modifications](/assets/blog/posts/decorators/decorators-variety.svg)

As you can see, it gets really messy. And this is where decorators come in! They
provide a simple and flexible way to customize functions and classes by adding
extra behavior. In our example, we create a decorator for each customization,
which takes the original function of the dish and adds the special wish of the
customer. By using this, we only have to code the special customer wishes once
and separated!

![Using decorators for dish modifications](/assets/blog/posts/decorators/decorators-burger-steak2.svg)

Then we can decorate our dish function according to the individual wishes of the
customer. A function call with some decorators can then look like this:

![Example call stack for a custom burger](/assets/blog/posts/decorators/decorators-burger.png)

As the figure shows, every decorator calls the previous function and appends
some additional content to it. Therefore, decorators depend on the function they
are called on.

## Implementation in Python

In Python, decorators are basically callables. A callable in Python can be a
function or class, in general any object that you can call. However, the easiest
and most common way to implement decorators in Python is to use functions. The
special thing about them is that they take a function as an input argument and
return a function as well.

```python
def decorator(function):
    
    # modify the function
    # ...
    
    return modified_function
```

Make sure to return only the function and don't call it like
_modified_function_() for instance instead.

Since we want to create a decorator, we want to modify the behavior of the
function given as input. A common way to do so is to create a new function that
wraps the input function. In this wrapper function we can call the input
function and do some other stuff too:

```python
def double_result(function):
    
    def wrapper(*args, **kwargs):
        result = function(*args, **kwargs)
        return result * 2
    
    return wrapper
```

It is important to note that the wrapper function takes _\*args_ and
_\*\*kwargs_ as arguments. If you are not familiar with this syntax, they take
the normal and the keyword arguments passed into the function and put it into a
list and a dictionary. Inside the wrapper, we reverse this by unpacking the
arguments in the function call. In this way, we ensure that we get all the
parameters that were passed to the original function and that we pass them on
correctly.

## Applying Decorators

To apply our decorator, we just use the **@** syntax and attach it to a
function. When the Python interpreter loads the code, it will call our decorator
and pass the function automatically as the argument. Then our function gets
**replaced** by the returned one from the decorator. With this in mind, you may
realize that the syntax for decorators is just syntactic sugar and simply
replace the function with a new one. So both of the following examples work the
same:

```python
from my_decorators import double_result

def calculate_answer(number: int) -> int:
    return number - number

# replace function with decorated one
calculate_answer = double_result(calculate_answer)

# the code above is equal to:

@double_result
def calculate_answer(number: int) -> int:
    return 1337 + number * 42
```

For this example, the output of the function **without** the decorator is the
following:

```
>>> calculate_answer(7)
1631
```

If we append the **double_result** decorator to the function, we can see that
the result is doubled.

```
>>> calculate_answer(7)
3262
```

## Overview: Advanced techniques for Python Decorators

Decorators are very powerful, and with the basic implementation of the previous
chapter, you are actually well served. Nevertheless, there are some advanced
techniques that can be valuable in some cases. The following is a brief overview
of what you can do. However, if you want to read more about specific topics, let
me know in the comments!

**Using functools.wraps**

Decorators replace a function with another one, but what if we want the new
function to have the attributes of the old one? If we take a closer look to the
decorated function of the previous example, the name in the code stays the same,
but the actual function name changed!

```
>>> calculate_answer.__name__
'wrapper'
```

In most the cases this should not matter, but if it does, we can use the
**wraps** decorator from the [functools
module](https://docs.python.org/3/library/functools.html). This will copy the
function attributes to the wrapper and prevent weird name confusion.

```python
from functools import wraps


def double_result(function):
    
    @wraps(function)
    def wrapper(*args, **kwargs):
        result = function(*args, **kwargs)
        return result + 42
    
    return wrapper
```

**Decorators for Classes**

Not only can you decorate functions, you can also decorate classes . This is
rather used in other object-oriented languages like Java, but there are
use-cases in Python as well. You can implement the Singleton pattern with
class-decorators for instance.

**Decorators with Arguments**

Maybe you want to customize your decorators too, so you don't have to write
multiple ones. In general, this is quite easy, but it also adds a new layer of
complexity to your decorator. The solution is to simply wrap your decorator in
another function! This function takes the arguments you need and returns the
decorator. With the decorator syntax it's possible to call this function with
your arguments and use the returned decorator instead. Python 3.9 also
introduced more syntactic flexibility with the decorator syntax, you can take a
look in our corresponding
[post](https://code-specialist.com/python/new-syntax-features-in-python-3-9/) if
you are interested.

**Stateful Decorators**

A decorator can also handle state. There are multiple possible solutions to
implement this. For instance, you can use instance objects of classes as
decorators by implementing the \_\_call\_\_ method. However, since functions are
also objects, you can simply add attributes to functions in order to add state.

I hope this post gave you a good introduction to decorators in Python and you
learned how to create them on your own. If you have any thoughts, feedback or
questions feel free to leave a comment below!


Decorators in Python


## The Concept Behind Bubble Sort

To remind yourself what the concept behind bubble sort -not especially in Python- is, it might help look into its naming. Think of the analogy like that: you have some random air bubbles below the surface. The larger the bubble is, the higher its buoyancy speed is to rise to the top, and that's it.

The idea behind the algorithm is to compare only two elements simultaneously and iterate over the whole list without going back. In the first step, you look at the first element in the list, in the second step at the second element, and in the last step, you look at the last element. You start at the beginning of the list and compare the first with the second item. If the first item is larger, you switch positions and go on.

```
Initial list: 	[4, 3, 2, 1]
First step: 	[4, 3, 2, 1] → 4 > 3? True → [3, 4, 2, 1]
```

In the next step, you would compare the second element (which is now the item that was the first one) with the third item and so on.

```
Second step: 	[3, 4, 2, 1] → 4 > 2 ? True → [3, 2, 4, 1]
```

The Algorithm has done its job if, in one run, no numbers have been switched.

```
...
Before last step: 	[2, 1, 3, 4] → [1, 2, 3, 4] → 1 change
last step: 		[1, 2, 3, 4]   0 changes → sorted
```

## Implementing the Algorithm

An actual implementation of bubble sort in Python might look like this:

```python
def bubble_sort(unsorted_list: List) -> List:
    number_of_changes = 0
    for index in range(len(unsorted_list) - 1):
        if unsorted_list[index] > unsorted_list[index + 1]:
            unsorted_list[index], unsorted_list[index + 1] = unsorted_list[index + 1], unsorted_list[index]  # swap
            number_of_changes += 1
    if number_of_changes != 0:
        return bubble_sort(unsorted_list)
    return unsorted_list
```

If you have problems understanding the 5th line

```python
 unsorted_list[index], unsorted_list[index + 1] = unsorted_list[index + 1], unsorted_list[index]  # swap
```

That is called an in-place swap. That's possible in Python because the interpreter evaluates expression strictly from left to right. In the case of an assignment, the interpreter will first read the assigning values and then assign them. For example:

```python
a = 5
b = 6
print(a, b)
a, b = b, a
print(a, b)

out1 >> 5, 6
out2 >> 6, 5
```

So the values `a` and `b` effectively swap positions as `a` gets assigned `b` and `b` gets assigned `a`, **after** they both have been read. You may read more on this in the [Python docs](https://docs.python.org/3/reference/expressions.html#evaluation-order).

Note that I used the concept of recursion (the function calls itself under given circumstances). Using recursion is not necessary here, and if the list is long enough, the system's max recursion depth might terminate your program. You may get to know your systems max recursion depth by using:

```python
import sys
sys.getrecursionlimit()
```

You might change this constant to 3000 for example:

```python
import sys
sys.setrecursionlimit(3000)
```

However, I do not recommend doing so. Recursive functions are often way faster than loops, but their memory usage is often immensely. The max recursion limit is set with the reason to prevent infinite loops and extensive memory-usage. Thereby, a non-recursive version might look like this:

```python
def bubble_sort(unsorted_list: List) -> List:
    while True:
      number_of_changes = 0
      for index in range(len(unsorted_list) - 1):
          if unsorted_list[index] > unsorted_list[index + 1]:
              unsorted_list[index], unsorted_list[index + 1] = unsorted_list[index + 1], unsorted_list[index]  # swap
              number_of_changes += 1
      if number_of_changes == 0:
          return unsorted_list
```

## Time Complexity in Big-O-Notation

A common thing in computer science is evaluating algorithms' time complexity for a certain problem to decide which algorithm fits the best. However, it's important to know, that the Big-O Notation only makes statements about the **growth in relation to the input-parameter, not the time it takes to compute**. Take these two function for example:

<math>f_1(x)=x^2 f_2(x)=3x^2</math>

Both functions are considered quadratic in growth in relation to their input, even though the second function will always return a larger value.

Theoretically, there's also a memory complexity to be considered. Still, as most of the time, memory isn't a problem today, computer scientists tend only to give the time complexity of algorithms. The usual procedure is to evaluate the best- and the worst-case scenario of the algorithm concerning a given input of length `n`.

Given this knowledge, let's try to assume the best possible runtime for this algorithm. The best possible scenario would be the whole list is already sorted, then we would iterate once over the **list of length `n`** and return it, as we detected `zero changes`. The runtime in relation to `n` would be `n-1`, as we compare each element with the next until we are at the penultimate element comparing it with the last one.

You might wonder what happens to the operations performed each time the algorithm is called - let's call that factor `k` - a rule for big-o-notation that then applies is that constant factors are ignored.

<math>
  \mathcal{'{'}O{'}'}(k*(n-1))=\mathcal{'{'}O{'}'}(kn-k)=\mathcal{'{'}O{'}'}(n)
</math>

Even though we have `n` calls with `k` operations, `k` is a constant factor and is thereby ignored.

<math>
  BestCase_{'{'}bubblesort{'}'} = \mathcal{'{'}O{'}'}(n)
</math>

The worst-case scenario, in contrast, would be the list totally reverse-sorted. Then would call the algorithm `n-1` times as we only shift one element each time we call the algorithm. This would effectively lead to:

<math>n(n-1) = n^2-n</math>

`n^2-n` calls. Another rule for the big-o-notation is that only the term with the largest exponent is relevant. Thereby the worst-case scenario would be described as `n^2`.

<math>
  WorstCase_{'{'}bubblesort{'}'}= \mathcal{'{'}O{'}'}(n^2)
</math>

## Optimizing Bubble Sort in Python

Suppose you read closely and know about bubble sort, you probably found the implementation in Python not to be optimal. That's because I wanted to keep it simple to understand. In the following section, I want to describe how we can optimize the algorithm drastically with a slight modification. If you don't want to optimize the algorithm or think that math is extraordinarily boring, I urge you to skip this section.

If you think about the algorithm extensively you might notice that in the first iteration over the whole list, the last element will be the largest. In the second iteration over the list, the second-last item will be the second-largest, and so on.

Using this knowledge we can cut the list in length by the last item each time we iterate over the list as it is already sorted. If we formularize this regarding the worst case, we get to the following formula:

<math>(n-1)+(n-2)...+(n-(n-2)) + (n-(n-1)) = (n-1)+(n-2)...+2 + 1</math>


In the first iteration, we have n-1 steps, then n-2, and so on, until we have only one element left or no changes. So, for example if we have 8 elements, the worst case will be

<math>7+6+5+4+3+2+1 = 28</math>

28 steps. So the maximum number of steps is exactly the sum of all numbers from 1 to `n-1`. Something that comes into play now is the Gaussian sum.

<math>
  \displaystyle\sum_{'{'}i=1{'}'}^n=\frac{'{'}n(n+1){'}'}
  {'{'}2{'}'}
</math>

Since we only need the sum of numbers from 0 to `n-1`, we have to modify the formula slightly

<math>
  \frac{'{'}(n-1)((n-1)+1){'}'}
  {'{'}2{'}'} =\frac{'{'}(n-1)(n){'}'}
  {'{'}2{'}'} =\frac{'{'}n(n-1){'}'}
  {'{'}2{'}'}
</math>

This, in turn can be rewritten to

<math>
  \frac{'{'}n(n-1){'}'}
  {'{'}2{'}'} =\frac{'{'}n^2-n{'}'}
  {'{'}2{'}'}= \frac{'{'}1{'}'}
  {'{'}2{'}'}(n^2-n)
</math>

As `1/2` is again a constant factor, the time complexity again would be considered:

<math>
  \mathcal{'{'}O{'}'}(n^2)
</math>

This is confusing as the runtime surely will be better. You have to remind yourself that the Big O Notation only makes statements about the growth, not the computation time. The optimized (recursive) code could look like this:

```python
def optimized_bubble_sort(unsorted_list: List) -> List:
    number_of_changes = 0
    for index in range(len(unsorted_list) - 1):
        if unsorted_list[index] > unsorted_list[index + 1]:
            unsorted_list[index], unsorted_list[index + 1] = unsorted_list[index + 1], unsorted_list[index]  # swap
            number_of_changes += 1
    if number_of_changes != 0:
        return bubble_sort(unsorted_list[:-1]) + [unsorted_list[-1]]
    return unsorted_list
```

In row 8 you see, that instead of passing the whole list, we pass only the list minus the last element to the function call and append the last element. To prove to you that the optimized version is still considerably faster, I plotted the average runtimes (100 runs) of bubble sort and the optimized version with increasing list sizes from 10 to 1000:

![Bubble Sort Runtime Plot, 100 runs with increasing list sizes from 10 to 1000 elements](/assets/blog/posts/bubblesort/bubblesort-optimized-graph.webp)

## Visualizing Bubble Sort in Python

The following animation (gif) was created using only Python and matplotlib (pyplot):

![Bubble Sort Algorithm Visualization](/assets/blog/posts/bubblesort/animated-bubblesort.gif)

To reproduce this with any kind of sorting algorithm, you could use a variation of my code:

```python
import os
import pathlib
from typing import List

import imageio
import matplotlib.pyplot as plt
from numpy.random.mtrand import rand
from io import BytesIO

output_path = pathlib.Path(__file__).parent # Directory of execution
frames = list()  # Global list of the frames (file paths)
changes = 0  # Global variable to track the frame number


def bubble_sort(unsorted_list: List) -> List:
    number_of_changes = 0  # Number of changes in the current iteration
    global changes  # Visualization only: Global number of changes → Frame number
    for index in range(len(unsorted_list) - 1):
        if unsorted_list[index] > unsorted_list[index + 1]:
            unsorted_list[index], unsorted_list[index + 1] = unsorted_list[index + 1], unsorted_list[index]  # swap
            visuzalize_step(unsorted_list)  # Visualization only: create a frame and append the path
            number_of_changes += 1
            changes += 1  # Visualization only: increment the global frame number
    if number_of_changes != 0:
        return bubble_sort(unsorted_list)
    create_gif(frames)  # Visualization only: If the list is sorted, create the gif, using the frames
    return unsorted_list


def visuzalize_step(input_list: List):
    global frames  # Ensure the global usage of the variable
    bytes_object = BytesIO()
    plt.figure(facecolor="#111827")  # Background Color
    plt.bar(range(len(input_list)), input_list, facecolor=("#0c4a6e"))  # Create a bar chart using the current values of input_list
    plt.tight_layout()  # Make it use tight layout to reduce borders and margins
    plt.axis('off')  # Remove axis
    plt.savefig(bytes_object)  # Save the plot in-memory
    plt.close("all")  # Ensure everything from pyplot gets closed, to save memory
    frames.append(bytes_object)  # Append the frame path to the frame list


def create_gif(frames: List[BytesIO]):
    animation_path = os.path.join(output_path, "animation.gif")  # Open a new file
    with imageio.get_writer(animation_path, mode='I', duration=1 / 30) as writer:
        # Write images where each is display for 1/30s → 30 frames per second
        for frame in frames:  # Iterate over the frames
            frame = imageio.imread(frame)  # Convert to ndarray
            writer.append_data(frame)  # Write the image to the stream


if __name__ == "__main__":
    unsorted_list = rand(25)  # Create 30 random float numbers in a list
    bubble_sort(unsorted_list)
```

To briefly describe the code: The idea behind the visualization was to interpret each number in a given list as a bar of a bar chart. All that was left to do was take a picture of the current number order after each change. Then merge all the pictures to create an animated algorithm's illusion. I tried to follow all the [clean code rules](https://code-specialist.com/tag/clean-code/) regarding the animation code, but don't hesitate to comment on anything unclear. Code isn't always as self-explanatory as one might think...


Bubble Sort, Big O and visualization with Python


## About GRASP

This collection of **object-oriented design rules** goes back to **Craig
Larman** and his book **Applying UML and Patterns** from 2004. Larman didn't
invent any of these principles and ideas. He simply collected them.

**GRASP** is a pretty fancy name, but it somehow feels more like he wanted GRASP
as the acronym and linked random words to reach it. Even though it is about to
become an oldie, most rules collected under the GRASP banner are still helpful,
and they should be essential to every developer's toolkit.

The **General Responsibility Assignment Software Principles** list nine
principles:

1. Controller
2. Creator
3. Indirection
4. Information Expert
5. Low Coupling
6. High Cohesion
7. Polymorphism
8. Protected Variations
9. Pure Fabrication.

Though they are tailored to object-oriented design, some also apply for general
software development as well.

I write about some things that are not from Larman's book but my own opinion and
personal note. If you don't want the unaltered version, I suggest you read the
original book: _Applying UML and Patterns: An Introduction to Object-Oriented
Analysis and Design and Iterative Development_

## Controller

The controller pattern is a common tool in object-oriented software design.
Though most people might only associate the controller with the ancient
architecture of the MVC (Model-View-Controller - Which is far too often used for
cases it absolutely doesn't fit, but that's a story for a different day), the
controller is a construct of itself.

As the name implies, the controller's work is to 'control' any events that are
not directly related to the user interface. But controlling does not mean
implementing. The controller consists of barely any logic. It is merely a
mediator between the presentation layer and the (core) logic.

For example, if we imagine a simple web-shop application with the use case of
"User X bought item A"; The controller's job would be to receive the signal of
the pressed button from the UI and then run the necessary functions in the
correct order. In the example at hand that could be to certify the payment and
then initialize the shipment of the item.

Applying the principle of a controller can hugely improve your software's
lifetime as it naturally creates a resilient layer regarding the
interchangeability of code. If your UI or logic (unlikely, but possible), you
can adjust your mappings instead of having to rewrite large chunks of code.

Also, it becomes incredibly easier to add other layers, such as an app at the
overlaying presentation layer that uses a different UI but wants the same
responses.

You can also think of a controller as a driver. It knows both parts it connects
with one another but it acts merely as a broker. It consists of just a few and
essential parts of code.

## Creator

The creator is another pattern, but to me, it's more like an abstract idea than
a real pattern. I must admit I rarely use this pattern intentionally.

A creator is a class that is responsible for creating instances of objects.
Larman defines the following cases as _B is the creator of A_:

- B _aggregates_ A objects
- B _contains_ A objects
- B _records_ instances of A objects
- B _closely uses_ A Objects
- B _has the initializing_ data that will be passed to A when it is created

So an example for a creator could be a library that _contains_ books. In that
case, the library would be the creator of books, even though this sounds
syntactically weird as natural language.

## Indirection

Indirection isn't a pattern but an idea. It is also of no use of its own, only
in combination with other ideas such as low coupling. The goal behind
indirection is to avoid direct coupling.

To pick up an earlier example the controller for example is a kind of
indirection between the UI and the logic. Instead of coupling the UI directly to
the logic, the controller decouples this layer and thereby comes with all the
advantages of indirection.

## Information Expert

Another principle that falls into that area is the information expert. Sometimes
it is just called the expert or the expert principle. The problem that should be
solved with the information expert is the delegation of responsibilities. The
information expert is a class that contains all the information necessary to
decide where responsibilities should be delegated to.

You identify the information expert by analyzing which information is required
to fulfill a certain job and determining where the most of this information is
stored.

## Low Coupling

Low or loose coupling is the idea of creating very few links between modules. In
GRASP it is an inter-module concept. Meaning it describes the flow between
different modules, not the flow inside the modules.

Together with cohesion, it is the main reason upon which you make global design
decisions such as "Where do we pack this module or function to?".

Low coupling is another concept that supports interchangeability. By making very
few dependencies, very few code must be changed in case of a change.

## High Cohesion

in GRASP high (functional) cohesion is an intra-module concept. Meaning it
describes the flow inside a certain module not the flow between modules.

The main reason behind high cohesion is the idea of reducing complexity. Instead
of building large classes that have many functions that have few in common and
are hard to maintain, the goal should be to create classes that fit exactly
their defined purpose.

It's kind of hard to understand these abstract ideas but I also wasn't able to
come up with a simple example, which is why I will write an extra article on the
topic of low cohesion combined with high cohesion that is more detailed.

## Polymorphism

Polymorphism: `πολυ` (_polús_)= many/multi, `μορφή` (_morphé_) = shape/form,
`ισμός` (_ismós_) = imitation of.

Meaning, Polymorphism could be frankly translated as "something that imitates
many forms". And that might be a concise but useless explanation to someone who
has never heard of polymorphism.

Anyone who ever took a programming class is most likely familiar with
polymorphism. But, it can be a tricky question to define it sensefully.

The idea again is to reduce complexity by imagining that objects follow simple
and similar rules. Consider the following example: You have three objects A, B
and C. B has the same methods and attributes but has an additional method X. C
has the same methods and attributes as B, but an additional method Y.

The non-developer approach to this problem would be: "Great, as you said, we
have 3 objects, so we have A, B, and C. Problem solved. You little stinky
moron.". Yes, non-developers are insanely evil creatures.

![Three objects without polymorphism](/assets/blog/posts/grasp/polymorphism_wrong.svg)

```python
class A:
    attribute_a: str
    attribute_b: int
    attribute_c: float

    def being_useless(self):
        print("I am being of no use.")


class B:
    attribute_a: str
    attribute_b: int
    attribute_c: float

    def being_useless(self):
        print("I am being of no use.")

    def X(self):
        print("I am Batman.")


class C:
    attribute_a: str
    attribute_b: int
    attribute_c: float

    def being_useless(self):
        print("I am being of no use.")

    def X(self):
        print("I am Batman.")

    def Y(self):
        print("I am expensive to my parents.")
```

But as a programmer, you instinctively know that we need the concept of
inheritance here. A is an object, B is an object that inherits from A, and C is
an object that inherits from B. **Caution: Inheritance is not polymorphism** -
inheritance is an application that is allowed due to polymorphism.

![Simple polymorphism example - Venn
diagram](/assets/blog/posts/grasp/polymorphism_correct.svg)

```python
class A:
    attribute_a: str
    attribute_b: int
    attribute_c: float

    def being_useless(self):
        print("I am being of no use.")


class B(A):

    def X(self):
        print("I am Batman.")


class C(B):

    def Y(self):
        print("I am expensive to my parents.")
```

To wrap it up: polymorphism is the concept of disassembling objects and ideas
into their most atomic elements and abstracting their commonalities to build
objects that can act like they were others. Not only to reduce complexity but
also to avoid repetitions.

## Protected Variations

The protected variations pattern is a pattern used to create a stable
environment around an unstable problem. You wrap the functionality of a certain
element (class, interfaces, whatever) with an interface to then create multiple
implementations by using the concept of polymorphism. Thereby you are able to
catch specific instabilities with specific handlers.

An example of the protected variations pattern is working with sensory data,
such as a DHT22, a common temperature and humidity sensor often used for
Raspberry Pi or Arduino). The sensor is doing its job pretty well, but sometimes
it will say the temperature just rose by 200 celsius or won't return any data at
all. These are cases you should catch using the protected variations pattern to
avoid false behavior of your program.

## Pure Fabrication

To reach low coupling and high cohesion, it is sometimes necessary to have pure
fabrication code and classes. What is meant by that is that this is code that
does not solve a certain real-world problem besides ensuring low coupling and
high cohesion. This is often achieved by using factor classes.

## Some last words

Often when I look into programming principles, I feel mildly overwhelmed and
discouraged. That's in part because some of these concepts are very hard to
**grasp** if you never heard of them or got no practical use case in mind. But
with a growing set of programming principles and best practices, you will see
that all these principles refer to the same key statements and follow the same
rules.

Unfortunately, in my opinion, there's also no substitution for learning it the
hard way. Memorizing merely the key statements will eventually lead to greater
school and university scores, but it will still be useless incoherent
information without any practical use. Thereby, my advice is to learn as much as
possible about code and its principles but also get as much hands-on experience
as possible to abstract the key statements by yourself.

If you liked this post or are of any different opinion regarding any of the
written, please let us know via mail or comment. Happy coding!


GRASP - General Responsibility Assignment Software Principles

Visualize K Means Algorithm in Python


## What can Python Dictionaries store?

Dictionaries are datastructures containing key-value pairs (like a Map in Java), as you may 
know already. Besides that, dictionaries in Python are really special compared to similar 
data structures in other programming languages. They benefit from the fact that Python has 
dynamic typing, so you can use keys and values of different types in the same dictionary 
without any problems. You can use any object as value and **every hashable object** as key, 
which includes functions, classes, and even datatypes! You can find out whether an object is 
hashable just by passing it into the built-in **hash()** function.

```python
def my_function():
    pass  # do something here

class MyClass:
    pass  # here happens some cool shit

example_dict = {
    my_function: True,
    False: 4j,
    int: 'you can use even types as key',
    MyClass: lambda x: pow(x, 2),
}
```

As you can see, dictionaries are really powerful and enable you to write creative and smart 
code with them. However, there are some limitations to the key values. For instance, mutable 
objects like lists are not hashable, so you can't use them as a key.

## Looping over Dictionaries

In Python, it is widespread to loop over lists, tuples, or other simple iterables, but what 
about dictionaries? In fact, you can loop over dictionaries the same way you loop over lists. 
But you may notice that you only get the keys when you try it out.

Normally you also want the values when iterating over a dictionary. The easiest approach to 
do this would be the following:

```python
dictionary = {'key_1': 'value_1', 'key_2': 'value_2', 'key_3': 'value_3'}

for key in dictionary:
    value = dictionary[key]
    print(key, value)
```

It works, even though it's not really beautiful and pythonic. Luckily, there is a better solution 
that makes the code a bit prettier. Every dictionary has the **items()** method that returns an 
iterator providing a tuple containing key _and_ value pairs.

```python
dictionary = {'key_1': 'value_1', 'key_2': 'value_2', 'key_3': 'value_3'}

for key, value in dictionary.items():
    print(key, value)
```

Additionally, if you just want the dictionary values, you can use the **values()** method:

```python
dictionary = {'key_1': 'value_1', 'key_2': 'value_2', 'key_3': 'value_3'}

for value in dictionary.values():
    print(value)
```

## Unpacking Operator

In my opinion, the unpacking operator is probably the smartest trick when it comes to writing cool 
and simple code with dictionaries in Python. In short, the unpacking operator **\*\*** unpacks 
dictionaries to keyword arguments. Another unpacking operator **\*** which does nearly the same 
thing for other iterables like tuples or lists.

You also may have seen the double stars in some function parameters, where they do exactly the 
opposite: they wrap all (not defined) keyword arguments into a dictionary!

However, let's see what you can do with the unpacking operator:

```python
import requests


request_arguments = {
    'url': 'any url',
    'params': {'param': 'value'},
    'data': {'some': 'payload data'},
    'auth': 'API-KEY',
    'allow_redirects': False,
    'timeout': 30,
    'headers': {'content-type': 'application/json'}
}

response = requests.post(**request_arguments)
```

Instead of writing all the keyword arguments directly in the function call, we can simply store them 
in a dictionary and unpack them in the function call. This makes the code more flexible and readable. 
Furthermore, we could go one step further and wrap this code in another function that can take any 
arguments and pass them to the _post_ function! This technique can be particularly useful in larger 
projects where you want to generalize most of your functionality and write a lot of wrapper functions.

Besides that, there are many other cases to use the unpacking operator. Try it out and share your use 
cases in the comments!

## Dictionary Comprehension

You may have heard of list comprehensions before, but this trick works for dictionaries too!

The syntax is almost the same as for list comprehensions, the only difference is that you need two 
values on the left side: the key and the value, separated by a colon. The middle part represents the 
current value of the iterable which is on the right, exactly like list comprehensions or a classic for 
loop.

```python
squared_values = {n: n**2 for n in range(5)}

assert squared_values == {
  0: 0, 
  1: 1, 
  2: 4, 
  3: 9, 
  4: 16,
}
```

This example shows a dictionary that stores the numbers from 0 to 99 as keys and the 
corresponding squared numbers as values. Sure, you could create much more complex dictionaries with this 
approach. However, always remember that others are reading this code too. To make your code more readable, 
it is often best to write a classic for loop. For this reason, you should only use dictionary comprehension 
in simple cases where it is clear what is happening.

## Sorting a Dictionary

In general, the dictionary is not meant to be a sorted data structure. However, dictionaries keep the 
insertion order by default since Python version 3.6. Although dictionaries have an order based on the insertions,
Python has no built-in way to sort them (like lists, for instance).

Nevertheless, we can sort a dictionary by using the feature that you can create dictionaries with a list of 
tuples that contain key and value pairs. So we can get those key-value tuples with the _items()_ method, sort 
them and create a new dict with the sorted tuples.

```python
some_http_codes = {
    404: 'Not Found',
    401: 'Unauthorized',
    400: 'Bad Request',
    403: 'Forbidden',
    402: 'Payment Required',
    405: 'Method Not Allowed',
}

sorted_http_codes = dict(sorted(some_http_codes.items(), 
                                key=lambda kv: kv[0], 
                                reverse=False))

assert sorted_http_codes == {
    400: 'Bad Request',
    401: 'Unauthorized',
    402: 'Payment Required',
    403: 'Forbidden',
    404: 'Not Found',
    405: 'Method Not Allowed',
}
```

The _sorted()_ function takes the key-value tuples and sorts them. To sort them correctly, it is important to 
set the sorting key. This is just a function that takes the key-value tuples and returns the value by which the 
pairs should get sorted. In this case, we want to sort by the HTTP code (the key), which is always stored on 
the 0th index in each tuple. Therefore the lambda expression we use for the sorting key returns this value. If 
you want to sort the dictionary by the values, just return the value at index 1.

## Default Values

Whenever you want to get values from a dictionary it is possible that the given key doesn't exist. In this case, 
Python raises a _KeyError_ that you have to catch, otherwise the program exits. The problem with covering
unknown keys is the amount of boilerplate code it produces. It makes the code less readable and more clunky.

Another way to avoid _KeyErrors_ is to check if the key exists before accessing it:

```python
player_points = {
    "Bob": 1337,
    "Alice": 42,
}

def get_points_for_player(player_name: str) -> int:
    if player_name in player_points:
        return player_points[player_name]
    return 0

get_points_for_player("Bob")  # returns 1337
get_points_for_player("Kevin")  # returns 0
```

But this approach doesn't really solve the problem and produces some boilerplate code, although it's better than the 
previous one. Luckily, there are two ways to get a default value when a key doesn't exist!

The easiest way that works for any dictionary is to use the get() method when retrieving data from it. The first 
argument for this method is the key you are looking for. The interesting part is the second argument, where you can 
optionally pass a default value, which will be returned when the key was not found.

```python
player_points = {
    "Bob": 1337,
    "Alice": 42,
}


def get_points_for_player(player_name: str) -> int:
    return player_points.get(player_name, 0)


assert get_points_for_player("Bob") == 1337
assert get_points_for_player("Kevin") == 0
```

Another way to get a default value when the key doesn't exist is to use the _collections_ module's defaultdict data 
structure. The defaultdict works like a normal dictionary with the additional feature to define a default value, 
which is always returned when retrieving data from an unknown key. The first argument in the constructor is a 
callable whose return value is used as the default argument. Every argument after that will work with the normal 
dictionary constructor.

```python
from collections import defaultdict
from datetime import datetime

last_login = defaultdict(lambda: datetime.now(), {
    "Bob": datetime(year=2020, month=1, day=1),
    "Alice": datetime(year=2020, month=1, day=2),
})

assert last_login["Bob"] == datetime(year=2020, month=1, day=1)
assert isinstance(last_login["Kevin"], datetime)
```

This code snippet shows the same example as the previous one, but is more powerful because you don't have to specify
a fixed value and can create the value dynamically on the fly. Everybody using the dictionary doesn't need to care 
about setting defaults or catching a KeyError. However, notice every time accessing the dictionary with a non-existing 
key, a new entry with this key is added!

## Creating a Dictionary from two Lists

If you ever encounter the situation where two lists should create a dictionary where one list contains the keys and the
other one the values, there is a simple solution!

```python
def dict_from_two_lists(keys: list, values: list) -> dict:
    if not len(keys) == len(values):
        raise ValueError("List of keys must have the same length as the list of values!")
        
    return dict(zip(keys, values))

names = ["Bob", "Alice", "Kevin"]
ages = [21, 42, 69]

ages_of_persons = dict_from_two_lists(keys=names, values=ages)

assert ages_of_persons == {
    "Bob": 21,
    "Alice": 42,
    "Kevin": 69,
}
```

We use the feature to create a dictionary by passing a list of tuples to the constructor, where every tuple is a key-value 
pair. The _zip_ function creates such a list of tuples with two elements. The alternative approach would be a for loop with 
a counter variable, which adds every item to the dict, but the zip-approach is clearly more beautiful. In general, you can 
do really cool things with the _zip_ function combined with dictionaries. Make sure to keep this option in mind before 
writing ugly code!

## Deleting Items

Adding items to a dictionary is quite simple, but what about deleting items? Fair enough, deleting an item from a dictionary 
is really simple. However, there are a few things that you should have in mind when you want to delete items from a Python 
dictionary.

First of all, there are two methods how you can delete items from a dictionary:

- use the **del** keyword
- use the **pop()** method

The main difference between those possibilities is that the _pop_ method returns the removed value, whereas _del_ just deletes 
the item.

```python
sample_dict = {'value_one': 1, 'value_two': 2}

del sample_dict['value_one']
assert 'value_one' not in sample_dict

removed_value = sample_dict.pop('value_two')
assert 'value_two' not in sample_dict
assert removed_value == 2
```

It really becomes interesting when you want to delete items while iterating over the same dict. Python will throw a _RuntimeError_ 
when you try to do this because you change the dictionary's size. Unfortunately, I can't present you a really cool solution 
for this case, but here are two different workarounds. If you know a better solution that solves the problem in a better way, 
please let me know in the comments!

1. Create a copy of the dictionary just for the loop

    ```python
    def remove_even_numbers_1(dictionary: dict):
        for key, value in dictionary.copy().items():
            if key % 2 == 0:
                del dictionary[key]
    ```

2. Delete entries directly from previously calculated keys

    ```python
    def remove_even_numbers_2(dictionary: dict, items_to_delete: list):
        numbers_to_delete = set(key for key, value in dictionary.items() if key % 2 == 0)
        
        for number in numbers_to_delete:
            del dictionary[number]
    ```

I personally don't favor any of these approaches; I think it depends on the context which one fits better. Either way, neither of 
the two variants is really beautiful.

## Merging Dictionaries

If you want to merge dictionaries, you have 3 options:

1. **Use the update() method**
    
    Every dictionary has this method, which can take a dictionary as argument. The items of the passed dictionary get updated if 
    they already exist or added if not. Additionally you can create a new dict by copying one of the dictionaries you want to 
    merge before doing the update. This has the advantage that you don't change the original dicts when you merge them.
    
    ```python
    def merge_two_dicts(first: dict, second: dict) -> dict:
        merged_dicts = first.copy()
        merged_dicts.update(second)
        return merged_dicts
    ```

2. **Merge with the unpacking operator**
    
    A much simpler approach to merge dicts is to simply create a new dictionary with the items from both dicts. You can use the 
    previously explained unpacking operator for this.
    
    ```python
    def merge_two_dicts(first: dict, second: dict) -> dict:
        return dict(**first, **second)
    ```

3. **Use the merge operator (since Python 3.9)**
    
    Python 3.9 introduced the single pipe **|** as the merge operator for dictionaries. If you run your project with Python version 
    3.9 or newer, this might be the best approach.
    
    ```python
    first = {'a': 1, 'b': 2}
    second = {'c': 3, 'd': 4}

    merged = first | second

    assert merged == {'a': 1, 'b': 2, 'c': 3, 'd': 4}
    ```

## Making copies of a Dictionary

With the final dictionary trick, I want to raise awareness of the things to keep in mind when copying dictionaries. Dictionaries are 
mutable data structures, which means that every time you pass a dictionary as an argument to a function or assign it to a new variable, 
the dictionary gets passed by reference. So everything you do with the dictionary in this function will affect the original one too! 
In comparison, when you do the same thing with a string (which is immutable), the function's operations do not affect the original 
string because it gets passed by value.

This makes it harder to create copies of dictionaries. Therefore every dictionary has a **copy()** method, which creates a shallow copy 
of the dictionary. You can check this with the **id()** function that returns the internal id of an object.

```python
sample_dict = {'key': 'value'}

sample_dict_too = sample_dict  # doesn't create a copy
assert id(sample_dict) == id(sample_dict_too)  # same ids

sample_dict_copy = sample_dict.copy()
assert id(sample_dict) != id(sample_dict_copy)  # not the same ids
assert sample_dict == sample_dict_copy  # content is the same
```

Another important fact to keep in mind is that the copy() function only creates a 
[shallow copy](https://www.programiz.com/python-programming/shallow-deep-copy) of the dictionary. This means that the items are inserted 
as _references_ into the copied dictionary. If you have immutable objects as values, they will be the same in the original and the copy! 
To avoid this, you can use the copy module's **deepcopy()** function, which will insert _copies_ of the dictionary items.

```python
import copy


sample_dict = {'key': ['sample', 'list']}

shallow_copy_dict = sample_dict.copy()
assert id(sample_dict['key']) == id(shallow_copy_dict['key'])

deep_copy_dict = copy.deepcopy(sample_dict)
assert id(sample_dict['key']) != id(deep_copy_dict['key'])
```

## Summary

This post presented you 10 dictionary-tricks in Python. I hope you enjoyed them and learned new pythonic approaches to apply to your projects! 
However, I guess there are many more cool things and tricks you can do with dictionaries. If you have one that can't be missing here, 
leave a comment below!


Dictionary-Tricks in Python


## Data Structure for the Algorithm

The basis of the Marching Cubes Algorithm's functionality is the data structure. 
The data is represented by a 3D grid of points, where every point has an additional 
**value** (e.g a number like the brightness). You can subdivide this grid into many 
cubes by taking the points as corners.

![3D Grid Data Structure](/assets/blog/posts/marching_cubes/cubes.png)

Now we can take a closer look at the cubes we have created. Every cube has eight edges 
that hold values. The value can be a grayscale value, or just a weight for each point 
for instance.

![Cube with highlighted Corners](/assets/blog/posts/marching_cubes/cube-corners.png)

Because every of those eight points contain a certain value, we can define a threshold 
that separates all points in two groups: the points whose values are _above_ the threshold 
and those whose value are _below_ it.

With this information, we can create parts of the so-called [isosurfaces](https://svi.nl/Isosurface), 
which result in the whole object when they are put together. For every point whose value is 
above the threshold (in this example the points with the blue circles), we draw a triangle at 
the centers of the adjacent edges.

![Cube with Isosurfaces](/assets/blog/posts/marching_cubes/isosurface.png)

After repeating this process for every cube we have built the whole mesh, and we can see the 
resulting figure. The more cubes / points the grid has, the more detailed is the resulting object.

<iframe width="100%" height="500" src="https://www.youtube.com/embed/LfttaAepYJ8" class="mt-12" title="YouTube video player" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen></iframe>

## Implementation of the Marching Cubes Algorithm

After understanding the concept, we can take a look on how to implement this algorithm. The 
primary challenge for this problem is the determination of the isosurface. Where do we need 
to draw the triangles?

At first glance, one could think of just looping over each point and if it's over the threshold 
to draw lines to the centers of the adjacent edges. But this approach does not work for every 
constellation. If you take a look on the points on the right side of the picture above, you can 
see one of a few special cases that won't work with this method.

Luckily, there is a much better to solve this problem. For this approach, we try to find all 
possible Combinations that the box can have. By trying all the 28 possibilities and leaving 
out the symmetrical mirrored ones, we can find **14** different arrangements (15 with the 
empty one).

Additionally, we can give each combination a unique number by considering is as a binary number. 
Every cube has eight corners, so we can represent the combination for a cube as an 8-bit integer 
value!

```
11000001 = 193
```

This integer value represents the example cube from the previous chapter. In general, we have 256 
combinations (from 0 to 255), for which we can store the resulting triangles in a lookup table! 
This makes the implementation quite easy, because we just have to calculate the integer representation 
for each cube and draw the triangles that we get from the lookup table. In fact, you don't even need 
to create the lookup table on your own, this would be a lot of effort. You can find lookup tables 
for the most programming languages on the internet.

The following Code Snippet shows a simple implementation of the Marching Cubes Algorithm. Nevertheless, 
to make it as readable as possible, it is more pseudo-code than a full-working example.

```python
class Point:

    def __init__(self, x: int, y: int, value: float):
        self.x = x
        self.y = y
        self.value = value


class Cube:

    def __init__(self, corner_points: List[Point]):
        self.corner_points = corner_points


class Grid3D:

    def __init__(self, cubes: List[Cube]):
        self.cubes = cubes


def integer_representation_for_cube(cube: Cube, threshold: float) -> int:
    integer_value = 0
    for index, point in enumerate(cube.corner_points):
        if point.value > threshold:
            integer_value += 1 << index
    return integer_value


def marching_cubes(grid: Grid3D, threshold: float):
    for cube in grid.cubes:
        integer_representation = integer_representation_for_cube(cube, threshold)
        triangles_to_draw: List[Triangle] = lookup_table[integer_representation]

        for triangle in triangles_to_draw:
            draw_triangle(triangle)  # (not implemented here)
```

## Optimizing the Marching Cubes Algorithm

Like for nearly every algorithm, you can optimize the Marching Cubes Algorithm.

Firstly, you can improve the accuracy of the triangles by setting the corners not always in the 
center of the edges. The points can have different distances from the threshold value and not every 
time the same. Therefore, it may make sense to draw the triangle corners closer or further away 
from the point. But this means that you need to adjust the lookup table and maybe parameterize it. 
This will take much more effort to implement it.

Another, more common optimization, is to **parallelize** the computation of the mesh. This is really 
easy for the Marching Cubes Algorithm and one reason why it's so popular. All the cubes are completely 
independent of each other, so we can easily compute the cubes in parallel.

## Summary

To sum up, the Marching Cubes Algorithm is very useful when it comes to modelling objects with a mesh. 
All you have to do is to change the values of the grid points and run the algorithm to update a mesh. 
Additionally, the algorithm is easy to implement and can be parallelized efficiently.

I hope the idea of the Marching Cubes Algorithm is clearer and you understood why the algorithm is quite 
attractive for meshing. However, if there is anything left unclear or you have another question, please 
leave a comment below and I will try to come back to you or adjust the post.


Marching Cubes Algorithm


## Overview

The Midpoint Line Algorithm is based on the Bresenham algorithm and is not a
general approach, but a solution for screening lines between a slope of 0 and 1
(0 - 45°). If you draw a line in computer graphics, you are limited to the
amount of pixel you can use. For example, a simple line could be realized in
several different pixel arrangements

### Original Representation

![Original representation](/assets/blog/posts/midpoint-line/midpoint-line-original.svg)

### Interpretation

![Pixel interpretation
  1](/assets/blog/posts/midpoint-line/midpoint-line-interpretation-1.svg)

### Another interpretation

![Pixel interpretation
  2](/assets/blog/posts/midpoint-line/midpoint-line-interpretation-2.svg)

And these are just two of many possible solutions of creating the line.

So, besides the problem that we usually can't solve algorithmic problems
graphically, we need a rule that tells us which pixels should be drawn and which
not. The midpoint line algorithm does this in two steps:

1. Calculate the mid of two given pixels
2. Evaluate if the line that should be drawn is below or above that mid
   - If the line crosses above the mid the northeast (NE) pixel of the midpoint
     will be inked
   - if the line crosses below the mid, the east (E) pixel of the midpoint will
     be inked

![Midpoint Line Algorithm
example](/assets/blog/posts/midpoint-line/midpoint-line-cropped.svg)

As you see in this example, the mid between `A (3 | 1)` and `B (3 | 2)` is `M (3 | 1.5)`. And the line we want to draw crosses above this midpoint. Thereby we
can ink the pixel **north-east** of the midpoint.

## Mathematical basics

As you now should understand the problem and the suggested solution, we jump
right into the mathematical basics for implementing this algorithm. This section
should cover anything new for you; it's merely a reminder and a mutual
definition of terms.

### Linear function

<math>f(x) = y = mx+b</math>

Whereas `x` and `y` are the corresponding `x` and `y`\-coordinates in our
euclidean grid, `m` is our slope coefficient, and `b` our y-intercept.

### Slope Coefficient

Next we need a way to calculate the `slope coefficient m` we achieve that
by:

<math>
  m=\frac{'{'}\Delta y{'}'}
  {'{'}\Delta x{'}'}
</math>

Whereas `Δy` describes the `y-difference` and `Δx` the `x-difference`.

Given these basics, we may now determine the linear function for the given
example. As we want to draw the line between `A (1 | 1)` and `B (9 | 4)`, we the
following slope coefficient:

<math>
  m = \frac{'{'}4-1{'}'}
  {'{'}9-1{'}'}=\frac{'{'}3{'}'}
  {'{'}8{'}'}
</math>

This alters our original definition of the linear function:

<math>
  f(x) = y=\frac{'{'}\Delta y{'}'}
  {'{'}\Delta x{'}'}x+b
</math>

### y-Intercept

To calculate the `y-intercept`, we need to rearrange the equation to `b` and
insert any point on the line.

<math>y=mx+b \\ b=y-mx</math>

Using the `start point (1 | 1)` we get `b = 5/8`:

<math>
  b=1-\frac{'{'}3{'}'}
  {'{'}8{'}'}*1 =\frac{'{'}5{'}'}
  {'{'}8{'}'}
</math>

Which finally, leads us to the linear function of

<math>
  f(x) =\frac{'{'}3{'}'}
  {'{'}8{'}'}*x+\frac{'{'}5{'}'}
  {'{'}8{'}'}
</math>

for our specific example.

### Calculating the Midpoint

To calculate the midpoint of two points in a vertical line in the grid, we take
their x value (both points got the same value for x) and calculate the y value
by calculating their mean value.

Given these conventions, we may calculate the midpoint of given coordinates `A = (x | y)` and `B = (x | y+1)` by:

<math>
  F(M) = ( x_A | \frac{'{'}y_A+y_B{'}'}
  {'{'}2{'}'} )
</math>

As the `y-difference` between these two points always is `1`, the formula can
further be simplified as:

<math>
  F(M) = ( x_A | \frac{'{'}y_A+y_B{'}'}
  {'{'}2{'}'} ) = ( x_A | y_A + \frac{'{'}1{'}'}
  {'{'}2{'}'} )
</math>

### Determining a Midpoints Relative Position

All that is left now is to evaluate either a given line crosses above or below
the calculated midpoint. To do so, we need the `y-difference` between the
`midpoint` and the line at specific `x-coordinates`.

This, in turn, can be achieved by merely calculating the difference between the
`y-value` of the midpoint and the `y-value` of the line. To ease things further,
we take the linear function and rewrite it to the implicit form:

<math>f(x, y) = \Delta y*x - \Delta x * y + b*\Delta x</math>

For each point that belongs to our original linear function

<math>
  f(x) =\frac{'{'}3{'}'}
  {'{'}8{'}'}*x+\frac{'{'}5{'}'}
  {'{'}8{'}'}
</math>

This formula will return `0`. For each point below our line, the function will
return a **positive value,** and for each point above the line, the function
will return a **negative value**. So we can easily evaluate if a calculated
midpoint is:

1. above the line → the line crosses below the midpoint and `E` will be inked
2. below the line → the line crosses above the midpoint and `NE` will be inked

By convention, we call the return value of our function `f(x,y)` the **decision
parameter** `d` as it decides which pixel will be inked. To make things even
clearer we can fill in the values we already know into the abstract function:

<math>
  \Delta x = 8 \\ \Delta y = 3 \\ b=\frac{'{'}5{'}'}
  {'{'}8{'}'}
</math>

<math>f(x_i, y_i) = 3x_i - 8y_i + \frac{'{'}5{'}'}{'{'}8{'}'}*8 = f(x_i, y_i) = 3x_i - 8y_i + 5</math>

`xi` and `yi` are specific values, whereas `Δx` and `Δy` belong to the original
function. So if we test this for a point `A = (1, 1)` on the line, we get:

<math>f(1, 1) = 3*1 - 8*1 + 5 = 0</math>

## Formulate the Algorithm

Given you understood the mathematical basics, you should now be able to
formulate an algorithmic approach to solve the problem of drawing pixels.

To ease things, I will start by defining simple classes that describe points and
linear functions in the scope we require.

```python
class Point:

    def __init__(self, x, y):
        self.x = x
        self.y = y

    def get_midpoint(self):
        return Point(self.x, self.y + 0.5)
    
    def __repr__(self):
        return f'({self.x} | {self.y})'
```

As you can see, I implemented a method `get_midpoint()` that returns the midpoint of the current coordinates and the same point in which the `y-coordinate`has been incremented by `1`. And a`repr-function` that alters the way the object is
returned as a string.

```python
class LinearFunction:

    def __init__(self, start: Point, end: Point):
        self.start = start
        self.end = end
        self.delta_y = point_b.y - point_a.y
        self.delta_x = point_b.x - point_a.x
        self.b = self._calculate_b()

    def _calculate_b(self):
        return self.start.y - self.delta_y / self.delta_x * self.start.x

    def point_is_below(self, point: Point):
        return 0 > self.delta_y * point.x - self.delta_x * point.y + self.b * self.delta_x
```

The LinearFunction class got a pseudo-private method `_calculate_b()` that
calculates the `y-intercept`. If you read our meaningful names article, you
might protest now that calculate`y_intercept()` might be a better name. And this
is indeed a point; the name might be useless to someone defining linear
functions with different variable names. However, next, we got a function
point `is_below()` that implements the part from the implicit form of the
"Determining a Midpoints Relative Position"-section. It returns True if a given
point is below the line that is created by its function. And that's it. That's
all we need to run the algorithm. Let's bring all this together with a
`MidpointLineAlgorithm`-class.

```python
class MidpointLineAlgorithm:

    def __init__(self, linear_function: LinearFunction):
        self.linear_function = linear_function

    def run(self):
        points = list()

        iterations = linear_function.delta_x
        y = linear_function.start.y
        x = linear_function.start.x

        for x_i in range(iterations):
            current_point = Point(x + x_i, y)
            current_mid_point = current_point.get_midpoint()

            if linear_function.point_is_below(current_mid_point):
                points.append((current_point, "East"))

            else:
                points.append((current_point, "Northeast"))
                y += 1

        return points
```

The Algorithm is initialized with a linear function. The number of iterations we
need corresponds to the number of steps we can do or simply `Δx` of our linear
function. Additionally, we need a starting point; we simply choose the first
point (`(1 | 1)` in our specific case).

The point of interest is in each step defined by the initial `x-value` and the
current iteration `(x + xi),` and the current `y`. Here's the only new thing:
each time the algorithm decides we go northeast, we have to increment `y` as it
only increases if we switch lines. So y is determined by the initial `y` plus
the times we went northeast.

The full code for our specific example looks the following:

```python
class Point:

    def __init__(self, x, y):
        self.x = x
        self.y = y

    def get_midpoint(self):
        return Point(self.x, self.y + 0.5)

    def __repr__(self):
        return f'({self.x} | {self.y})'


class LinearFunction:

    def __init__(self, start: Point, end: Point):
        self.start = start
        self.end = end
        self.delta_y = point_b.y - point_a.y
        self.delta_x = point_b.x - point_a.x
        self.b = self._calculate_b()

    def _calculate_b(self):
        return self.start.y - self.delta_y / self.delta_x * self.start.x

    def point_is_below(self, point: Point):
        return 0 > self.delta_y * point.x - self.delta_x * point.y + self.b * self.delta_x


class MidpointLineAlgorithm:

    def __init__(self, linear_function: LinearFunction):
        self.linear_function = linear_function

    def run(self):
        points = list()

        iterations = linear_function.delta_x
        y = linear_function.start.y
        x = linear_function.start.x

        for x_i in range(iterations):
            current_point = Point(x + x_i, y)
            current_mid_point = current_point.get_midpoint()

            if linear_function.point_is_below(current_mid_point):
                points.append((current_point, "East"))

            else:
                points.append((current_point, "Northeast"))
                y += 1

        return points


if __name__ == "__main__":
    point_a = Point(1, 1)
    point_b = Point(9, 4)
    linear_function = LinearFunction(start=point_a, end=point_b)

    midpoint_line_algorithm = MidpointLineAlgorithm(linear_function=linear_function)

    points = midpoint_line_algorithm.run()

    print(points)
```

If you run the code it will output the following:

```python
[
  ((1 | 1), 'East'),
  ((2 | 1), 'East'),
  ((3 | 1), 'Northeast'),
  ((4 | 2), 'East'),
  ((5 | 2), 'Northeast'),
  ((6 | 3), 'East'),
  ((7 | 3), 'East'),
  ((8 | 3), 'Northeast')
]
```

Which corresponds to the following picture:

![Midpoint line algorithm in
action](/assets/blog/posts/midpoint-line/midpoint-line-result.svg)

## Summary

I hope you could grasp the functionality of the midpoint line algorithm and
understood why and how the mathematical concepts are used to let it work
properly. If there is anything left unclear, please don't hesitate to comment,
and I will try to come back to you or adjust the post as soon as possible. Happy
coding!


Midpoint Line Algorithm


## Hello World

I guess nearly everyone who ever got in touch with Python wrote the famous 'Hello World' 
program in Python, or at least know how to code it. But there is another easy way besides 
printing it, you can also use the built-in implementation:

```python
>>> import __hello__
Hello world!
```

## The Zen of Python

This easter egg is probably the most known one, at least the experienced Python programmers 
are likely to know it. Tim Peters created 20 guiding principles, 19 ones are written down as 
the **Zen of Python**. This feature is also documented in the 
[PEP 20 guideline](https://www.python.org/dev/peps/pep-0020/).

The Zen of Python is telling Python's philosophy and best practices. The message is mainly 
about code quality, design, and how to solve problems. If you follow them, you will 
[improve your code](https://code-specialist.com/category/write-better-code/) and the way you 
solve problems.

You can see the Zen of Python by importing the module _this_, try it out!

```python
>>> import this
```

> Beautiful is better than ugly.  
> Explicit is better than implicit.  
> Simple is better than complex.  
> Complex is better than complicated.  
> Flat is better than nested.  
> Sparse is better than dense.  
> Readability counts.  
> Special cases aren't special enough to break the rules.  
> Although practicality beats purity.  
> Errors should never pass silently.  
> Unless explicitly silenced.  
> In the face of ambiguity, refuse the temptation to guess.  
> There should be one-- and preferably only one --obvious way to do it.
> Although that way may not be obvious at first unless you're Dutch.  
> Now is better than never.  
> Although never is often better than \*right\* now.  
> If the implementation is hard to explain, it's a bad idea.  
> If the implementation is easy to explain, it may be a good idea.  
> Namespaces are one honking great idea -- let's do more of those!
> 
> The Zen of Python, by Tim Peters

Indeed, there is another awesome aspect of this easter egg. If you are interested, take a look 
at the [source code file of the module _this_](https://github.com/python/cpython/blob/master/Lib/this.py).

## Uncle Barry

In the evolving process of Python, the debate of the inequality operator implementation had two 
opposite sides some time ago. The **!=** version finally won the battle against the diamond operator 
**&gt;&lt;** to avoid any ambiguity and improve the readability.

The well-known Python programmer Barry Warsaw (aka Uncle Barry) had the honor to become the Friendly 
Language Uncle For Life (FLUFL). Sadly, he is on the losing side of the debate and prefers the version 
for the inequality operator. But if you have the same vision as Uncle Barry, you can simply import a 
library to change the rules and use the other variant!

```python
>>> from __future__ import barry_as_FLUFL

>>> 0 != 1
SyntaxError: with Barry as BDFL, use '<>' instead of '!='
  
>>> 0 <> 1
True

>>> 1 <> 1
False
```

Using the official inequality operator then raises a SyntaxError, you have to use the other operator instead. 
As an April fool, [PEP 401](https://www.python.org/dev/peps/pep-0401/) says that this operator got enacted by 
the FLUFL in 2009 (and some other stupid changes).

## Braces

One of the most amazing differences between coding in Python and most of the other programming languages (Java, 
C++, JavaScript, etc.) is the following: No curly braces! (except for dicts and sets, but they don't count). I 
think everyone who had to deal with a code containing thousands of curly braces will appreciate the simplicity 
of Python with nothing more than indentations. It's just easier and more readable without curly braces, so 
Python wants to show that they aren't tolerated in the language.

You can try to import the braces from the \_\_future\_\_ build-in module if you can't resist the temptation to 
use them, but look what happens:

```python
>>> from __future__ import braces
SyntaxError: not a chance
```

## Antigravity

You can experience antigravity by Python coding, believe me! Just run the following code and see what appears 
on your screen:

```python
>>> import antigravity
```

## Hash

Is it possible to create a hash of infinity or a NaN value? In Python it is!

```python
>>> hash(float('inf'))
314159

>>> hash(float('nan'))
0
```

## Monty Python References

The Name **Python** is an homage to the English comedy group **Mont**y **Python**. Furthermore, the docs say that 
the language is named after the BBC show “Monty Python's Flying Circus”.

> Making references to Monty Python skits in documentation is not only allowed, it is encouraged!
> 
> [Official Python Tutorial](https://docs.python.org/3.8/tutorial/appetite.html)

There are some references to Monty Python jokes hidden in the whole Python environment, here are a few examples:

- Funny quote of _Monty Python and the Holy Grail_ in the [official input and output documentation](https://docs.python.org/3.8/tutorial/inputoutput.html#the-string-format-method)
- Instead of using _foo_ and _bar_, many tutorials use _spam_ and _egg_
- The name of the [wheel](https://www.python.org/dev/peps/pep-0427/) ZIP-format for Python's sdist packages is based on the [Cheese Shop sketch](https://www.youtube.com/watch?v=Hz1JWzyvv8A) by Monty Python

These are a few Monty Python reference easter eggs, I am sure you can find more by exploring the Python documentation 
and tutorials!

* * *

I presented some cool and funny easter eggs you can find by using Python, I hope you enjoyed it! If you want to explore 
more funny and interesting Aspects of Python, you can take a look at the [Python Humor](https://www.python.org/doc/humor/) 
page in the documentation. If you have any feedback or another cool easter egg that is still missing here, let me know in 
the comments below!


7 Python Easter Eggs


A switch-case statement is smarter than writing lots of large if-statements instead, 
so does Python wants us to mess up our code with the worse alternative?

Definitely not! Despite the fact that switch-statements are indicators for 
**[code smell](https://t2informatik.de/wissen-kompakt/code-smell/)** and you should 
be careful of using them anyway, there is a clever way to make up for the lack of 
switch statements in Python!

**Disclaimer**: Python 3.10 introduced [structural pattern matching](https://peps.python.org/pep-0636/)
which might also be an alternative for the switch statement depending on the 
use-case. However, this post is focussed on an alternative that works with 
all versions of Python 3, if you want a dedicated post for the new pattern 
matching please let me know in the comments!

## What is the Switch-Statement?

The switch statement is a control flow for comparing a value with multiple variants. 
In general, this is just syntactic sugar for a large if-statement. The syntax may
look like this:

```JavaScript
switch (x) {
  case 'variant1':
    // do something here if x == 'variant1'
    break;
  case 'variant2':
    // do something here if x == 'variant2'
  default:
    // do something here if no case matched
    // or no break statement jumped out of the switch before
}
```

This syntax applies to nearly all common programming languages which support the 
switch-statement, like Java, JavaScript, or C for instance. Note that you have to use 
a _break_ if you want the program to leave the switch statement after a case matched! 
Otherwise, the program will continue running the instructions for the next case, without 
checking if the case actually matches.

## The Code Smell of Switch-Statements

The previous explanation already showed one of the most common pitfalls with switch 
statements: The syntax can easily lead to unintended bugs when you mess something up 
with the break-statements.

Besides that, you can do **anything** within the body of a case. That means every case 
can have a different behavior which may be really unintentional. For instance, you can 
stop a loop in the above statements or return something and exit the current function. 
This can get really messy when your project grows!

Additionally, the switch-statement is not optimal for code that is supposed to get extended 
in the future. Every time you want to add a case, you have to modify the statement. Depending 
on the language you are using, you maybe need to redeploy the whole application every time 
you want to add some functionality. This clearly violates the 
[Open-Closed Principle](https://code-specialist.com/code-principles/ocp-principle/)!

## Alternative in Python

If you want to check a value for many possibilities, you clearly want something like the 
switch-statement. The worst thing you can do is to solve the problem with tons of if-statements 
that pollute your code – you should never do that! But the good thing is: There's a smart 
pythonic approach to compensate for the switch-statement!

In general, the pattern is based on the enormous power of dictionaries in Python. As an example, 
we take a look into a simple command-line game. The game will just ask you for a decision at 
each step. In this context, the dictionary mapping gives you the power to do something like this:

```python
direction_messages = {
    "north": "You freeze to death",
    "east": "You die in the desert after 42 days of suffering",
    "south": "You win the Superbowl!",
    "west": "You fall into a gigantic ravine and die instantly",
}

direction = input("where do you want to go? ")
message = direction_messages.get(
    direction.strip().lower(), 
    "You have to give a compass direction"
)
print(message)
```

In this example, the value for which we check the cases is the user's input and the cases are 
represented as keys of the dictionary. Note that do the dictionary lookup with the **get()** 
function, which every dictionary provides. The advantage is, that you can specify a default 
value with the second parameter. This corresponds to the default case of the switch statement 
and we don't have to catch a KeyError, in case the user gives an invalid answer.

## Adding functionality to the cases

Okay, this covers basically the functionality of a switch-statement, but we only have strings 
as values. Indeed, the switch-statement allows you to run any code you want for a case. To 
achieve this, we can use a **dictionary of functions**. At first, this may seem a bit strange, 
but we can use function objects as dictionary values too! With this technique, we can create a 
mapping for functions that define the blocks of code for each case.

But first, let me start with the easiest version: **lambda functions**. They are simply 
single-expression and anonymous functions. They take as many arguments as you want, but can only 
execute one expression and return its result. A simple example for using lambdas in this context 
would be a basic calculator:

```python
operations = {
    "+": lambda x, y: x + y,
    "-": lambda x, y: x - y,
    "*": lambda x, y: x * y,
    "/": lambda x, y: x / y,
}

operation = input("Which operation should I execute? ")
operation_function = operations.get(
    operation,
    lambda x, y: "unsupported operation"
)

numbers = input("Numbers for the calculation: ")
numbers_list = numbers.split(' ')
result = operation_function(*list(map(int, numbers_list)))
print(f"Result: {result}")
```

Although this is just a very basic and a little bit ugly calculator you can do much better for 
sure, it shows how you can use lambdas for simple operations. This time, we get a callable object 
instead of a string from the dictionary. This callable is stored in a variable, which we can call 
with the numbers the user gives the calculator.

## We can use functions too!

Though the lambdas can be really useful, in most cases you want to do more than one expression. 
Sure, you can call a function in a lambda expression – but you can also replace the lambda with 
a function right away. Then you have a function for every case, where you can do everything you 
want. Now we want to expand our little command-line game from before.

```python
def player_goes_north():
    pass  # do here whatever you want

def player_goes_east():
    pass  # do here whatever you want

def player_goes_south():
    pass  # do here whatever you want

def player_goes_west():
    pass  # do here whatever you want

def unknown_direction():
    print("You have to enter a compass direction!")

direction_messages = {
    "north": player_goes_north,
    "east": player_goes_east,
    "south": player_goes_south,
    "west": player_goes_west,
}

direction = input("where do you want to go? ")
perform_player_action = direction_messages.get(
    direction.strip().lower(),
    unknown_direction
)
perform_player_action()
```

Now we can do anything we want for each way the player wants to go. It works the same way as before, 
we just replaced the lambdas with normal functions.

A common pitfall, especially when one isn't familiar with the concept of functions as objects, might 
be to put parentheses behind the function names in the dictionary. Make sure to only have the name 
in there. You don't want to call the function, you only want the function **object** stored in the 
dictionary.

There are a few more possibilities to use dictionaries with functions, but these are the basic ones 
that should suffice most of the time. Finally, I want to point out the fact that you can use 
**any mutable object** as a key in dictionaries, not only strings like in the previous examples. 
You can even use data-types for it and much more! So you have great opportunities to take advantage 
of this pattern!

## Conclusion

In Python, you can easily create a good alternative for the switch-statement in a really pythonic 
way. I personally think that this way is even better than the switch, it has almost none of its 
disadvantages.

Although, there is even another approach. There is an object-oriented alternative that uses 
**polymorphism** to get rid of the switch-statement. This pattern is particularly common in other 
languages where you cannot use the pythonic approach. If you're interested in this alternative or 
have questions about this topic, leave a comment and I do my best to explain it!


Python Switch-Statement Alternative


## Types of underscores

There are various possibilities to encounter or use underscores in Python, and
they seem confusing if you aren't familiar with the conventions and concepts
they are used in. There are basically these five types of underscores you might
face:

1. simple underscore: **\_**
2. leading underscore: **\_x**
3. subsequent underscore: **x\_**
4. double leading underscore: **\_\_x**
5. double leading and subsequent underscore: **\_\_x\_\_**

## The simple underscore in Python

The simple underscore is often used as a variable when someone wants to indicate
that the variable doesn't matter.

```python
for _ in range(42):
    print("The meaning of life")
```

As you see the variable is useless here, I just use it to print "the meaning of
life" 42 times.

Another usage for the simple underscore in the Python REPL (read evaluate print
loop) is to print the last evaluated expression:

```python
In[1]:   a = 42
In[2]:   b = 1337
In[3]:   a*b
Out[3]:  56154
In[4]:   _
Out[4]:  56154
```

## The leading underscore in Python

The leading underscore indicates that the following variable should be in the
private scope of a class. However, Python does not have a concept of visibility,
this is just a convention. When someone uses a leading underscore, all he or she
want's to say is that this should not be visible from outside. If you have
written in Java or another language that comes with visibility concepts you
should have come along this concept yet. For those starting with Python this
seems odd.

```python
class PublicClass:
  
  def __init__(self, attribute):
    self.attribute = attribute
    self._private_attribute = "Initialized" # I am private 
    
  def get_private_attribute(self):
    return _private_attribute
  
  def set_private_attribute(self, value):
    value += "I am statically appended"
    # Do something fancy
    self._private_attribute = value
```

So, looking at this method, the underscore indicates that the
_\_private\_attribute_ is private and should not be accessed directly. In
contrast to the current example: If there are also no methods to interact with
the attribute, this usually indicates that this variable should only be visible
to the inner context, hence the class, method or whatever.

## Subsequent underscore in Python

The subsequent underscore merely indicates that the original name is conflicting
with a reserved keyword. For example if the input of my function might be
lambda, the interpreter will return a syntax error as it conflicts:

```python
def function(lambda):
    pass
```

If I simply use an underscore after lambda, the interpreter will accept it again
and I am still able to use my meaningful name:

```python
def function(lambda_):
    pass
```
## The double leading underscores in python

This is the first underscore in Python we covered so far, that isn't a bare
convention. When you add double leading underscores to a variable the so called
name mangling will be applied. Name mangling instructs the compiler to protect
the variable from being overwritten by sub classes. A short example:

```python
class MyClass:

    def __init__(self):
        self.__MyAttribute = 42
```

If we now create an instance of MyClass:

```python
In[1]: from underscores import MyClass
In[2]: myclass = MyClass()
In[3]: dir(myclass)
Out[3]: ['_MyClass__MyAttribute', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']
```

We see that our class now has a weirdly named attribute
_\_MyClass\_\_MyAttribute_ and as you see below, you don't may access it with
the name you gave it in your constructor.

```python
In[1]: myclass.__MyAttribute
Out[1]:    Traceback (most recent call last):
              File "<input>", line 1, in <module>
           AttributeError: 'MyClass' object has no attribute '__MyAttribute'

In[2]:  myclass._MyClass__MyAttribute
Out[2]: 42
```

To access the attribute you instead have to use the class name with a leading
underscore. This prevents attributes from being overwritten from sub classes as
the name becomes more unique. In your class itself, you may still use
_\_\_MyAttribute_ instead of the shown name. Nevertheless as you may assume,
this isn't a guarantee. If you name your variable _\_MyClass\_\_MyAttribute_ you
could still overwrite it (but I can absolutely not imagine why you would do
this)

![disgusted displeased senior bald man in yellow
shirt](/assets/blog/posts/underscores/disgusted-old-man.webp)

## Double leading and subsequent underscores in Python

Finally we get to the most common example of underscores in Python. If you look
sharply you notice that we got nearly a ton of those up there when we printed
the built-in function
_[dir](https://docs.python.org/3/library/functions.html#dir)_ of our class. So
why didn't I just explain the double leading and trailing underscores first?
Because name mangling doesn't apply here! And the only reason is, that trailing
double underscores disable the name mangling feature.

What are those double leading and subsequent underscores now? In short: they
indicate built-in methods and you shouldn't name your functions with double
leading and trailing underscores, as this might result into overwriting Python
core functions unintentionally. However there are some cases in which you
especially want to override those functions deliberately, which is allowed of
course. For some constructs like context managers it is even necessary to
override certain build-in functions.

If you're deep into Python you might also encounter the term _dunder_ which is
short for "double under". So we would pronounce our class as "dunder
MyAttribute". But note that we would also pronounce "\_\_init\_\_" as dunder
init not "dunder init dunder".

This concludes my article on underscores in Python. I hope you enjoyed reading
and didn't miss anything. Whatever your impression of my post was, I would love
to hear your feedback!


Underscores in Python


## Why should we use concurrency?

When you execute a set of instructions on a CPU, it takes a given amount of time
by default. The major advantage of this concurrency is, that in theory and in a linear
example we save 50% of computing time if we use 100% more CPU cores.

![The idea of using multiple cores](/assets/blog/posts/concurrency/concurrency.svg)

To understand how the concept works and which possibilities we have to improve
our computation speed, we need a few terms clarified. If you feel confident
about these terms, you may skip the first few headings and select one of
the three possibilities I introduce:

- [Multithreading in Python using threading.Thread](#mutlthireading-threading)
- [Multithreading in Python using joblib](#multithreading-joblib)
- [Multiprocessing in Python using multiprocessing.pool](#multiprocessing)
- [Comparing the parallelization techniques](#comparing)

## When to apply concurrency?

Generally spoken you can apply concurrency whenever you can execute your code
parallelly. Imagine a simple example where you want to add 8 numbers.

<math>1+2+3+4+5+6+7+8</math>

To further simplifiy things, let's assume each addition would take a full
second. So it would take 7 seconds to add these 8 number as we have to do 7
operations. How could you compute this in multiple threads? Well you could
simple split these operations up into 7 additions of their own:

<math>
    \begin{'{'}aligned{'}'}
    1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 \\ \\
    1 + 2 = 3 \\
    3 + 4 = 7 \\
    5 + 6 = 11 \\ 
    7 + 8 = 15 \\ \\
    3 + 7 = 10 \\
    11 + 15 = 26 \\ \\
    10 + 26 = 36 \end{'{'}aligned{'}'}
</math>

So in the first phase we would need 4 CPU cores in the second only 2 and in the
last only 1. By applying this we theoretically save 4 seconds as this takes only
3 seconds to compute the 7 additions. We have to say theoretically as this is a
insanely simplified example that doesn't cosider various problems that occur
when applying this.

Nevertheless, as you might have noticed, you have to know pretty exactly what
your actual computation task is to apply concurrency in a meaningful way. You
also have to know, if your concurrent computation is mathematically valid. We
get into trouble if we just do the same with:

<math>(1+2+3)+(4+5*6) / (7+8)</math>

## What is a process?

A process or task is a program in execution. It is a specific instantiation of a
program on a system and has additional information besides its actual code. To
execute it, your program must be translated to instructions your CPU is able to
understand. This is something you usually don't have to care about as compilers
and interpreters do this for you.

But to actually execute it, your translated code still has to move to the CPU
somehow. This happens by loading the information into the main memory and from
there to your CPU (Theres actually happening a lot more but it's way too much
for this article, you may read it up here:
[https://en.wikipedia.org/wiki/Process_(computing)](https://en.wikipedia.org/wiki/Process_(computing))

Depending on which operating system you use, your system will assign a fixed or
variable amount of main memory to this process wherein its information will be
stored.

## Process information

The allocated memory usually is splitted into four crucial segments:

- Heap: Contains dynamically assigned memory for storing relevant information
- Stack: Contains jump labels and local variables
- Data: Contains global variables
- Text: Contains the actual instructions that should be executed on your CPU

![](/assets/blog/posts/concurrency/process.svg)


Despite the fact that you do not have infinite main memory heap and stack may
not increase infinitely either. They usually share an allocated size of memory.
You may have heard the term „stack overflow“ and now you know what is meant with
it. When your program code is stored in the main memory the so called scheduler
will assign CPU computation time to it and then it may finally be executed (Not
fully, just as long as its computation time allows).

As this shouldn't go too much into boring details, I will not elaborate further
here. If you want to know more on this, please leave a comment and I will try to
write a more detailed version.

## What is a thread?

A thread is a part of a program that can be executed concurrently. A process may
contain various threads. There are two major classes of threads:

- Kernel threads: the operating system supervises them
- User-Threads: a program must supervise them

A thread shares the data- text-segment with all other threads of the same
process. Threads can be executed on multiple CPU cores. However, threads also
have some additional information of their own:

- Stack
- Program Counter

![](/assets/blog/posts/concurrency/thread.svg)

The stack usually is stored in the memory allocated for the stack of the
process. Whereas the program counter is independent. These terms clarified we
may finally come to the actual concepts of applying parallelization.

## Multithreading

One way to apply concurrency we somehow already covered a bit is multithreading.
There is some confusion about this term as there is hardware and software based
threading.

If we talk about hardware threading we mean a single CPU core that may execute
more than one thread at once. This is often called a logical processor or
thread. Modern Intel and AMD CPUs mostly have two logical processors on each
physical core. So for example an [AMD Ryzen Threadripper
3970x](https://amzn.to/2EaGaja) (Affiliate Link) with 32 physical cores has 64
logical ones which means it may execute up to 64 threads at once on a single
CPU.

If we talk about software based threading we mean that we give specific
instructions on how our program may be executed on multiple (logical)
processors.

## Multiprocessing

Another way to reach parallelization of computation is to simply start multiple
processes instead of threads. An obvious disadvantage of this method is, that
this will produce a lot of redundant data (overhead) in the main memory, if you
simply produce multiple process instances of the same job as they don't share
their memory anymore. However, this might sometimes be intended as you may see
when you for example encounter the [global interpreter lock (GIL)](#) in Python.

## A problem for concurrency

As a basis for looking deeper into how to apply concurrency in Python we need an
example that isn't too far from reality. Therefore I took a simple crawler that
visits websites and returns their content.

```python
import requests


def get_content(url: str) -> str:
    get = requests.get(url)
    return get.text[:100]  # Returns the first 100 characters


def crawl(url: str) -> str:
    content = get_content(url)
    # do something with the content
    return content
```

To do the actual HTTP-GETs we use the [requests
library](https://requests.readthedocs.io/en/master/). Our crawl function takes
an URL as an argument and will return the first 100 characters of the pages HTML
content as a string via _get_content()_.

Next we need some URLs to run the crawler on. To simplify things further, we
will take advantage of a Python feature and use a list of 4 URLs and multiply it
by 4 to get a list of 16 URLs:

```python
URLS = ["https://code-specialist.com/errors/pyaml5-3-1-github-actions/",
        "https://code-specialist.com/write-better-code/comments/",
        "https://code-specialist.com/code-principles/kiss-principle/",
        "https://code-specialist.com/code-principles/dry-principle/"] * 4
```

The usual approach to run this would simply be, to call the function with an
URL:

```python
from concurrency.crawler import crawl
from concurrency.URLS import URLS


def non_parallel_crawling(adresses):
    results = list()

    for url in URLS[:adresses]:
        results.append(crawl(url))
    return results
```

As we got a baseline, now we can look further into how to apply the different
techniques.

## Multithreading in Python using threading.Thread

As elaborated earlier our problem at hand is to speed up a crawler by applying
concurrency. I prepared some simplified code to pose how this could possibly be
realized via the threading.Thread class:

```python
from threading import Thread

from concurrency.URLS import URLS
from concurrency.crawler import crawl


def fourwise_iterartion(iterable):
    x = iter(iterable)
    return zip(x, x, x, x)


class CrawlThread(Thread):
    def __init__(self, address: str):
        Thread.__init__(self)
        self.address = address
        self.terminated = False

    def run(self):
        return crawl(self.address)


def multithread_crawling(adresses: int) -> list:
    results = list()

    # Results into 4 parallel threads at once

    for url1, url2, url3, url4 in fourwise_iterartion(URLS[:adresses]):
        threads = [CrawlThread(url1), CrawlThread(url2), CrawlThread(url3), CrawlThread(url4)]
        for thread in threads:
            thread.start()
        for thread in threads:
            results.append(thread.join())
        # Wait for results to synchronize

    return results
```

In this example we will run 4 threads concurrently which is why we need a
function that will cut our URL list into tuples of the length 4. This is
realized in the function _fourwise_iteration()_ which uses the built-in
functions _iter_ and _zip_.

Next we need our actual Thread class. We reach this by introducing our own class
_CrawlThread_ that extends the Thread class and overwrites it's _init()_ and
_run()_ function. In our init function we pass all the variables we require
later on. In our run function we define what should be executed when the thread
is started. For further information please look into the [threading
documentation](https://docs.python.org/3/library/threading.html).

Next we have our _multithread_crawling_ function that actually runs 4 threads
to crawl 4 URLs nearly simultaneously. Please note that our result list will
contain tuples of length 4 too.

## Multithreading in Python using joblib

A pretty easy and lucid way to do this is by using the
[joblib](https://joblib.readthedocs.io/en/latest/) module:

```python
from joblib import Parallel, delayed

from concurrency.crawler import crawl
from concurrency.URLS import URLS


def joblib_crawling(adresses: int, n_jobs: int = 4) -> list:
    results = Parallel(n_jobs=n_jobs)(delayed(crawl)(url) for url in URLS[:adresses])
    return results
```

The syntax is straight forward and there is barely anything to explain. To run
joblib we need a list of arguments that is provided in _URLS\[:adresses\]_. So
for example if our variable addresses is 4 this will provide the first 4 URLs
from the list URLS.

The _n_jobs_ parameter of _Parallel_ defines how many jobs we want to run
concurrently. Next in delayed we provide the function we want to run, please
note that we don't use any brackets there as we want to pass the function and
not the result of the function. Following that we do a simple for loop to pass
our URLs to the crawl function via Parallel.

The simplicity of this code is beautiful and I personally prefer this approach
when dealing with concurrency in Python.

## Multiprocessing in Python using multiprocessing.Pool

Another way to reach is to spawn processes instead of threads. This causes
overhead to some extend but also leads to sparing the global interpreter lock
(GIL).

The straightest way to reach this is by using the Pool function of the [built-in
multiprocessing module](https://docs.python.org/3/library/multiprocessing.html):

```python
from multiprocessing import Pool

from concurrency.crawler import crawl
from concurrency.URLS import URLS


def multiprocess_crawling(adresses: int, n_processes: int = 4):
    process_pool = Pool(processes=n_processes)
    results = process_pool.map(crawl, URLS[:adresses])
    process_pool.close()
    return results
```

As we saw in an earlier example we again need a list of arguments and a factor
of concurrency. In this example it is again or list of URLs up to a certain
point and 4 processes. First we need to define a _Pool()_ which will define how
many processes can be spawned at once. Next all we need to do is simply map our
function to our arguments list.

Please note that you have to close these processes, otherwise they will be kept
in background as long as your program runs which might even result into system
failure if you spawn a very large number of processes.

As always I used this as an opportunity to apply the [clean
code](https://code-specialist.com/tag/clean-code/) principles to these code
examples. If you think I missed something I would be glad to hear your opinion
and improve!

## Comparing the Python parallelization techniques

Last but not least and maybe the most interest, we want to look into how our
execution time actually changes by applying these techniques. As we covered some
basic technical details earlier, you should be able to at least give some
educated guesses on how the execution time evolves.

The result is pretty much what we should have expected:

![Evaluation of the different version (concurrent and non-concurrent mixed)](/assets/blog/posts/concurrency/concurrency-evaluation.webp)

The two fastest concurrent version of the crawler are the threaded ones, as they
produce the least overhead. Next at about 7 URLs the multiprocessing version
starts also to become faster than the single executed crawler.

This concludes my report about concurrency in Python. Thanks for reading this
article. I hope I neither bored you with technical details nor disappointed you
with missing information. However, if there is anything that is left unclear or
I may improve on, don't hesitate to leave a comment or contact me directly.


Concurrency in Python


## What are Python type hints?

Type hints in Python specify the type of variables or functions parameters as well as the return type. 
You can read more about them in the [PEP 484 Guidelines](https://www.python.org/de#v/peps/pep-0484/).

In general they work like type declarations in every typed programming language, like Java or C++. The 
only difference is that type hints are completely optional. Python remains a dynamic language, because 
the Interpreter simply ignores the type hints. They are only relevant for development and have no influence 
on the execution. Accordingly, you can run a Python program with completely incorrect type hints without 
getting any errors.

Type hints exist just for you as a developer to make your life easier - so you should use them!

## Types in Python

Python has many [built-in types](https://www.informit.com/articles/article.aspx?p=453682&seqNum=5), way 
more than you might have expected. Some common examples for build-in types are _str_ for strings, _int_ 
for integers and _bool_ for boolean values.

Furthermore, types are objects like everything else in Python. By the way, you can actually use types as 
keys for a dictionary. So the following code really works!

```python
types_mapping = {
  int: "Integer",
  str: "String",
  bool: "Boolean",
}

print(types_mapping[int])
```

Okay, you can do lots of things with the build-in types, but they are not really specific. For instance, 
you can have a variable of the type _list_, but what's actually in the list? Nobody knows.

That's why Python 3.5 introduced the [typing module](https://docs.python.org/3/library/typing.html)! You 
can import various types from this module, which you can specify really precisely. The following examples 
show some features of the typing module, but feel free to take some time and explore the module on your own.

## How do type hints look like?

The syntax for type hints is, like the Python syntax in general, very simple.

In the following code example, you can see some type specifications for variables. The according syntax is 
just a colon after the variable name followed by the type.

```python
# Type Hints with build-in types
age: int = 42
names: list = ["Alice", "Bob"]
translations: dict = {1: "one", 2: "two"}
are_type_hints_great: bool = True


# Type Hints with the typing module
from typing import List, Tuple, Callable

names: List[str] = ["Alice", "Bob"]
translations: dict = {1: "one", 2: "two"}
fruits: Tuple[str] = ("apple", "banana", "kiwi")
my_lambda: Callable = lambda name: f"Hello {name}" 

  
# Type Hints using self defined classes
class Dog:
  pass

alice: Dog = Dog()
```

For these examples the types might seem a bit redundant and obvious. In fact, you will use this kind of Type 
Hint rather seldom in comparison to the others. But when you use the returned object of a function for instance, 
you may not know the type of it.

More frequently used are type hints for parameters, and they basically work just the same as for variables. But 
note that this applies for parameters with default arguments too!

```python
from typing import List, Tuple, Dict


class Car:
  pass

def too_many_parameters(name: str, amount: float = 6.66, the_number: int = 42):
  pass

def more_parameters(scores: List[int], persons: Dict[str, Tuple[str, int]]):
  pass

def and_even_more_parameters(police_car: Car, racing_cars=List[Car]):
  pass
```

Another possibility to specify type hints is for the return type of a function. There is also another syntax for 
this case: You specify the return type with an "arrow", followed by the type. It is important that the type is 
in the function definition **before** the colon.

```python
from typing import List, Tuple


class House:
  pass

def calculate_average(numbers: List[int]) -> float:
  pass

def get_all_customer_names() -> List[str]:
  pass

def get_my_house() -> House:
  pass

def get_my_neighbours() -> Tuple[House]:
  pass
```

This is the complete syntax you need to know if you want to use type hints! Easy, isn't it?

## Benefits of specifying types

Some IDEs like [PyCharm](https://www.jetbrains.com/pycharm/) support type hints in different ways. First of all, 
PyCharm complains when a type hint does not matches with your coding. Additionally, you get a suggestion what the 
type should be or what is expected. This prevents unintentional behaviours where you expect a type and are surprised 
when you get something completely different. With type hints, you don't have to reengineer every function you want 
to call just for knowing what it will return anymore. In most cases, the combination of the function name and the type 
specifications make it clear how to use the function. If it doesn't, you should improve the readability of your code, 
for example by applying the [KISS Principle](https://code-specialist.com/code-principles/kiss-principle/) or giving 
[Meaningful Names](https://code-specialist.com/write-better-code/meaningful-names/).

Another really useful feature is the auto-completion! A common use-case for this scenario is an instance object from a 
class with some attributes and methods.

```python
class Dog:

  def __init__(self, name):
    self.name = name

  def bark(self):
    print(f"{self.name} barks!")


def get_dog_from_name(dog_name):
  return Dog(dog_name)


alice = get_dog_from_name("Alice")
alice.bark()
```

If you call a function which returns an object, you might store it in a variable and work with it. Without type hints, 
the IDE wouldn't know what _alice_ is. If you type a point after the name, you won't get any useful suggestions for 
functions or attributes of the _Dog_ class. This changes when you add type hints!

```python
class Dog:

  def __init__(self, name: str):
    self.name = name

  def bark(self):
    print(f"{self.name} barks!")


def get_dog_from_name(dog_name: str) -> Dog:
  return Dog(dog_name)


alice = get_dog_from_name("Alice")
alice.bark()
```

Now you will get all possible suggestions for working with the class instance.

As we have seen, type hints are a really powerful tool to increase your productivity and motivation. It also greatly 
improves the readability of your code as everyone can quickly see what kind of objects you are working with. So all in 
all, you should definitely use type hints!


Use Python Type Hints!


## Why are good names so important?

Every name you write in your code is only for humans. The compiler, interpreter or whatever component runs your code
will throw your names away and use internal identifiers. So the purpose of your names is literally just for you and
every other developer to understand the code. Furthermore, code is far more often read than written.

> Indeed, the ratio of time spent reading versus writing is well over 10 to 1. We are constantly reading old code as
> part of the effort to write new code.
> 
> **Robert C. Martin,** Clean Code: A Handbook of Agile Software Craftsmanship

Consequently, writing readable code by using meaningful names is much more rewarding than just doing it "quick and
dirty". It may take a lot longer than doing it quickly, but in the long term you and your colleagues will benefit from
the code quality.

## Giving bad names is quick

Even if it doesn't seem like it at first glance, sometimes it can be really difficult to give meaningful names. It is
often very challenging, especially to make it understandable for outsiders.

Everybody knows the situation, where you have a topic or a goal, and just start coding right away. You are incredibly
productive in a short time and the outcome is great. But most of the code we write in this attitude is anything but
clean and readable. Consider this small example where the names don't say anything about what the code does.

```python
def statistics_page_7():
    inp = get_data("sales")
    inp = clean_data(inp)
    inp = inp[::-1]
    av = sum(inp) / len(inp)
    sum_ = sum([pow(num - av, 2) for num in inp])
    res = math.sqrt(sum_ / len(inp))
    return get_page("Statistics", 7, res).render().get_content()
```

It is very hard to understand what the code actually does. The writer, with all the thoughts and concepts in mind,
will understand the code because he wrote it. But after a while, when not dealing with the code for a long time, those
insights are gone and the code isn't easily understandable anymore. Furthermore, when the code needs to be adjusted
because the requirements changed or there is a bug in it, there's a problem. Getting familiar with poorly written code
can be very frustrating and may take a long time. To fully understand the code, the developer possibly needs to debug
and dig into it line by line, which is really time-consuming and exhausting.

## How to give meaningful names

So, how can you come up with meaningful names? Here are a few tricks that will help you to improve the names in your
code!

### 1. Don't be afraid to use long names

Even if it seems confusing at the fist moment, long names are mostly better than short ones. Especially when the
shorter name would hide some relevant information, the better choice is always better to include it in the name.
The code will grow, but most information about purpose of the code is in the names. By the way, explaining what
the code does with lots of comments isn't a good idea, as you can see in our post
to [Comments - What you should keep in mind](https://code-specialist.com/write-better-code/comments/).

### 2. Keep your functions doing only one thing

By applying this principle, you keep your functions small and reusable. Every function has only one purpose, and
therefore it is easy to give them meaningful names. Additionally, you will have more functions and more
opportunities to give names. As a result, your code contains more names and tells what it's doing.

### 3. Let others review your code

Only if someone else understands your code you should be satisfied with the code and your names. There are lots of
other advantages of code reviews, so it's recommended to do it anyway. Nonetheless, after your code passed the
review, it's not "finished". Writing code is an iterative process that never ends, so you should keep improving it!

With these tricks and some creativity, we can refactor the previous code.

```python
def get_sales_statistics_page() -> str:
    sales_data: List[float] = get_data(sales)
    cleaned_sales_data: List[float] = clean_data(sales)
    sales_standard_deviation: float = calculate_standard_deviation(cleaned_sales_data)
    return get_html_page(topic="Statistics", page_id=7, template_data=sales_standard_deviation)

def calculate_standard_deviation(numbers: List[float]) -> float:
    variance = calculate_variance(numbers)
    return math.sqrt(variance)

def calculate_variance(numbers: List[float]) -> float:
    average = calculate_average(numbers)
    sum_of_squared_difference_from_average = 0

    for number in numbers:
        difference_from_average = number - average
        sum_of_squared_difference_from_average += pow(difference_from_average, 2)

    return sum_of_squared_difference_from_average / len(numbers)

def calculate_average(numbers: List[float]) -> float:
    return sum(numbers) / len(numbers)

def get_html_page(topic: str, page_id: int, template_data: dict) -> str:
    page_object = get_page(topic=topic, page_id=page_id, template_data=template_data)
    rendered_page = page_object.render()
    return rendered_page.get_content()
```

After this revision, the code grew a lot but it looks much more inviting. Other developers looking at this code will
more likely be motivated to read, understand and extend it.

I hope this post inspired you to reflect the namings in your code and improve the readability. Do you have any tips and
best practices for giving meaningful names in code? You're welcome to share and discuss them in the comments!


Meaningful Names


## Do we really need comments?

Nowadays, everyone uses source code management systems (like [GitHub](https://github.com/) for instance), 
especially when working with multiple developers on one project. If you don't use a code management system 
for your projects, consider migrating your project to use one. Even if you work alone on your project - 
it's still very useful.

With source code management systems, you can find every written line in the project. It doesn't matter if 
someone deleted it a long time ago, you can always look up every older version. So you don't need to 
comment-out code segments any more! Furthermore, when you have commented-out code in your project, it's 
very likely for it to remain for a long time. And someday you ask yourself: What is this code and why is 
it here? Maybe you don't even dare to delete it, because it could be from someone else and still relevant. 
In the end, it's always better to instantly delete unused code.

Okay, to comment-out code isn't a good idea. But it's still very useful to describe the code with comments, 
so other ones can understand it better, right? No, not in most cases. If you need to explain in every line 
what your code does, it's just not readable. There are so many ways to structure and style your code so it
becomes more readable. Consider this example, where you only have a chance to understand what's going on by 
reading the comments:

```python
def get_customer_data(csv):
    d = open(csv, 'r').readlines()  # read csv file as list with rows as strings
    cust_d = list()  # list of customer data which gets returned
    for i in range(1, len(d)):  # iterate over rows of csv file
        e = (d[0].split(';'), d[i].split(";"))  # split columns and rows to lists
        f = []  # helper array for better datastructure
        for n in range(len(e[0])):
            # add pairs of column name and value to helper array
            f.append([e[0][n], e[1][n]])
        # create dict with columns as keys and data as values for each customer
        cust_d.append({x.strip(): y for x, y in f})
    return cust_d
```

This code actually works with semicolon-separated CSV files, but it's really hard to read. So how can we 
improve this little mess?

## How to let your code speak for itself

There are a few ways to let your code tell what it's doing. The goal is that the person reading your code can 
easily understand what's going on, just by reading the actual code - and not any comments.

1. **Use meaningful names**
    
    Every programmer probably has already heard this advice somewhere. And few programmers will say: "Ah, 
    nothing easier than that! My names are always understandable, it's no hard deal". I think most of the
     people who said that (and I'm including me too) wouldn't understand some code they have written a few 
     years ago in some old project because it can be really hard to give meaningful names an outsider would 
     understand.
    
2. **Don't get too big**
    
    Especially in large and complex projects with thousands of lines of code, there are often situations where 
    functions, classes, etc. get really big. This causes various problems: On the one hand, it's just hard to 
    read and keep the context in mind when you have to scroll down a few times for reading one function. On the 
    other hand, giving meaningful names is much easier when you separate your code into different parts which 
    you can give describing names.
    
3. **Abstractions are your friends**
    
    This point goes hand in hand with the previous ones. Try to abstract everything you do. By using abstractions, 
    you have lots of opportunities to give explaining names for the different abstraction levels. Additionally, 
    you prevent your functions, classes, etc. to get too big. The idea of this approach is to hide further details 
    behind every abstraction, and every time you dive deeper, you get to see more detail. Mostly it isn't necessary 
    to know the details from another point of view, and it's way easier to understand the code's purpose from a 
    high-level perspective.

With these concepts in mind, we can refactor the previous code example a bit. After doing that, it looks like this:

```python
from typing import Dict, List

def extract_values_from_string(string: str) -> List[str]:
    cleaned_string = string.strip()
    string_as_list = cleaned_string.split(";")
    return string_as_list


def map_columns_to_values(column_names: str, rows: List[str]) -> List[Dict[str, str]]:
    column_value_mappings = list()
    column_names_list = extract_values_from_string(column_names)

    for row in rows:
        row_values = extract_values_from_string(row)
        customer_data = dict()

        for column_index, column_name in column_names_list.items():
            column_value = row_values[column_index]
            customer_data[column_name] = column_value

        column_value_mappings.append(customer_data)

    return column_value_mappings


def get_customer_data_from_csv(csv_filepath: str) -> List[Dict[str, str]]:
    csv_file = open(file=csv_filepath, mode='r')
    file_data: list = csv_file.readlines()
    column_names, rows = file_data[0], file_data[1:]

    return map_columns_to_values(column_names, rows)
```

This looks much more readable now, right? The code got three times longer than the old version, but now it's way more 
readable. There are also a few layers of abstraction: The top-level function doesn't show much detail. But every time 
you move an abstraction level deeper, it reveals more information.

## Comments lie

Another problem with comments is that they lie. By saying that, I mean that you can't really trust what comments say 
about the code. They always could be outdated and give wrong information about the code.

In the common case, the code we write changes many times and is never "perfect". As the project you're working on evolves 
and the requirements change, the code has to evolve too. Furthermore, nobody can write clean and perfect code instantly.
Writing code is always an iterative process where you revise and improve the code in each step.

These continuous changes make it really hard to keep all the comments up to date. Imagine there is a comment in every 
second line of your code, or even more! On the one hand, it's really nasty and time-consuming having to change a comment 
every time you change something in the code. On the other hand, it's nearly impossible to keep all comments up to date, 
there will always be an unintentionally outdated one. With modern IDEs, it is very easy to refactor code by allowing the 
program to do so. Nowadays, changing names or cleaning up unused code are standards for every professional IDE. But in 
this way, the IDE only refactors the code and not the comments, how could it? It has inadvertently made the comments 
lying about the code.

## So it's not allowed to write comments any more?

After all these problematic issues about writing comments, we could come to the conclusion that we shouldn't write comments 
at all. But I can tell you this is not the case. And you can definitely write really good code with the usage of useful 
comments. First and foremost, this post wants to draw attention to the various problems when writing comments. We can 
also write better code that speaks for itself without writing comments.

There are still a few cases where it's inevitable to use comments. Most projects should have documentation, which often 
uses comments. Especially when the code or some interfaces to use it are available for externals, good documentation is 
essential. Either way, it would definitely make sense to explain the code's functionality when it solves complex problems. 
Then it's quite helpful for others to understand what the code does, without needing to understand the complex code. By 
writing comments like this, they also are less likely to lie about the code after refactoring. But you still have to adjust 
the comments and the documentation whenever the general behavioral changes.

To sum up, comments can be very helpful in writing understandable code. However, overusing them is never a good choice. 
They pose some relevant difficulties that everyone needs to be aware of.


Comments - What you should keep in mind


## About do one thing

Do one thing at a time - As stated in our earlier posts like [DRY Principle (Computer Science)](https://code-specialist.com/code-principles/dry-principle/), [KISS Principle (Computer Science)](https://code-specialist.com/code-principles/kiss-principle/), or even [Comments – What you should keep in mind](https://code-specialist.com/write-better-code/comments/), **readability** should be crucial to anyone's code.

Something that lastingly improves the readability of your code is a technique called "do one thing". It is often confused with the single responsibility principle (SRP), and the names are indeed confusing, in my opinion.

The main concept behind this is to reduce the code's complexity by diminishing the things statements and functions do, to exactly **one simple thing**.

## Garbage Code

To show you an example of how it works and how significant the improvements can be, look at the following (garbage) code:

```python
class Sieve:
    start = 2

    def __init__(self, n):
        self.n = n

    def run(self):
        l = [True] * self.n
        prims = list()
        for i in range(self.start, self.n):
            if l[i] is True:
                prims.append(i)
                for j in range(i + 1, self.n):
                    if l[j] is True:
                        if j % i == 0:
                            l[j] = False
            else:
                continue
        return prims


s = Sieve(120)
print(s.run())
```

## Sieve of Erastothenes

What does this code do? Rarely anybody can tell that after just reading it. If you read it thoroughly or inspect the output

```
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113]
```

You might find out that this is actually the [sieve of Eratosthenes](https://en.wikipedia.org/wiki/Sieve_of_Eratosthenes) or at least any algorithm dealing with prime numbers. But if you never heard of the sieve of Eratosthenes, you will most likely search a while until you figure out what this code does and why.

And even if you wrote this and are fully aware of what it does, you might forget all of it in just a few days.

## Why is that code garbage?

Some people might argue that the code works perfectly fine, and some might even argue, "but it is optimized!". Besides someone telling you that most likely never heard of broadcasting in Python, he or she also didn't respect the first rule of optimization:

> The first rule of optimization is: _Don't do it._
>
> Bjarne Stroustrup - C++ Coding Standards

And the reason for that is that you never really know what your compiler or interpreter actually does. Things that look complex to calculate might be faster than some "optimized" branch of code. To effectively optimize code, you need profound insights into the language at hand and the compiler or interpreter.

This is a job for **experts** that are specialized in this kind of work. As a developer, you should always rate **readability over optimization**. However, refactorings regarding optimization also should only be done if the time required for computation largely exceeds the expectations.

> The second rule of optimization (for experts only) is: _Don’t do it yet_.
>
> Bjarne Stroustrup - C++ Coding Standards

## Improved Code

With the concept of the sieve at hand and the idea of a simple job, let's rephrase our code in a more meaningful way.

```python
class SieveOfEratosthenes:
    start = 2
    prime_numbers = list()

    def __init__(self, maximum):
        self.maximum = maximum
        self.boolean_array = [True] * self.maximum

    def run(self):
        self.sieve()
        return self.prime_numbers

    def add_prime_number(self, number):
        self.prime_numbers.append(number)

    def is_prime(self, index):
        return self.boolean_array[index]

    def sieve(self):
        for number in range(self.start, self.maximum):
            if self.is_prime(number):
                self.add_prime_number(number)
                self.remove_multiples_from_sieve(number)
            else:
                continue

    def remove_multiples_from_sieve(self, prime):
        for number in range(prime, self.maximum):
            if number % prime == 0:
                self.boolean_array[number] = False


sieve = SieveOfEratosthenes(maximum=120)
print(sieve.run())
```

Isn't this, in contrast, way easier to read and get? Even if your code is still garbage, using the name of the concept as the class name, you can reduce someone else's research effort from hours to minutes as they can easily find what this should be about. The functions themselves are easy to read as the function name represents exactly the one thing the function does.

Thank you for reading this article! I hope I was able to make the point. I am aware that the improved sieve code isn't perfect. If you come up with an even better to read solution, please e-mail us or post a comment. The smartest solution you can come up with will be permanently displayed here.


Do one thing - Not the single responsibility principle (SRP)


## What does KISS stand for?

KISS stands for **Keep it simple stupid**. It is the first usability principle
of product design but is also used as a coding principle. [Legends
say](https://en.wikipedia.org/wiki/KISS_principle#Origin) that the acronym was
minted by [Kelly
Johnson](https://www.lockheedmartin.com/en-us/news/features/history/johnson.html),
a lead engineer at Lockheed. Johnson handed his team of design engineers a
handful of tools and challenged them to build a jet aircraft that was fixable by
what an average mechanic in the field under combat circumstances would have.

## What does KISS mean?

The key design principle itself, however, is as simple as its name indicates:
Keep it simple. This can be simple. But: there are countless ways to describe
something in a complex way, but there might only be one that is precisely and
short. It's not only about simplifying things. You must be able to take a step
back and view whatever you just did from one's perspective, who is totally
clueless. You might have already engaged this problem by writing code and
reviewing it some months after that. Many lines are unreadable crap because you
weren't able to take a step back and simplify the concepts.  

As we mentioned earlier, KISS isn't a principle that is originated in computer
science. It can be applied to design processes in general, such as product or
software design. We, in this case, will focus on how to apply simplistic design
to code itself.

## Code is not written for machines

The code you write and the code you actually execute largely differ, depending
on the language and compiler or interpreter used. In fact, machines couldn't
care less about your code. All the programs and scripts you write are for
humans. You write it only once but read it possibly a thousand times. Thereby
it's your responsibility to make this as easy, and thereby quick, to read as
possible. Your code design objective thereby should be to offer the best user
experience possible. 

## Simple doesn't always mean short

Another mistake that is often made dealing with the KISS principle is that
people try to make code as short as possible. But the shortest solution is
rather seldom the easiest to read. Consider the following "Non-KISS" example:

```python
f = lambda x: 1 if x <= 1 else x * f(x - 1)
```

Now look at this version:

```python
def faculty(number: int) -> int:
    if number <= 1:
        return 1
    else:
        return number * faculty(number - 1)
```

Sure, the first version is also readable, but it is considerably harder to do
so. Adding a few descriptive names, type hints, and splitting the line into
multiple ones gets clearly easier to read. The main goal in design should always
be to be as easy to understand as possible. One of many common bad practices of
developers is to create fancy and unreadable "oneliners". There is absolutely no
reason to do so. This will most certainly lead to someone, if not themself
struggling to decipher it.

## How to apply the KISS principle?

To apply KISS to design and code, there hundreds of ways but, I want to provide
you with some ideas:

### Simplify and explain concepts

Try to code as simple as possible, unfold your concepts, name things properly
and explain with comments where the code itself can't (Most of the time, it can
and should!).
    
### Use descriptive names
    
As easy as this sounds, trying to find [names that indicate something's true
meaning]() can be very challenging. You may try this by telling someone a name
and let them assume what they think that is. Don't be afraid of long variable or
function names. As stated earlier, the machine doesn't care and won't use your
name anyway.
    
### Deconstruct problems into smaller ones
    
Something that does real magic to the readability of your code is deconstructing
the problem and solving just one simple one with each function. Try refactoring
your code and solve a single problem at a time, but with a maximum of precision
and readability.
    
### Try rubber
    
Did you ever try to explain your code to a dumb duck? No? Try [duck
debugging](https://rubberduckdebugging.com/)! By doing this, you may not only
find bugs, but you may also find gaps in your or someone else's understanding of
the current code.
    
### Learn more about Clean Code
    
Many principles and ideas of clean code build upon the KISS principle. By
applying those concepts, you may find it easier to identify the simplest
solution.
    

## References

- The origin of KISS -
  [https://en.wikipedia.org/wiki/KISS\_principle#Origin](https://en.wikipedia.org/wiki/KISS_principle#Origin)
- Kelly Johnson -
  [https://www.lockheedmartin.com/en-us/news/features/history/johnson.html](https://www.lockheedmartin.com/en-us/news/features/history/johnson.html)
- Meaningful Names -
  [https://code-specialist.com/code-principles/meaningful-names/](https://code-specialist.com/code-principles/meaningful-names/)
- Rubber Duck Debugging -
  [https://rubberduckdebugging.com/](https://rubberduckdebugging.com/)
- More Posts on Code Principles Blog -
  [https://code-specialist.com/code-principles/](https://code-specialist.com/code-principles/)


KISS Principle (Keep it simple stupid)


## The DRY Acronym

DRY stands for “**d**on’t **r**epeat **y**ourself”. The DRY principle is one of
the most important principles in software development, if not even the most
important. Whenever you write or read code and find repetition in it, chances
are high you or whoever planned it, didn't plan well enough.

## What's the problem about code repetition?

Code isn't static. When you write code twice you have to test it twice and if
you have to change it afterwards you also have to change it twice, in fact we
already got the four-fold effort to invest. There is also a large chance, you
miss the second occurrence. Seriously, nobody can remind himself to every line
or routine of code he wrote in a project, at least if it isn't an insanely small
one.

> There is a word for software that isn't changed: Hardware
> 
> Unknown author

In fact, redundant code is not only one of the most common causes for bugs, it's
also the cause for unnecessary complex or at least long code. The DRY principle
helps you to avoid these problems.

## Example of duplicated code

```python
first_player_points = [2, 3, 0, 5]
second_player_points = [4, 1, 1, 2]

def get_score_for_first_player():
  return sum(first_player_points)

def get_score_for_second_player():
  return sum(second_player_points)
```

Whats wrong by changing this to:

```python
player_points = {
  1: [2, 3, 0, 5],
  2: [4, 1, 1, 2]
}

def get_score_for_player(player_number: int):
  points = player_points[player_number]
  return sum(points)
```

I am aware that this is a rather simplified example and that the solution
doesn't apply to every problem, but that's not the point. Fact is, by
generalizing concepts and for example moving redundant code to functions you
save time and spare potential bugs. If applied correctly it may also increase
the readability of your code.

## How to apply the DRY principle?

There is no general concept that allows you to always spare repetition, but
there are steps that may help you to minimize it.

### Plan thoroughly 
    
By planing your program or software carefully you may identify patterns and
generalized concepts in an early stage.
    
### Review frequently 
    
Review your code frequently, or even better let someone else review your
code (That will also ensure your code doesn't exceed a certain level of
illegibility).
    
### Use tools that support you 
    
Depending on how large your project is, it's nearly impossible to find
repetition manually. But there are tools that allow you to find repeated
code anyway. For instance PyCharm¹ offers a duplicates tool window that
allows you to identify code repetition.

## References 
- [Link to PyCharm](https://www.jetbrains.com/de-de/pycharm/)


Form Endpoint to send Emails with AWS Lambda and AWS SES

Recent Posts