catnotes

Symbol Python New Project

December 22, 2023

Is Python your language of choice? Do you have dreams about the Symbol Python SDK? Would you like to get the most out of your Python projects? If so, read on for some quick tips. 👇

Use pyenv

If you're like me, you have a lot of Python projects on your computer. Sometimes, you need to use a newer version. Other times, you need to use an older version. Managing all the different versions can be a real pain. Thankfully, there's a tool for that - pyenv.

If you're using a MacOS, you can easily install pyenv using homebrew:

brew update
brew install pyenv

If it was installed correctly, you can see all the available Python versions on your machine by running:

pyenv versions

Once it's installed, pyenv will manage one or many installed versions of Python. Even better, you can use an isolated Python environment for each of your projects! This is strongly encouraged and will make it easier for you to manage your project dependencies!

Let's create a new environment based on Python 3.11 for a new Symbol project:

pyenv virtualenv 3.11 symbol-project-1

In your project directory, run the following command:

pyenv local symbol-project-1

This will do two things:

  1. Activate the newly created Python environment
  2. Create a .python-version file that will cause pyenv to automatically activate the symbol-project-1 Python environment when entering its containing directory

You can verify that a new Python environment was created by running:

pip list

The output will show a minimal number of installed packages, as opposed to all the packages installed in your system default Python version.

Track Requirements

pip supports loading packages from a file using the -r switch. For example, the following will load the packages listed in the file requirements.txt:

python3 -m pip install -r requirements.txt

It's generally a best practice to separate packages used in your code directly from ones that are only used as part of your development processes (e.g. build and test dependencies). If you're familiar with the package.json file used in NodeJS projects, this is the same separation encouraged by dependencies and devDependencies. In order to emulate this in Python, we recommend using two requirements files:

  • requirements.txt - Packages used in your code directly
  • dev_requirements.txt - Packages only used as part of your development processes

Setup linter

Pretty code is always better than ugly code. Linters make it easier to enforce a set of best practices and common stylistic rules across a code base. This makes it really easy to collaborate with other people because it ensures everyone is using the same styles and conventions. Gone will be the days of receiving a one line change accompanied by a completely reformatted file! 😭 In addition, this forced consistency frees developers from a range class of trivial decisions giving them more mental bandwidth to focus on writing code and higher level problem solving.

The Symbol team has spent a good amount of time - maybe too much - fine tuning a standard Python ruleset that is used across all our projects. If you want to use the same rules, it's easy!

First, copy this directory to a linters/python directory within your project. Second, create a scripts/ci folder in your Python project or subproject and add the following two scripts.

setup_lint.sh

This script simply installs all project dependencies - lint requirements, project requirements and dev requirements:

#!/bin/bash

set -ex

python3 -m pip install -r "$(git rev-parse --show-toplevel)/linters/python/lint_requirements.txt"
python3 -m pip install -r requirements.txt
python3 -m pip install -r dev_requirements.txt

lint.sh

This script runs four linters:

  1. shellcheck - Checks all your shell scripts for correctness
  2. isort - Enforces a deterministic order of your Python imports
  3. pycodestyle - Checks some of the PEP8 style conventions
  4. pylint - Checks a large number of best practices and stylistic conventions

Together, these linters help you write very clean code. Nevertheless, there can be an adjustment period as some of the rules will likely annoy you - at least at first! You should try to live with the rules, but you can always turn some of them off. Checking some rules is still much better than checking none!

#!/bin/bash

set -ex
find . -type f -name "*.sh" -print0 | xargs -0 shellcheck
find . -type f -name "*.py" -print0 | PYTHONPATH=. xargs -0 python3 -m isort \
    --line-length 140 \
    --indent "  " \
    --multi-line 3 \
    --check-only
find . -type f -name "*.py" -print0 | PYTHONPATH=. xargs -0 python3 -m pycodestyle \
    --config="$(git rev-parse --show-toplevel)/linters/python/.pycodestyle"
find . -type f -name "*.py" -print0 | PYTHONPATH=. xargs -0 python3 -m pylint \
    --rcfile "$(git rev-parse --show-toplevel)/linters/python/.pylintrc" \
    --load-plugins pylint_quotes \
    --disable "${PYLINT_DISABLE_COMMANDS}"

Now let's make sure everything's configured correctly. Add a new Python file with the following contents:

import asyncio
from symbolchain.nem.KeyPair import KeyPair
from symbolchain.CryptoTypes import PrivateKey
from symbolchain.facade.NemFacade import NemFacade

def test():
    keyPair = KeyPair(PrivateKey.random());

Then run the linters:

./scripts/ci/lint.sh

Surprisingly, this small snippet of code has three warnings!

  1. isort detects the imports are out of order
  2. pycodestyle detects an errant semicolon
  3. pylint detects the variable keyPair should be key_pair

Fixing all of these results in:

import asyncio

from symbolchain.CryptoTypes import PrivateKey
from symbolchain.facade.NemFacade import NemFacade
from symbolchain.nem.KeyPair import KeyPair


def test():
    key_pair = KeyPair(PrivateKey.random())

It's a small difference, but it's worth it. It already looks a little better! Eventually, you'll appreciate the consistency and cleanliness throughout your code. I know I do!