Sumana Harihareswara – Cogito, Ergo Sumana: Some Novel Python Packaging/Distribution/Inspection/Installation Projects

Blog by Sumana Harihareswara, Changeset founder

03 Jul 2021, 21:32 p.m.

Some Novel Python Packaging/Distribution/Inspection/Installation Projects

People who program in Python have an easier time hearing about package-related tools that have been around for a while and that are under the banner of the Python Packaging Authority, or that are commercially supported (this simplified diagram showcases a lot of them). And if you’re looking for canonical guidance on what tools to use, check out packaging.python.org and tell your colleagues.

A simplified diagram illustrating some of the important tools in Python packaging and how they relate to each other. On the left, end user tools (pip, conda, and pipenv) are on a computer. They draw information from cloud-based tools to the right: Warehouse (PyPI), bandersnatch, conda-forge, and Anaconda Cloud. Those in turn draw information from developer tools to the right: conda-skeleton, twine, setuptools, auditwheel, wheel, and packaging utils.

But — since open source and open standards make things interoperable — people also develop new tools for their specific needs in packaging, distribution, inspection, and installation, and sometimes I come across them when people announce them. I haven’t tried any of these yet but here’s a list of some stuff I noticed from the last few years.

Pypitoken, “A library for generating and manipulating PyPI tokens”

Thoth, “an enhanced server-side resolution offered to the Python community” (related: thoth-solver: “A tool for aggregating Python package metadata” and Dependency Monkey which “can compute all the possible combinations of packages that can occur in a resolved software stack and verify the given stack works well”)

installer, “a low-level library for installing wheel distributions”

Dotlock “is a package management tool similar to pipenv, but with a different philosophy: instead of acting as a wrapper around pip, dotlock handles package resolution natively.”

simpleindex provides “a lightweight PEP-503 private index/proxy” that declares routing rules to serve files from local directories. Also see pywharf.

Mach-nix “allows one to package Python projects and environments with Nix, requiring minimal knowledge of Nix…. Why would you want to use this tool? Reproducible builds with all build and run-time dependencies provided by the same package manager, regardless of whether they’re Python dependencies or not.”

The Python Packaging platypus mascot, a purple platypus happily springing out of a crowded cardboard box

ipwhl: a downstream repository in which “Each repo release will ensure a single version for a project for each platform, and one can use it to replace PyPI for both build and runtime dependencies for reproducibility.” Per the repo for “interplanetary wheels (or floating cheeses)”: “platform-unique, singly-versioned Python binary distributions backed by IPFS for security and reproducibility.”

Python devirtualizer: “a preliminary implementation which manages
shared packages so that only one copy of each package version is
required.”

pip-deepfreeze: “a small tool that aims at managing the dependencies of a Python application in a virtual environment.”

And one more thing that is a PyPA project: the Python Advisory DB. After public discussion, there’s a new community-owned repository of security advisories for packages published on https://pypi.org.

Author: Shantun Parmar

Leave a Reply

Your email address will not be published.