Packaging Python Projects

1

python -m pip install --upgrade pip

The project structure

1
2
3
4
5
6
7
8

edx-helper/
├── LICENSE
├── pyproject.toml
├── README.md
├── edx_helper/
│   ├── __init__.py
│   └── edx_dl.py
└── tests/

Creating pyproject.toml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "edx-helper"
dynamic = ["version"]
description = "Simple tool to download video and lecture materials from edx.org."
readme = "README.md"
license = {file = "LICENSE"}
maintainers = [
    { name = "Ye Zheng", email = "csyezheng@gmail.com" },
]
keywords = [
    "MOOCs",
    "edX",
    "edX-dl",
    "download",
    "education",
    "video",
]
classifiers = [
    "Development Status :: 4 - Beta",
    "Environment :: Console",
    "Intended Audience :: End Users/Desktop",
    "License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)",
    "Operating System :: OS Independent",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.7",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
    "Topic :: Education",
]
dependencies = [
    "beautifulsoup4>=4.6.0",
    "html5lib>=1.0.1",
    "six>=1.11.0",
    "youtube_dl>=2021.12.17",
    "requests>=2.18.4",
    "tqdm>=4.66.1",
]

[project.optional-dependencies]
dev = [
    "coverage>=3.7",
    "mock>=1.0.1",
    "pytest>=2.5",
    "pytest-cov",
    "pytest-xdist",
    "tox",
    "pandoc",
]

[project.scripts]
edx-helper = "edx_helper.edx_dl:main"

[project.urls]
Homepage = "https://github.com/csyezheng/edx-helper"

[tool.hatch.version]
path = "edx_helper/__init__.py"

[tool.hatch.build.targets.sdist]
include = [
    "/edx_helper",
    "/test"
]

[tool.hatch.build.targets.wheel]
packages = ["/edx_helper"]

Generating distribution archives

1

python -m pip install --upgrade build

Now run this command from the same directory where pyproject.toml is located:

1

python -m build

This command should output a lot of text and once completed should generate two files in the dist directory:

1
2
3

dist/
├── edx_helper-0.6.0-py2.py3-none-any.whl
└── edx_helper-0.6.0.tar.gz

The tar.gz file is a source distribution whereas the .whl file is a built distribution.

Uploading the distribution archives

  1. register an account on TestPyPI.

  2. Create a PyPI API token at https://test.pypi.org/manage/account/#api-tokens

  3. setting the “Scope” of a PyPI API token to “Entire account”.

  4. install Twine:

    1

    python -m pip install --upgrade twine

  5. upload the archives

    1

    python -m twine upload --repository testpypi dist/*

    You will be prompted for a username and password. For the username, use __token__. For the password, use the token value, including the pypi- prefix.

  6. verify package on TestPyPI

    1

    https://test.pypi.org/project/edx-helper

Installing your newly uploaded package

  1. You can use pip to install your package and verify that it works. Create a virtual environment and install your package from TestPyPI:

    1

    python -m pip install --index-url https://test.pypi.org/simple/ --no-deps edx-helper

  2. You can test that it was installed correctly by importing the package. Make sure you’re still in your virtual environment, then run Python:

    1

    python3

    1
2

    from edx_helper import edx_dl
edx_dl.main()

Real step

  • Register an account on https://pypi.org
  • Use twine upload dist/* to upload your package and enter your credentials for the account you registered on the real PyPI. Now that you’re uploading the package in production, you don’t need to specify --repository; the package will upload to https://pypi.org/ by default.
  • Install your package from the real PyPI using python3 -m pip install [your-package].

TODO

rewrite README.md