21Packages & program structure

In Chapter 20 we cracked the import machine open. We met sys.path, the chain of finders, the loaders, and the .pyc cache, then watched a single module get found, compiled, and cached. In this one we point that same machine at a whole directory. Your playlist has outgrown a single file. Shuffle logic, storage, and the data classes now sprawl across a dozen .py files, and import shuffle sits next to import config with no hint of which belongs to whom. So we fold the pile under one name, import playlist, the way import json hands you a whole folder of code through a single handle. Here's the plan. We'll see exactly what runs when a directory imports, and meet __init__.py as runnable module code rather than ceremony. Then we'll sort absolute imports from relative ones, learn what python -m quietly repairs, and watch two innocent imports poison each other. The whole way through we keep asking the one question that decides all of it: when a folder becomes a name, what code runs, and in what order? By the end we've restructured the whole playlist into a real package. There's one clean front door, the private plumbing hidden behind it, and python -m playlist starting the program from a single command.

01A package is a directory that imports as one name

Let's start with the mess. You've been writing one file per idea, like engine.py, storage.py, and model.py, and reaching across them with import storage. It worked, but the folder is now a junk drawer. Nothing says these files are one thing. Meanwhile import json hands you a whole directory of standard-library code through a single word. That's exactly what you want here: import playlist, one name for the whole project. So here's the question, and it's sharper than it looks. How does a folder become a single importable thing, and what code, if any, runs the moment it does?

Watch what import actually does, because it isn't what most people picture. import playlist does not go hunting for a file called playlist. It walks a list of finders, the objects sitting in sys.meta_path, the same crew you met last chapter. It asks each one the same question: do you know how to load the name 'playlist'? The finder that handles the filesystem, FileFinder, scans the directories on sys.path. The instant it spots a directory named playlist holding an __init__.py, it says yes. It hands back a module spec, a little record whose loader is a SourceFileLoader aimed straight at playlist/__init__.py. That's the handle the rest of the import is built on.

Now the import machinery runs a fixed four-step ritual, and every step matters later:

# 1. create an empty module object — a namespace, nothing in it yet
mod = new_module("playlist")

# 2. mark it as a PACKAGE by giving it __path__
mod.__path__ = [".../playlist"]   # where future sub-imports will search

# 3. register it BEFORE running any of its code
sys.modules["playlist"] = mod    # remember this line — §5 turns on it

# 4. exec __init__.py's bytecode, using mod.__dict__ as its globals
exec(compile(init_source), mod.__dict__)

Read step 2 twice. The single thing that separates a package from an ordinary module is the attribute __path__, a list holding the package's own directory. An ordinary module like math has no __path__. A package has one, and that list is the map future sub-imports consult when they need to find playlist's children. Step 4 is the reveal. __init__.py is not configuration, not a manifest, not boilerplate the interpreter reads for settings. It is the package's module body: ordinary Python that runs top to bottom, exactly like a script. The fresh module's __dict__ serves as its global namespace. Whatever names it binds become the package's attributes. Print something at the top of __init__.py and it prints the instant someone imports the package.

print("init running")
print("__name__    =", __name__)      # playlist
print("__package__ =", __package__)   # playlist
print("__path__    =", __path__)      # ['/abs/path/to/playlist']

And here is the surprise that trips everyone: importing the package does not import its submodules. import playlist runs __init__.py and stops, so there is no playlist.engine yet — touch it and you get AttributeError. A submodule springs into existence only when something explicitly imports it, and at that moment the machinery does something quietly elegant. After loading engine.py, it binds the submodule onto its parent, so playlist.engine becomes a real attribute of the playlist module object. The package is a namespace that fills in on demand.

you type
# ---------- jukebox/__init__.py ----------
print("jukebox/__init__.py running")


# ---------- jukebox/tracks.py ----------
print("jukebox/tracks.py running")

TITLES = ["Blinding Lights", "Titanium", "Levels"]


# ---------- run.py  (one level UP from the package) ----------
import os
import sys

import jukebox                      # runs jukebox/__init__.py -- once

print("name        :", jukebox.__name__)
print("__file__    :", "/".join(jukebox.__file__.replace(os.sep, "/").split("/")[-2:]))
print("__path__    :", [os.path.basename(p) + "/" for p in jukebox.__path__])
print("has tracks? :", hasattr(jukebox, "tracks"))

import jukebox.tracks               # NOW the submodule runs

print("has tracks? :", hasattr(jukebox, "tracks"))
print("bound onto  :", jukebox.tracks.__name__)
print("same object :", sys.modules["jukebox.tracks"] is jukebox.tracks)
print("first title :", jukebox.tracks.TITLES[0])

import jukebox                      # cached: not one line re-runs
print("import #2   : nothing printed -- sys.modules hit")


# ---------- the terminal, standing in myproject/ ----------
$ python run.py
$ python -c "import jukebox; jukebox.tracks"

you see
$ python run.py
jukebox/__init__.py running
name        : jukebox
__file__    : jukebox/__init__.py
__path__    : ['jukebox/']
has tracks? : False
jukebox/tracks.py running
has tracks? : True
bound onto  : jukebox.tracks
same object : True
first title : Blinding Lights
import #2   : nothing printed -- sys.modules hit

$ python -c "import jukebox; jukebox.tracks"
jukebox/__init__.py running
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'jukebox' has no attribute 'tracks'

That sys.modules line in step 3 also explains why a package initialises exactly once. Import playlist twice in the same program and __init__.py runs a single time. On the second import, the machinery finds playlist already sitting in sys.modules and hands back the cached object without re-executing a line. Every module is a process-wide singleton keyed by its dotted name. This is the same cache that, in Ch 22 onward, you'll watch your program lean on as it starts reaching past its own memory.

You have one name for the whole folder. But right now users can reach every file and every helper inside it. Next: how __init__.py lets you publish a clean front counter and hide the plumbing behind it. →

02__init__.py defines the public surface

Your users type from playlist import Track and it works. Good. But nothing stops them typing from playlist.engine._internal_shuffle import _fisher_yates — and now someone's code depends on a private helper you meant to rewrite next week. The moment they do, your freedom to change it is gone. You need a way to say, out loud and enforceable-in-spirit: this is the API I promise to keep; everything else is plumbing I reserve the right to move. Where does that line live, and how do you draw it?

The answer falls straight out of Section 1. A package's public surface is nothing more mystical than the set of names bound in the __init__ module's namespace after it finishes executing. There is no separate export table, no public: keyword. If a name is an attribute of the package module, it's reachable as playlist.that_name; if it isn't, it isn't. So you curate the surface by choosing what __init__.py binds. The tool is the re-export:

# pull the good names UP from the submodules where they live
from .model import Track, Playlist
from .engine import Player
from .storage import save, load

__all__ = ["Track", "Playlist", "Player", "save", "load"]

That from .engine import Player runs inside __init__.py, so it binds Player as an attribute of the playlist module, and now from playlist import Player resolves directly. The user never learns, and never needs to learn, that Player physically lives in engine.py. This is facade construction: your folder layout is your implementation, free to be reorganised, while the import path is your API, held stable. Move Player from engine.py into engine/core.py next year, fix the one line in __init__.py, and every caller keeps working. The facade decouples the two.

you type
# ---------- jukebox/tracks.py ----------
"""tracks.py -- what a track IS. No I/O."""
from dataclasses import dataclass


@dataclass
class Track:
    title: str
    artist: str
    seconds: int

    def __str__(self):
        minutes, secs = divmod(self.seconds, 60)
        return f"{self.title} - {self.artist} ({minutes}:{secs:02d})"


# ---------- jukebox/storage.py ----------
"""storage.py -- the only file that knows what JSON is."""
import json

from .tracks import Track


def save(tracks, path):
    with open(path, "w", encoding="utf-8") as f:
        json.dump([vars(t) for t in tracks], f)
    return path


def load(path):
    with open(path, "r", encoding="utf-8") as f:
        return [Track(**row) for row in json.load(f)]


def _tmp_name(path):                 # plumbing: not part of the promise
    return path + ".part"


# ---------- jukebox/__init__.py ----------
"""jukebox -- a tiny music library."""
from .tracks import Track
from .storage import load, save

__all__ = ["Track", "load", "save"]
__version__ = "1.0.0"


# ---------- surface.py ----------
import jukebox

print("__version__ :", jukebox.__version__)
print("__all__     :", jukebox.__all__)

ns = {}
exec("from jukebox import *", ns)
print("import *    :", sorted(n for n in ns if not n.startswith("__")))

print("same class  :", jukebox.Track is jukebox.tracks.Track)
print("submodule   :", jukebox.tracks.__name__, "| still reachable")
print("private     :", jukebox.storage._tmp_name("library.json"))

ns = {}
exec("from jukebox.storage import *", ns)      # storage has NO __all__
print("storage star:", sorted(n for n in ns if not n.startswith("__")))

you see
$ python surface.py
__version__ : 1.0.0
__all__     : ['Track', 'load', 'save']
import *    : ['Track', 'load', 'save']
same class  : True
submodule   : jukebox.tracks | still reachable
private     : library.json.part
storage star: ['Track', 'json', 'load', 'save']

$ python -c "from jukebox import Track; print(Track('Titanium', 'David Guetta', 245))"
Titanium - David Guetta (4:05)

$ python -c "from jukebox.storage import _tmp_name; print(_tmp_name('a.json'))"
a.json.part

Now the two things people expect to be privacy controls, and neither one is. First, __all__. It is a list of strings, and it governs exactly one thing: what from playlist import * pulls into the caller's namespace. That's it. It does not hide anything. With the __all__ above, from playlist import * brings in those five names. But from playlist import debug_dump still works fine for any name that actually exists, listed or not. So __all__ is a curated star-export manifest and a documentation signal ("these are the headline names"), never a lock.

Second, the leading underscore. Naming a helper _fisher_yates or _seed also changes exactly one behaviour: names starting with _ are skipped by import * when there's no __all__. Attribute access still reaches them perfectly, so playlist.engine._fisher_yates reads out the function without complaint. The underscore is a message to humans and linters: touch this and you're on your own; I may delete it. Be exact about the mechanism, because half-truths here breed superstition. Python has no access control. There is no private keyword and no runtime that forbids reaching a name. "Private" in Python means precisely two non-enforcing things: omitted from __all__, and marked with an underscore, plus the social contract that goes with them. The interpreter will hand out every name you ask for, because the convention is a fence, not a wall.

Here's why the facade earns its keep, in milliseconds. Suppose __init__.py eagerly imports all six submodules, and each drags in a heavy dependency costing about 200 ms on a cold cache. That's 6 × 200 = 1200 ms — over a second — burned before import playlist even returns. A lean facade that re-exports three light names pays perhaps 5 ms, and defers the heavy work until something actually reaches for it.

You've published names. But those from .engine import Player lines with a leading dot are doing something specific — and they behave differently depending on how the file is run. Time to understand the dot. →

03Absolute vs relative imports

You reorganise: engine.py moves into a subfolder, and suddenly import storage throws ModuleNotFoundError, even though storage.py is right there next to it. You patch it to from .storage import save, and imported through the package, it works. Relieved, you run python engine.py to test that file alone, and it detonates: ImportError: attempted relative import with no known parent package. Same line, same file, opposite outcomes. You cannot move on until you understand why.

# ---------- jukebox/__init__.py ----------
print("jukebox/__init__.py running")

# ---------- jukebox/tracks.py ----------
LIBRARY = ["Blinding Lights", "Titanium", "Levels"]

# ---------- jukebox/report.py ----------
print("report __package__ =", repr(__package__))

from .tracks import LIBRARY

print("report sees", len(LIBRARY), "tracks")

# ---------- standing in myproject/, launch the SAME file two ways ----------
$ python jukebox/report.py
$ python -m jukebox.report

$ python jukebox/report.py
report __package__ = None
Traceback (most recent call last):
  File "C:\tmp\myproject\jukebox\report.py", line 3, in <module>
    from .tracks import LIBRARY
ImportError: attempted relative import with no known parent package

$ python -m jukebox.report
jukebox/__init__.py running
report __package__ = 'jukebox'
report sees 3 tracks

There are two import styles, and they resolve through completely different machinery. An absolute import names the full dotted path from a top-level entry on sys.path: from playlist.storage import save. The finder walks it segment by segment — find playlist, descend into it, find storage, pull out save. It depends on nothing but sys.path and the name.

A relative import, like from .storage import save or from ..util import log, never touches a filesystem name directly. The compiler encodes the leading dots as a level count: one dot is level=1, two dots level=2. At runtime the interpreter resolves the target from the current module's __package__ string. It walks level-1 dots up from it, then appends the name after the dots. The whole thing is relative to one attribute, __package__, and hangs entirely on that string being set correctly.

Count it yourself, because the dots are just arithmetic. Say playlist/engine/core.py is imported, so its __package__ is "playlist.engine". Now from ..model import Track carries two dots, so level=2. The resolver walks level-1 = 1 dot up from playlist.engine, landing on playlist. Then it appends model, giving playlist.model. Every dot past the first is one step up the tree.

# inside module  playlist.audio.engine   →  __package__ = "playlist.audio"
from ..util import log        # level = 2 (two dots)

# resolver: start at __package__      -> "playlist.audio"
#           strip (level-1)=1 name     -> "playlist"
#           append the target "util"   -> "playlist.util"
# result: from playlist.util import log

Now the crash makes sense. When you import playlist.engine, the machinery stamps engine.__package__ = "playlist", so .storage resolves to playlist.storage — clean. But when you run python engine.py, that file is loaded as the startup module: its __name__ becomes "__main__" and its __package__ becomes "" — the empty string. The dot resolver is asked to walk up from nothing. There is no parent package string to strip a name from, so the interpreter can only raise: attempted relative import with no known parent package. The line isn't wrong. The context is missing. This is the exact hole that python -m, the next section, was built to fill.

you type
# ---------- jukebox/report.py ----------
from . import tracks                 # the sibling MODULE object
from .tracks import as_clock         # a NAME out of the sibling

print("__name__    =", __name__)
print("__package__ =", __package__)
print("total       =", as_clock(785), "|", tracks.as_clock(785))


# ---------- jukebox/formats/m3u.py ----------
from ..tracks import as_clock        # level=2: strip one name, then append

print("__name__    =", __name__)
print("__package__ =", __package__)
print("line        =", "#EXTINF:" + as_clock(245))


# ---------- the terminal, standing in myproject/ ----------
$ python -m jukebox.report
$ python jukebox/report.py
$ python -m jukebox.formats.m3u

you see
$ python -m jukebox.report
__name__    = __main__
__package__ = jukebox
total       = 13:05 | 13:05

$ python jukebox/report.py
Traceback (most recent call last):
  File "C:\tmp\myproject\jukebox\report.py", line 2, in <module>
    from . import tracks                 # the sibling MODULE object
    ^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

$ python -m jukebox.formats.m3u
__name__    = __main__
__package__ = jukebox.formats
line        = #EXTINF:4:05

So which style should you use? The trade is real, and it's about who you're protecting. Relative imports say "my neighbour." from .storage import save means the sibling module in whatever package I live in, and it never spells the top package's name. So if you rename playlist to setlist tomorrow, every relative import inside keeps working untouched, which makes a package renamable and relocatable. Absolute imports say the full address. from playlist.storage import save is explicit and unambiguous, and it always resolves under -m, so it's the safe default for anything you might run as a program. The discipline that keeps you out of trouble is a single rule: never rely on a relative import in a file you also run directly as a script. For library code deep in the package, relative is lovely. For an entry point, use absolute, or run it through -m.

# ---------- jukebox/tracks.py ----------
"""tracks.py -- one function, about to be imported five ways."""
print("tracks.py body running")


def as_clock(seconds):
    minutes, secs = divmod(seconds, 60)
    return f"{minutes}:{secs:02d}"


# ---------- jukebox/__init__.py ----------
"""jukebox -- the facade re-exports as_clock."""
from .tracks import as_clock

__all__ = ["as_clock"]


# ---------- import_styles.py ----------
# import_styles.py -- five spellings, ONE function object
import sys

import jukebox                              # 1. the package
import jukebox.tracks                       # 2. the submodule, dotted
from jukebox import tracks                  # 3. the submodule, bare name
from jukebox.tracks import as_clock         # 4. the function itself
from jukebox import as_clock as viafacade   # 5. through __init__.py

spellings = [
    ("jukebox.tracks.as_clock", jukebox.tracks.as_clock),
    ("tracks.as_clock", tracks.as_clock),
    ("as_clock", as_clock),
    ("viafacade", viafacade),
    ("jukebox.as_clock", jukebox.as_clock),
]

for name, fn in spellings:
    print(f"{name:<24} is as_clock -> {fn is as_clock}   {hex(id(fn))}")

print()
print("sys.modules keys  :", [k for k in sys.modules if k.startswith("jukebox")])
print("one module object :", jukebox.tracks is tracks is sys.modules["jukebox.tracks"])
print("where it lives    :", as_clock.__module__)
print("what it computes  :", as_clock(245), viafacade(245), tracks.as_clock(245))

$ python import_styles.py
tracks.py body running
jukebox.tracks.as_clock  is as_clock -> True   0x18927226520
tracks.as_clock          is as_clock -> True   0x18927226520
as_clock                 is as_clock -> True   0x18927226520
viafacade                is as_clock -> True   0x18927226520
jukebox.as_clock         is as_clock -> True   0x18927226520

sys.modules keys  : ['jukebox.tracks', 'jukebox']
one module object : True
where it lives    : jukebox.tracks
what it computes  : 4:05 4:05 4:05

If relative imports need a real package context, and running a file directly destroys that context, there must be a way to run a package as a package. There is — one command, and it's the front door your project has been missing. →

04python -m and __main__.py: running a package

You want your project to run with one clean command, python -m playlist, not python playlist/some_file_i_happened_to_pick.py. And you're done with the relative-import crash from the last section. It turns out both problems have a single answer. It's a flag you've probably typed without knowing what it repairs. To use it well, you need to see how differently CPython starts up in the two cases.

When you run python playlist/engine.py, the interpreter takes the script path. It sets sys.path[0] to the script's own directory (playlist/), then executes the file as the startup module, with __name__ = "__main__" and __package__ = "". That's the broken world of Section 3. There's no parent package, so every relative import inside dies. And because playlist/ itself is on the path but its parent isn't, import playlist may not even resolve.

When you run python -m playlist, a different subsystem takes over: it's called runpy, and it does three things in order:

# 1. put the CURRENT WORKING DIRECTORY on sys.path[0]
#    so the top-level package 'playlist' is importable
sys.path.insert(0, os.getcwd())

# 2. IMPORT the package normally -> runs playlist/__init__.py,
#    so 'playlist' is a real, correctly-initialised package
import playlist

# 3. find playlist/__main__.py and exec it AS the startup module,
#    but with package context preserved:
#      __name__    = "__main__"     (so the guard fires)
#      __package__ = "playlist"     (so relative imports RESOLVE)

Step 3 is the whole trick, and it's genuinely clever. The entry module runs under the name __main__, so if __name__ == "__main__": fires and it behaves like the program's start. And it carries __package__ = "playlist", so relative imports inside it resolve exactly as they would in any library module. Both truths hold at once: I am the program and I live inside a package. A plain script can never have both. That is why python -m is the invocation that makes from .engine import Player work in an entry point.

The file that runs is __main__.py, and now its role is clear. It is the package's designated entry point, the folder-level equivalent of the if __name__ == "__main__": block you'd put at the bottom of a single script. You don't pick a random file to be the front door. You write playlist/__main__.py, and python -m playlist finds it by convention.

you type
# ---------- jukebox/__init__.py ----------
print("[1] __init__.py  -- the package builds itself first")


# ---------- jukebox/tracks.py ----------
print("[2] tracks.py    -- imported by the entry point")

LIBRARY = ["Blinding Lights", "Titanium", "Levels"]


def as_clock(seconds):
    minutes, secs = divmod(seconds, 60)
    return f"{minutes}:{secs:02d}"


if __name__ == "__main__":
    print("tracks.py self-test:", as_clock(245))


# ---------- jukebox/__main__.py ----------
import sys

from .tracks import LIBRARY, as_clock     # resolves ONLY with a package context

print("[3] __main__.py  -- and only now the program runs")
print("    __name__    =", __name__)
print("    __package__ =", __package__)
print("    sys.argv[1:] =", sys.argv[1:])

if sys.argv[1:2] == ["play"]:
    print("    now playing :", LIBRARY[int(sys.argv[2]) - 1])
else:
    for i, title in enumerate(LIBRARY, start=1):
        print(f"    {i}. {title}")


# ---------- the terminal, standing in myproject/ ----------
$ python -m jukebox
$ python -m jukebox play 2
$ python -m jukebox.tracks
$ python jukebox/__main__.py

you see
$ python -m jukebox
[1] __init__.py  -- the package builds itself first
[2] tracks.py    -- imported by the entry point
[3] __main__.py  -- and only now the program runs
    __name__    = __main__
    __package__ = jukebox
    sys.argv[1:] = []
    1. Blinding Lights
    2. Titanium
    3. Levels

$ python -m jukebox play 2
[1] __init__.py  -- the package builds itself first
[2] tracks.py    -- imported by the entry point
[3] __main__.py  -- and only now the program runs
    __name__    = __main__
    __package__ = jukebox
    sys.argv[1:] = ['play', '2']
    now playing : Titanium

$ python -m jukebox.tracks
[1] __init__.py  -- the package builds itself first
[2] tracks.py    -- imported by the entry point
tracks.py self-test: 4:05

$ python jukebox/__main__.py
Traceback (most recent call last):
  File "C:\tmp\myproject\jukebox\__main__.py", line 4, in <module>
    from .tracks import LIBRARY, as_clock     # resolves ONLY with a package context
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package

from .engine import Player       # resolves under -m, would crash as a script
from .storage import load

print("__name__    =", __name__)     # __main__
print("__package__ =", __package__)  # playlist  — both true at once

tracks = load("library.json")
Player(tracks).run()

This also demystifies __name__ itself, the most cargo-culted line in Python. __name__ is simply the string the loader stamped on the module when it loaded it. It's "playlist.engine" when imported as a library, and "__main__" when it's the startup module. So if __name__ == "__main__": is not magic. It's a plain string comparison asking one question: am I the program being run, or am I being imported by someone else? Run this only when I'm the program, and stay quiet when I'm a library. The guard has meant that, and only that, all along.

$ python -m jukebox
jukebox/__init__.py running
C:\Python312\python.exe: No module named jukebox.__main__; 'jukebox' is a package and cannot be directly executed

Traceback (most recent call last):
  File "C:\tmp\myproject\run.py", line 1, in <module>
    import jukebox.storage
  File "C:\tmp\myproject\jukebox\storage.py", line 1, in <module>
    from ..settings import DB_PATH     # "up one folder on disk", surely?
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import beyond top-level package

Traceback (most recent call last):
  File "C:\tmp\myproject\run.py", line 1, in <module>
    from jukebox import Player
ImportError: cannot import name 'Player' from 'jukebox' (C:\tmp\myproject\jukebox\__init__.py)

# ---------- myproject/jukebox/__init__.py ----------
"""jukebox -- a package with one sore spot."""


# ---------- myproject/jukebox/tracks.py ----------
"""tracks.py -- the leaf."""
LIBRARY = [("Blinding Lights", 200), ("Titanium", 245), ("Levels", 340)]


def as_clock(seconds):
    minutes, secs = divmod(seconds, 60)
    return f"{minutes}:{secs:02d}"


# ---------- myproject/jukebox/report.py ----------
from .tracks import LIBRARY, as_clock


def summary():
    total = sum(secs for _title, secs in LIBRARY)
    return f"{len(LIBRARY)} tracks, {as_clock(total)}"


if __name__ == "__main__":
    print(summary())


# ------------- $ python jukebox/report.py -------------
Traceback (most recent call last):
  File "C:\tmp\myproject\jukebox\report.py", line 2, in <module>
    from .tracks import LIBRARY, as_clock
ImportError: attempted relative import with no known parent package


# The three answers.
#
# 1. __package__. Launched as a script, report.py IS the startup module, so the
#    interpreter sets __name__ = "__main__" and __package__ = "". The dot
#    resolver is handed an empty base and has nothing to strip a name from.
#
# 2. It wanted "jukebox". Import it instead -- `import jukebox.report` -- and the
#    loader stamps __package__ = "jukebox" from the dotted name it resolved. The
#    string does not come from the folder on disk; it comes from the import.
#
# 3. Because the script form never consults the working directory. sys.path[0]
#    is the folder of the FILE you launched, which is jukebox/ every time. Where
#    you happen to be standing is not an input.


# ------------- FIX A: change the launch, not the code -------------
$ python -m jukebox.report
3 tracks, 13:05

$ python -c "from jukebox.report import summary; print(summary())"
3 tracks, 13:05


# ------------- FIX B: absolute import. This is the surprise. -------------
# ---------- jukebox/report_abs.py ----------
from jukebox.tracks import LIBRARY, as_clock

if __name__ == "__main__":
    total = sum(secs for _title, secs in LIBRARY)
    print(f"{len(LIBRARY)} tracks, {as_clock(total)}")


$ python jukebox/report_abs.py
Traceback (most recent call last):
  File "C:\tmp\myproject\jukebox\report_abs.py", line 2, in <module>
    from jukebox.tracks import LIBRARY, as_clock
ModuleNotFoundError: No module named 'jukebox'

$ python -m jukebox.report_abs
3 tracks, 13:05

# Absolute imports do NOT rescue the script form. The error simply changes its
# name. sys.path[0] is jukebox/ -- the package's INSIDE -- so the top-level name
# `jukebox` is not on the path at all. Different message, same missing context.


# ------------- FIX C: the sys.path hack, and what it really costs -------------
# ---------- jukebox/report_hack.py ----------
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parent))   # jukebox/ itself
import tracks                                              # now a TOP-LEVEL name
import jukebox.tracks                                      # and the real one

print("bare import works :", tracks.as_clock(785))
print("same module?      :", tracks is jukebox.tracks)
print("two keys in cache :", [k for k in sys.modules if k.endswith("tracks")])
tracks.LIBRARY.append(("Ghost Track", 1))
print("edit through one  :", len(tracks.LIBRARY), "vs", len(jukebox.tracks.LIBRARY))


$ python -m jukebox.report_hack
bare import works : 13:05
same module?      : False
two keys in cache : ['tracks', 'jukebox.tracks']
edit through one  : 4 vs 3

# There it is. The hack "works" -- and it loaded tracks.py a SECOND time, under a
# second name, into a second module object. Two Track lists, two sets of globals,
# and an append through one that the other never sees. Every class defined in
# there now exists twice, so isinstance() starts returning False for objects that
# obviously match. This is the bug that eats an afternoon.


# The ranking, and why.
#   A  (python -m)          SHIP THIS. No code changed, no name leaked, and the
#                           package keeps exactly one identity in sys.modules.
#   B  (absolute import)    Fine as a style for entry points, but it is not a fix
#                           for the script form -- run it with -m regardless.
#   C  (sys.path.insert)    Never. It converts a private submodule into a global
#                           name and buys you two copies of one module.
#
# The one honest sentence to take away:
#   A relative import needs a package context, and only an IMPORT can create one.
#   Launching a file cannot -- so launch the package with -m instead.

You have a package that imports as one name, publishes a clean surface, and runs from a real front door. One structural landmine remains — the one where two perfectly correct imports poison each other. →

05Circular imports and the half-built module

You split a class across two files. model.py needs a type from engine.py, and engine.py needs a type from model.py. Then import playlist dies with a line that sounds like an accusation, the circular import: ImportError: cannot import name 'Player' from partially initialized module 'playlist.engine' (most likely due to a circular import). Nothing is misspelled. Both files are correct in isolation. You could shuffle import lines until it stops crashing, but that's superstition. You want to know precisely why a valid pair of imports poisons each other, and the exact fix.

This builds directly on Section 1, on that one line you were told to remember. Import inserts an empty module object into sys.modules before running the module's code. Watch the timeline of import playlist.model when the two files import each other at the top:

t0  import model      -> sys.modules["...model"]  = <empty>   # seed, before code
t1  model line 1:  from .engine import Player
t2    import engine   -> sys.modules["...engine"] = <empty>   # seed, before code
t3    engine line 1: from .model import Track
t4      model already in sys.modules  -> DO NOT re-run it

Read t4 slowly, because it's the crux. When engine does from .model import Track, the machinery finds model already in sys.modules, the empty seed from t0. Correctly, it does not re-run it, because re-running would loop forever, so it hands engine the half-built model object exactly as it stands. But at t4, model has only executed its first line and never reached class Track, so the name Track isn't in model's namespace yet. from .model import Track demands that name right now, can't find it, and raises cannot import name 'Track' from partially initialized module. The module isn't broken — it's just unfinished, caught mid-execution by a neighbour who reached across too early.

Now the distinction that unlocks everything — the difference between two import forms you probably thought were interchangeable:

from .model import Track   # needs the NAME 'Track' to exist RIGHT NOW, at import time
import playlist.model     # binds the MODULE object; you touch .Track LATER, at call time

from .model import Track copies a name out of model's namespace at the instant the import runs, during module execution, when model may still be half-built. import playlist.model only binds the module object, which exists from t0, empty or not. You don't read model.Track until later, at call time, and by then both module bodies have finished and Track is present. So the cycle itself is not fatal. What's fatal is depending on a not-yet-executed name during module execution. Change when you need the name, and the cycle becomes harmless.

That understanding gives four fixes, ranked worst to best:

# 1. move the import to CALL time — runs after both bodies finished
def shuffle(self):
    from .model import Track   # local import, deferred until called
    ...

# 2. bind the module, read the attribute lazily
import playlist.model
# ...later...  playlist.model.Track

# 3. type-checking only — exists for the checker, never at runtime
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from .model import Track   # used only in string annotations

# 4. BEST — break the cycle: extract Track into a leaf module both import

Every mechanism is now on the table: the package name, the facade, the dot, the front door, the cycle. Time to cash them all in — turn the sprawling playlist into a real, layered, cycle-proof program. →

06Restructuring the playlist into a package

Everything so far has been mechanism. Now you spend it. The playlist has swollen into one bloated file mixing data classes, playback logic, and file-saving. You can't test the storage in isolation, can't hand a teammate "just the engine," and can't change the save format without risking the shuffle. You want to convert the pile into a program: a package with clear modules, one public surface, one entry point, and, by construction, no import cycles. This is the exact shape the capstone in Ch 23 will ship.

The single design decision that makes all of it fall into place is dependency direction: decide which module depends on which, and make every arrow point the same way, toward a foundation. Here's the layout:

playlist/
  __init__.py     # facade: re-exports the public names + __all__
  model.py        # LEAF — Track, Playlist. imports nothing internal
  storage.py      # imports model. save/load to JSON
  engine.py       # imports model. Player, shuffle, next/prev
  __main__.py     # entry point: wires storage + engine. python -m playlist

Look at the arrows. model.py is a leaf: pure data, the Track and Playlist dataclasses, importing nothing else in the package. storage.py imports model, one direction, and engine.py imports model, one direction too. Neither storage nor engine imports the other. They meet only at the top, in __main__.py, which wires them together. Because every arrow points down toward the model leaf, there is no cycle possible — it's not "we avoided one," but "the shape forbids one." This is the deep lesson of Section 5 turned into architecture: you don't dodge circular imports with clever import placement, you make them structurally impossible by layering.

you type
# ---------- tree.py  (at the repo root) ----------
import os

SKIP = {"__pycache__", ".git", ".venv"}

for root, dirs, files in os.walk("."):
    dirs[:] = sorted(d for d in dirs if d not in SKIP)
    depth = root.count(os.sep)
    mark = "  <- package" if "__init__.py" in files else ""
    print("  " * depth + os.path.basename(root or ".") + "/" + mark)
    for name in sorted(files):
        if not name.endswith(".pyc"):
            print("  " * (depth + 1) + name)


# ---------- the terminal ----------
$ python tree.py
$ cd jukebox  &&  python -m jukebox
$ cd ..       &&  python -m jukebox
$ python -m unittest discover -s tests -t .

you see
$ python tree.py
./
  README.md
  pyproject.toml
  tree.py
  jukebox/  <- package
    __init__.py
    __main__.py
    storage.py
    tracks.py
    formats/  <- package
      __init__.py
      json_io.py
  tests/  <- package
    __init__.py
    test_tracks.py

$ cd jukebox  &&  python -m jukebox
C:\...\python.exe: No module named jukebox

$ cd ..       &&  python -m jukebox
  Blinding Lights - The Weeknd (3:20)
  Titanium - David Guetta (4:05)
jukebox 1.0.0

$ python -m unittest discover -s tests -t .
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

# model.py   — the leaf, imports nothing internal
from dataclasses import dataclass

# storage.py — one arrow down to model
from .model import Track, Playlist

# engine.py  — one arrow down to model
from .model import Track, Playlist

# __init__.py — the facade on top
from .model import Track, Playlist
from .engine import Player
from .storage import save, load
__all__ = ["Track", "Playlist", "Player", "save", "load"]

Every prior mechanism is doing a job here, in situ. The relative imports (from .model import ...) mean the package is renamable and works under -m, from Section 3. Those imports resolve because __package__ is set correctly when the package is imported, and __main__.py keeps that context under python -m playlist, from Section 4. __init__.py re-exports the public trio behind a stable facade and declares __all__, from Section 2. And the acyclic layering is the structural cure for the poisoning of Section 5. None of the modules is meant to be run directly. The only front door is __main__.py.

# ---------- the layout, after ----------
myproject/
    jukebox/
        __init__.py     # the facade
        tracks.py       # ch 19's tracks.py, unedited
        storage.py      # ch 19's storage.py, ONE import line changed
        __main__.py     # ch 19's jukebox.py, now the front door


# ---------- jukebox/tracks.py ----------
"""tracks.py -- what a track and a playlist ARE. No I/O, no menu."""
from dataclasses import dataclass


@dataclass
class Track:                                  # ch 12's dataclass, unmoved
    title: str
    artist: str
    seconds: int
    plays: int = 0

    def __str__(self):
        minutes, secs = divmod(self.seconds, 60)
        return f"{self.title} - {self.artist} ({minutes}:{secs:02d})"


class Playlist:
    def __init__(self, name, tracks):
        self.name, self.tracks = name, list(tracks)

    def __len__(self):
        return len(self.tracks)

    def __iter__(self):
        return iter(self.tracks)

    def __repr__(self):
        minutes, secs = divmod(sum(t.seconds for t in self), 60)
        return f"Playlist({self.name!r}, {len(self)} tracks, {minutes}:{secs:02d})"

    def play(self, number):
        track = self.tracks[number - 1]
        track.plays += 1
        return track


# ---------- jukebox/storage.py ----------
"""storage.py -- the only file that knows what JSON is. Chapter 18, boxed."""
import json
from pathlib import Path

from .tracks import Track, Playlist          # ch 19 said: from tracks import ...

DB = Path("library.json")


def save(playlist, path=DB):
    payload = {"version": 1, "name": playlist.name,
               "tracks": [vars(t) for t in playlist]}
    with open(path, "w", encoding="utf-8", newline="\n") as f:
        json.dump(payload, f, indent=2)
    return path.stat().st_size


def load(path=DB):
    with open(path, "r", encoding="utf-8") as f:
        data = json.load(f)
    return Playlist(data["name"], [Track(**row) for row in data["tracks"]])


# ---------- jukebox/__init__.py ----------
"""jukebox -- the package. Chapter 19's three flat files, folded under one name."""
from .tracks import Track, Playlist
from .storage import save, load

__all__ = ["Track", "Playlist", "save", "load"]
__version__ = "2.0.0"


# ---------- jukebox/__main__.py ----------
"""__main__.py -- the front door. Chapter 19's jukebox.py, promoted."""
from . import __version__                    # from the facade above it
from .storage import DB, load, save
from .tracks import Track, Playlist


def seed():
    return Playlist("Friday Night", [
        Track("Blinding Lights", "The Weeknd", 200),
        Track("Titanium", "David Guetta", 245),
        Track("Levels", "Avicii", 340),
    ])


def report(playlist):
    print(repr(playlist))
    for i, track in enumerate(playlist, start=1):
        print(f"  {i}. {track}  played {track.plays}x")


def main():
    print(f"jukebox {__version__}  |  __name__ = {__name__}  |  __package__ = {__package__}")
    night = load() if DB.exists() else seed()
    print("loaded from disk:", DB.exists())
    report(night)

    print()
    for number in (2, 1, 2):
        print("  now playing", night.play(number))

    size = save(night)
    print()
    print(f"saved {size} bytes to {DB.name}")


if __name__ == "__main__":
    main()

$ python -m jukebox
jukebox 2.0.0  |  __name__ = __main__  |  __package__ = jukebox
loaded from disk: False
Playlist('Friday Night', 3 tracks, 13:05)
  1. Blinding Lights - The Weeknd (3:20)  played 0x
  2. Titanium - David Guetta (4:05)  played 0x
  3. Levels - Avicii (5:40)  played 0x

  now playing Titanium - David Guetta (4:05)
  now playing Blinding Lights - The Weeknd (3:20)
  now playing Titanium - David Guetta (4:05)

saved 392 bytes to library.json

$ python -m jukebox
jukebox 2.0.0  |  __name__ = __main__  |  __package__ = jukebox
loaded from disk: True
Playlist('Friday Night', 3 tracks, 13:05)
  1. Blinding Lights - The Weeknd (3:20)  played 1x
  2. Titanium - David Guetta (4:05)  played 2x
  3. Levels - Avicii (5:40)  played 0x

  now playing Titanium - David Guetta (4:05)
  now playing Blinding Lights - The Weeknd (3:20)
  now playing Titanium - David Guetta (4:05)

saved 392 bytes to library.json

$ python -c "from jukebox import Track, load; print(load().name, '|', Track('X', 'Y', 61))"
Friday Night | X - Y (1:01)

# ---------- the tree, after ----------
$ python tree.py
./
  tree.py
  jukebox/   <- package
    __init__.py
    __main__.py
    storage.py
    tracks.py
    formats/   <- package
      __init__.py
      json_io.py


# ---------- jukebox/tracks.py ----------
"""tracks.py -- the leaf."""
from dataclasses import dataclass


@dataclass
class Track:
    title: str
    artist: str
    seconds: int

    def __str__(self):
        minutes, secs = divmod(self.seconds, 60)
        return f"{self.title} - {self.artist} ({minutes}:{secs:02d})"


# ---------- jukebox/formats/json_io.py ----------
"""json_io.py -- the only file in the project that imports json."""
import json

from ..tracks import Track            # two dots: up to jukebox, then down


def write_tracks(tracks, path):
    rows = [vars(t) for t in tracks]
    with open(path, "w", encoding="utf-8", newline="\n") as f:
        json.dump(rows, f, indent=2)
    return path


def read_tracks(path):
    with open(path, "r", encoding="utf-8") as f:
        return [Track(**row) for row in json.load(f)]


# ---------- jukebox/formats/__init__.py ----------
"""formats -- one module per file shape. A package inside a package."""
from .json_io import read_tracks, write_tracks

__all__ = ["read_tracks", "write_tracks"]


# ---------- jukebox/storage.py ----------
"""storage.py -- delegates the file shape to formats/, keeps the policy."""
from .formats import read_tracks, write_tracks

DB = "library.json"


def save(tracks, path=DB):
    return write_tracks(tracks, path)


def load(path=DB):
    return read_tracks(path)


# ---------- jukebox/__init__.py ----------
"""jukebox -- now with a formats subpackage."""
from .tracks import Track
from .storage import load, save

__all__ = ["Track", "load", "save"]


# ---------- jukebox/__main__.py ----------
"""__main__.py -- the front door."""
from .storage import load, save
from .tracks import Track

save([Track("Blinding Lights", "The Weeknd", 200),
      Track("Titanium", "David Guetta", 245)])

for track in load():
    print(" ", track)

from .formats import json_io
print("json_io.__package__ =", json_io.__package__)
print("json_io.__name__    =", json_io.__name__)


# ------------- $ python -m jukebox -------------
  Blinding Lights - The Weeknd (3:20)
  Titanium - David Guetta (4:05)
json_io.__package__ = jukebox.formats
json_io.__name__    = jukebox.formats.json_io


# ------------- the dot arithmetic, checked -------------
# In jukebox/formats/json_io.py, __package__ is "jukebox.formats".
#   from ..tracks import Track   ->  level = 2
#                                    strip level-1 = 1 name  ->  "jukebox"
#                                    append "tracks"         ->  "jukebox.tracks"
# One dot would have resolved to "jukebox.formats.tracks", which does not exist.


$ python -c "import jukebox.formats.json_io as j; print(j.__name__, j.__package__)"
jukebox.formats.json_io | jukebox.formats


# The rename question:
#   ZERO import lines inside the package. Every one of them is relative, so not
#   one of them spells `jukebox`. Rename the folder to setlist, run
#   `python -m setlist`, and it works untouched. Only code OUTSIDE the package --
#   which said `from jukebox import ...` -- has to change.
#
# And exactly one file imports json: jukebox/formats/json_io.py. Adding an
# m3u.py beside it later needs no edit to storage.py, because storage.py asks
# formats/ for a shape and never for a format.

The payoff is the seams, and this is why the layering is worth the ceremony. Because storage.py isolates all the I/O, you can swap JSON for SQLite later without touching a line of engine.py. The engine never knew where tracks came from, only that load() returns them. Because engine.py holds the algorithm, you can swap the shuffle strategy without touching storage. And model.py is the stable contract both sides agree on: the Track shape that storage produces and engine consumes. Layering doesn't just prevent cycles. It hands you a program you can change one piece at a time. That's exactly the surface Ch 23's capstone builds on.

# ---------- the tree ----------
$ python tree.py
./
  README.md
  pyproject.toml
  tree.py
  habits/   <- package
    __init__.py
    __main__.py
    model.py
    storage.py
    streaks.py
  tests/   <- package
    __init__.py
    test_streaks.py

# The arrows, all pointing down at the leaf:
#
#            __main__.py            (the front door: wires, owns no logic)
#            /    |     \
#     storage  streaks  model
#            \    |     /
#              model.py             (the LEAF: imports nothing internal)
#
# __init__.py sits above all of it as the facade. No module imports __main__,
# and no module imports __init__ except __main__ (for the version string), so
# there is no cycle available to write.


# ---------- habits/model.py ----------
"""model.py -- LEAF. What a habit IS. Imports nothing internal."""
from dataclasses import dataclass, field


@dataclass
class Habit:
    name: str
    done: list = field(default_factory=list)     # ISO dates, most recent last


# ---------- habits/streaks.py ----------
"""streaks.py -- pure logic. One arrow down to model, no I/O at all."""
from datetime import date, timedelta

from .model import Habit


def streak(habit, today):
    """Consecutive days ending today (or yesterday) -- 0 if the chain is broken."""
    days = {date.fromisoformat(d) for d in habit.done}
    cursor = today if today in days else today - timedelta(days=1)
    count = 0
    while cursor in days:
        count += 1
        cursor -= timedelta(days=1)
    return count


# ---------- habits/storage.py ----------
"""storage.py -- the only file that knows what JSON is. One arrow down to model."""
import json
from pathlib import Path

from .model import Habit

DB = Path("habits.json")


def load(path=DB):
    if not path.exists():
        return []
    with open(path, "r", encoding="utf-8") as f:
        return [Habit(**row) for row in json.load(f)]


def save(habits, path=DB):
    with open(path, "w", encoding="utf-8", newline="\n") as f:
        json.dump([vars(h) for h in habits], f, indent=2)
    return path


# ---------- habits/__init__.py ----------
"""habits -- track a streak. Run it with `python -m habits`."""
from .model import Habit
from .storage import load, save
from .streaks import streak

__all__ = ["Habit", "load", "save", "streak"]
__version__ = "1.0.0"


# ---------- habits/__main__.py ----------
"""__main__.py -- the front door. Wires storage + streaks; owns no logic."""
import sys
from datetime import date, timedelta

from .model import Habit
from .storage import load, save
from .streaks import streak

TODAY = date(2026, 8, 16)                    # fixed so the output is reproducible


def seed():
    days = [(TODAY - timedelta(days=n)).isoformat() for n in (3, 2, 1, 0)]
    return [Habit("read 20 pages", days), Habit("run 5k", days[-2:])]


def main(argv):
    habits = load() or seed()
    if argv[:1] == ["add"]:
        habits.append(Habit(argv[1], [TODAY.isoformat()]))
    for habit in habits:
        print(f"  {habit.name:<16} {streak(habit, TODAY)} day streak")
    save(habits)


if __name__ == "__main__":
    main(sys.argv[1:])


# ---------- tests/__init__.py ----------
"""tests -- a package, so `unittest discover` can import it."""


# ---------- tests/test_streaks.py ----------
"""The logic is testable because streaks.py never touches the disk."""
import unittest
from datetime import date

from habits import Habit, streak


class StreakTests(unittest.TestCase):
    TODAY = date(2026, 8, 16)

    def test_unbroken_chain(self):
        h = Habit("read", ["2026-08-14", "2026-08-15", "2026-08-16"])
        self.assertEqual(streak(h, self.TODAY), 3)

    def test_broken_chain_resets(self):
        h = Habit("read", ["2026-08-10", "2026-08-16"])
        self.assertEqual(streak(h, self.TODAY), 1)

    def test_never_done(self):
        self.assertEqual(streak(Habit("run"), self.TODAY), 0)


# ------------- both commands, from the repo root -------------
$ python -m habits
  read 20 pages    4 day streak
  run 5k           2 day streak

$ python -m habits add "walk the dog"
  read 20 pages    4 day streak
  run 5k           2 day streak
  walk the dog     1 day streak

$ python -m unittest discover -s tests -t .
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK


# The two defences.
#
# 1. streak() takes `today` because a function that reads the clock cannot be
#    tested -- its answer changes at midnight and you cannot ask it about a
#    Tuesday in 2026. Passing the date makes the rule a pure function of its
#    inputs, which is why all three tests are three lines each. The clock is
#    read in exactly one place, __main__.py, where the program meets the world.
#
# 2. tests/ is outside habits/ because it must import the package the way a
#    stranger does -- `from habits import Habit, streak`, through the facade.
#    Put it inside and it ships to users, it can reach private names by
#    accident, and it stops testing the surface you actually promised.
#
# The layout rule, in one line:
#   Every arrow points at model.py, and only __main__.py is allowed to know
#   about more than one layer at a time.

Your program has a shape. Next volume-thread: the moment it needs to speak to anything outside itself — a file, a socket, another process — the objects stop crossing and the bytes begin. That's Ch 22. →