19Modules & import — find, compile, exec, cache
All through Volume 1 we lived inside a single process — one file the interpreter read top to bottom, every object at a real address, every name pointing straight at it. In this chapter your code outgrows that one file. The instant code here wants to reuse code over there, you hit the first real boundary of Volume 2. import is the machine that crosses it, and I want to slow down on it. Nearly everyone reads it as "load a file," and it is nothing of the sort. Here's the plan. We take a single import prices and watch it run four steps — find, compile, exec, cache. Those steps turn a source file on disk into a live module object whose __dict__ is just an ordinary namespace. And they run the file's top-level code exactly once per process, no matter how many times you ask for it. The whole way through we keep asking the one thing that actually matters — when you write import, what does Python do to that file, and how does it decide never to do it again? By the end you'll read a module as a plain object you can poke at, watch sys.modules enforce run-once, see why if __name__ == "__main__": works at all, and know exactly what from X import Y copies — and why it can quietly go stale.
01The need: one file that only grows
Let's start where every program starts: one file, and one file was the honest size of the problem. shop.py read a CSV of inventory off disk and ran the price math — discounts, tax, the rounding rule your accountant insists on. It also drove the menu loop a cashier uses to ring things up. It fit on a screen. It worked. Then it grew, the way working code always does, and now it's nine hundred lines. Watch what that costs you. To change the rounding rule you scroll past two hundred lines of storage code you didn't come to read. To fix the menu you edit dangerously close to the math. And because every name in the file shares one flat global namespace, a total in the menu and a total in the tax code are literally the same name. Rebind one and you've silently changed the other.
Then the real pain shows up. Your teammate is building a different program and wants just your price math — forty good lines she doesn't want to rewrite. But there's no way to hand someone forty lines of a file. A file is all-or-nothing. So she does the only thing the language seems to allow: she copies the forty lines into her file. It works, for now. Two weeks later you find a bug — a discount could go negative — and you fix it in your copy. Hers still has the bug. Nobody edited both, because nobody could: they're two independent copies now, and they've quietly drifted. That's the felt need in one sentence: a single file cannot be partially reused, and copy-paste reuse rots the instant either copy changes. What you want is one authoritative copy of the price code that many programs can pull in by name.
Name the constraint precisely before naming the cure. A running Python program is one process with one interpreter, but your source lives in many files on disk. Without import there is exactly one path from file to running program. The interpreter reads the single file you named on the command line — python shop.py — compiles it, and runs it. One file per run. So every function, class, and constant your program uses must live in that one file or be pasted into it. import is the mechanism that breaks that one-file ceiling. It lets code in file A cause code in file B to be found, compiled, run, and made reachable from A, all inside the same process, sharing the same heap and the very same object identities.
tracksThere is no step you are missing. A file on disk is the whole declaration, and its name minus .py is the import name.sys.path first, so a sibling file is found with no setup at all.tracks.py, then binds one name in your namespace to the resulting module object..pyimport "tracks.py" is a SyntaxError. Import takes a name, not a path — the finder turns the name into a path.tracks.as_clockNot a copy of the function. It is the same function object your file built, reached by a key lookup in tracks.__dict__.__pycache__/ appearsThe first run writes tracks.cpython-312.pyc beside your source. That is step 2 saving its answer; delete it any time.tracks.py once and every importer gets the fix.you type
# ---------- tracks.py ----------
# tracks.py -- your first module: one constant, one function
LIBRARY = "Friday Night"
def as_clock(seconds):
minutes, secs = divmod(seconds, 60)
return f"{minutes}:{secs:02d}"
# ---------- play.py ---------- (same folder, nothing else installed)
# play.py -- a SIBLING file, same folder, that imports it
import os
import sys
import tracks # the file name, minus .py. No path, no quotes.
print("module :", tracks.__name__)
print("from file:", os.path.basename(tracks.__file__))
print("constant :", tracks.LIBRARY)
print("function :", tracks.as_clock(245))
print("object :", type(tracks).__name__, "| cached:", "tracks" in sys.modules)
# ---------- the terminal ----------
$ python play.pyyou see
module : tracks
from file: tracks.py
constant : Friday Night
function : 4:05
object : module | cached: True
$ ls __pycache__
tracks.cpython-312.pycimport tracks.pyraisesModuleNotFoundError: No module named 'tracks.py'— the dot means package member, not file extension.- Name a file
json.pyorrandom.pyand your folder shadows the standard library, because the script's directory is searched first. - A hyphen is legal in a filename and illegal in an identifier.
my-utils.pycan never be imported by name; usemy_utils.py. - Running
python tracks.pyfrom a different directory still works, butimport tracksfrom there will not — the search path follows the launched script. - Import does not copy the file anywhere. Move or rename
tracks.pyand the next process cannot find it. tracks.__file__is an absolute path on most systems. Printos.path.basename(...)if you want output that looks the same on every machine.
To see exactly what import buys, watch two tempting non-solutions fail. The first is copy-paste, which we've met. It duplicates the source text, so fixes never propagate, and it gives each file its own separate discount function object — two functions that merely look alike. The second is closer, and instructive, because it actually runs the other file:
>>> exec(open('prices.py').read(), globals())
prices module loading... # a top-level print in prices.py fired — right now
>>> discount(100) # the name leaked STRAIGHT into your globals, no prefix
90.0
>>> exec(open('prices.py').read(), globals())
prices module loading... # ran the WHOLE file AGAIN — re-read from disk, re-printedexec does read and run the file. But it dumps every name straight into your globals with no namespace boundary — a helper in prices.py now silently overwrites any helper of yours. It has no memory, so a second call re-reads the file off disk and re-runs it top to bottom, firing every side effect again. import is the disciplined version of exactly this. It runs the file once, wraps the resulting names in a single named container so they cannot collide with yours, and remembers the result so the next import is nearly free. That discipline is not one feature but four mechanisms stacked in sequence — find, compile, exec, cache — and the rest of this chapter takes them apart one at a time.
import is not "load a file." It is the disciplined version of running another file: run it once, put its names in a named box, and remember the box. Every strange thing import ever does to you is one of those three promises being kept — or a step you didn't know was there.You've been told import "just works." Before you trust it, you need the shape of what it actually does — four steps, so that every weird symptom later maps to exactly one of them. →
02The four steps, once
You type import json and it just works. And "just works" is precisely the kind of magic that bites you later — when a module prints something the moment you import it, or imports the wrong file that happens to share a name, or seems to ignore an edit you just saved. To debug any of those you need the silhouette of what import does, because each symptom lives at exactly one stage. So before popping any hood, fix the shape. When you write import prices, the import machinery — ultimately the function importlib._bootstrap.__import__ — does four things in order.
STEP 1 — FIND. It first checks the cache, sys.modules (its own section, next). On a miss it walks sys.meta_path, a list of finders. The default finders walk sys.path — a list of directories — looking for prices.py, or a package directory, or a compiled C extension. A finder that recognises the name returns a ModuleSpec: a small record describing where the code is and how to load it, meaning which loader and what origin path. Find answers one question — "given this name, what file, and who loads it?"
STEP 2 — COMPILE. The loader reads the source bytes off disk. Compilation is expensive, so CPython caches its result. It looks in a __pycache__ directory for a matching .pyc. If a fresh one exists — matched against the source file's modification time and size, or optionally a hash of the source — it skips straight to loading that pre-compiled bytecode. Otherwise it calls compile() to turn the source string into a code object, the same bytecode you met when you disassembled a function. Then it writes a .pyc so the next process starts faster.
STEP 3 — EXEC. It creates a fresh, empty module object and — this detail is load-bearing — inserts it into sys.modules before executing anything. It seeds the module's __dict__ with __name__, __file__, __spec__, and friends. Then it execs the code object with that __dict__ serving as both globals and locals. Running the top-level code is what actually fills the module: every top-level def, class, and assignment binds a name in that dict.
STEP 4 — CACHE. The module is already sitting in sys.modules from step 3. On success it simply stays there, and the name prices is bound in your namespace to that same module object. The whole point of the cache is the load-bearing claim of this chapter: steps 1–3 happen at most once per process, because step 4's stored module short-circuits step 1 on every later import of the same name.
import line itself is not one of them; it is the declaration that asks for them. It is how a file states up front which modules it needs, so Python reads those lines before the rest of the file runs and makes the names available throughout. That is the whole reason imports have to sit at the top. Putting one inside a function would be like writing #include halfway down a C file — the name has to be known for the entire file, so the line has to come first.>>> def parse(text):
... import json # a statement -- so when does it run?
... return json.loads(text)
...
>>> import sys
>>> "json" in dir(), "json" in sys.modules # before any call
>>> parse('{"ok": 1}')
>>> "json" in dir(), "json" in sys.modules # after the call returned(False, False)
{'ok': 1}
(False, True)import is an ordinary executable statement: it runs when control reaches it, and it binds its name in whatever namespace is executing at that moment — module top level, a function's locals, one branch of an if. The transcript is only three lines of output, and it caught two separate things happening on one line of code. Read the last pair first. After parse() had already returned an answer, "json" in dir() is still False: the name json was a local, born when the call started and thrown away when it ended, exactly like any other name a function binds. Yet "json" in sys.modules flipped to True and stayed there, because the module was found, compiled, executed and cached, process-wide, the way it always is. So loading a module and binding a name for it are two different acts, and only the second one obeys scope. There is nothing left to argue with once you disassemble that function yourself: type import dis; dis.dis(parse) into the same session and you will find IMPORT_NAME followed immediately by STORE_FAST — the same fast-local store it uses for x = 1 — sitting in the instruction stream between the other opcodes, with no privileges and no special position. The #include instinct is a fair one to have arrived with, and worth naming rather than waving away: in C that line really is a textual splice performed before the compiler runs, and a Java import really is resolved at compile time. Python borrowed the keyword from that family and none of the mechanism. Now label the simplification before you take the freedom too far, because “anywhere is legal” is not the same as “anywhere is wise.” Top of file is still the right default, for a reason that has nothing to do with the interpreter: an import at the top fails at startup, loudly, on the first machine that is missing the package, while the same import buried in a branch that runs once a month fails the first time that branch is taken — a far worse place to find out. So the deferred import is a deliberate tool with two jobs, not a style you sprinkle: breaking an import cycle (the error clinic later in this chapter shows you the traceback it cures) and skipping a heavy optional dependency you may never touch. And be honest about its cost in both directions. It does not reload anything on the second call — section 5's cache turns that into a single dict lookup — but you do pay that lookup, and the name-binding after it, on every call, so it is cheap rather than free.These stages are not a metaphor. Every artifact between them is a real object you can hold and inspect. Import a small module and interrogate what find and compile produced:
>>> import prices
>>> type(prices)
<class 'module'> # step 3 built a real object of type 'module'
>>> prices.__name__
'prices'
>>> prices.__file__ # the file FIND located
'/home/you/shop/prices.py'
>>> prices.__spec__ # the ModuleSpec ticket, still attached
ModuleSpec(name='prices', loader=..., origin='/home/you/shop/prices.py')
>>> import os
>>> os.listdir('__pycache__') # step 2's compiled bytecode, written to disk
['prices.cpython-312.pyc'] # the name encodes the interpreter versionThat .pyc filename carries cpython-312 for a reason. Bytecode is not portable across interpreter versions, so each version keeps its own cache and never runs another's. And the run-once claim is directly observable. Put a top-level print in the module, import it twice in one session, and the print fires only on the first import. Steps 1–3 ran once; the second import was a pure cache lookup. Each of the next four sections pops the hood on exactly one of these stages. This one only fixes the silhouette, so you always know which stage a symptom belongs to.
__pycache__ never changes what your program does — the next import just recompiles from source and rewrites it. The .pyc is purely step 2 remembering its answer so process startup is faster. If it and the source ever disagree, Python trusts the .py: mtime and size mismatches invalidate the cache automatically.Step 3 built "a real object of type module." That phrase is the whole next section — a module is not a special language construct, it is an ordinary object with an ordinary __dict__. →
03A module is an object with a __dict__
You write json.dumps(data) and treat json as a special, keyword-ish thing — a namespace handed down from the language. But if a module is magic, you cannot reason about it. You cannot ask what is inside it, cannot store it in a variable, cannot explain why json.dumps and a plain attribute lookup on any object feel identical. Demystifying the module is what turns import from an incantation into a thing you can inspect and predict. So here is the chapter's core reframe, stated flatly: a module is an ordinary Python object. It is an instance of types.ModuleType, and like almost every object in the language it carries a __dict__ — a plain dict mapping name strings to value objects. That __dict__ is the module's global namespace.
Recall step 3 from the last section: when exec ran the module's top-level code, it used the module's __dict__ as the globals. So every top-level PI = 3.14159, every def area(r):, every class Circle: bound a key in that one dict. After import json, then, the name json in your namespace is bound to one ModuleType instance. And json.dumps is not special syntax at all — it is attribute access on that object, which for a module resolves by looking up the string 'dumps' in json.__dict__. Make the identity chain explicit and it stops being mysterious. import json binds the name json to a module object; that object's __dict__['dumps'] is a function object; and json.dumps(x) fetches that function, then calls it. Two steps you already know — a name lookup and a dict lookup — with nothing extra in between.
The convincing way to prove a module is ordinary is to do ordinary-object things to it. Alias it into another variable; drop it into a list; open its namespace with vars(); even bolt a brand-new name onto it with setattr. Every one of these works, because there is nothing special to violate:
import mymath # mymath.py defines PI = 3.14159 and def area(r): ...
print(type(mymath)) # <class 'module'> — an instance of types.ModuleType
print(vars(mymath).keys()) # dict_keys(['__name__', '__file__', ..., 'PI', 'area'])
j = mymath # a module is a value: alias it like anything else
toolbox = [mymath, str, len] # put it in a list
print(j.area(2)) # 12.566... — the alias reaches the same function
print(mymath.PI is mymath.__dict__['PI']) # True — attribute access IS the dict lookup
setattr(mymath, 'E', 2.71828) # write a new key into its namespace
print(mymath.E) # 2.71828 — you just edited the module's __dict__The line mymath.PI is mymath.__dict__['PI'] returning True is the whole reframe in one is: the dotted access and the raw dict lookup reach the identical object, because they are the same operation. This connects straight back to the binding chapters of Volume 1. Back then you learned that a module's global namespace "is just a dict" — the same fact that let module-level globals work. Now you can see that dict, print its keys, and write into it. There is a small, honest subtlety worth stating exactly. For a module, attribute access goes through __dict__ directly. A module does not run the full instance-and-class descriptor machinery for its own globals the way a class instance does. That is why module.__dict__, vars(module), and the namespace the code executed in are not three related things but literally one and the same object.
help(tracks) has something to say.'__main__' if this file is the program. Section 6 turns that into the guard.dir(m) vs vars(m)dir gives you a sorted list of the keys; vars gives you the dict. One is for reading, the other is the real thing.LIBRARY and as_clock are stored exactly like __file__ is. Nothing separates “your” names from the interpreter's.__module__ on a functionEvery function remembers which module executed its def. That is how a traceback can name the file a frame came from.you type
# ---------- tracks.py ----------
"""Duration helpers for the jukebox."""
LIBRARY = "Friday Night"
def as_clock(seconds):
"""Turn a whole number of seconds into m:ss."""
minutes, secs = divmod(seconds, 60)
return f"{minutes}:{secs:02d}"
def total(tracks):
"""Sum the seconds of an iterable of (title, seconds) pairs."""
return sum(secs for _title, secs in tracks)
# ---------- inspect_module.py ----------
import os
import tracks
print("__name__ :", tracks.__name__)
print("__doc__ :", tracks.__doc__)
print("__file__ :", os.path.basename(tracks.__file__))
print("__package__:", repr(tracks.__package__))
print("__spec__ :", tracks.__spec__.name, "| origin ends with:", os.path.basename(tracks.__spec__.origin))
print()
print("yours :", [n for n in dir(tracks) if not n.startswith("_")])
print("born with:", [n for n in dir(tracks) if n.startswith("_")])
print()
print("vars(tracks) is tracks.__dict__:", vars(tracks) is tracks.__dict__)
print("as_clock is __dict__['as_clock']:", tracks.as_clock is tracks.__dict__["as_clock"])
print("help text :", tracks.as_clock.__doc__)
print("its module :", tracks.as_clock.__module__)you see
__name__ : tracks
__doc__ : Duration helpers for the jukebox.
__file__ : tracks.py
__package__: ''
__spec__ : tracks | origin ends with: tracks.py
yours : ['LIBRARY', 'as_clock', 'total']
born with: ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']
vars(tracks) is tracks.__dict__: True
as_clock is __dict__['as_clock']: True
help text : Turn a whole number of seconds into m:ss.
its module : tracks- A docstring must be the first statement. Put a comment or an
importabove it and__doc__comes backNone. dir()with no argument lists the current namespace, not a module's. The two look similar and answer different questions.vars(tracks)is writable. Assigning into it really does add a module global — convenient, and a fine way to confuse a colleague.__file__is missing on a few module kinds, notably built-ins likesys. Reach for it withgetattr(m, '__file__', None)in library code.__cached__points at the.pyc, not the source. Reading it when you meant__file__gives you a path inside__pycache__.help(tracks)in the REPL prints all of this for you. It is reading the very same attributes, not consulting documentation somewhere.
__dict__ is its namespace; a dotted access like mod.name is a dict lookup of 'name' in that dict. There is no special "module operator." Once you believe this, the last question left is who else holds a reference to that one object — which is the entire next section.You bolted a new name onto mymath in one file. If another file imported it, would that file see your E? To answer that, you have to know whether the two imports share one object or hold two copies. →
04exec runs top-level code — and its side effects — once
You import a module and something prints — or a config file gets read, or a network call fires — before you have called a single function. That surprise is a message: you do not yet believe that importing a module runs it. Until that fact is bone-deep, import-order bugs and import-time side effects feel random and malicious, when they are in truth perfectly deterministic. So zoom all the way into step 3. Executing a module is not conceptually different from executing your main script. The interpreter takes the module's code object and runs every top-level statement, in source order, in a frame whose globals are the fresh module's __dict__. There is no separate "loading" phase that skips the interesting lines. The lines are the loading.
Concretely: a top-level data = compute() calls compute() now. A top-level print('loading') prints now. A top-level config = open('config.json').read() does real file I/O now — all during the import statement, before control returns to the importer. This is exactly why a def at module scope defines a function without calling it. The def statement, when executed, builds a function object and binds its name. That is all a def does. A bare function call at module scope is different: it actually runs the function, because a call expression is executed like any other. The distinction is not about def being special. It is that one statement binds a name and the other invokes a call.
import time
print('boot loading...') # TOP-LEVEL CALL — runs during import
START = time.time() # TOP-LEVEL CALL — the clock is read at import time
def greet(): # def executes: it BINDS the name 'greet'...
print('hi') # ...but the body does NOT run yet
greet() # a BARE call — THIS line runs the body, at import timeImport boot and, before you touch anything, you see boot loading... then hi. The def line bound greet; the line after it called greet. Ordering is strict and total: statements run top to bottom, so a name must be bound above the line that uses it, or you get a NameError. This is not a style rule — it is the direct consequence of executing lines in order. Move the definition below its first use and it fails; move it back up and it works.
The ordering rule reaches across files too, and this is where import-order surprises are born. Say module A's top-level code does import B and then uses something from B. All of B's top-level code runs to completion before A's next line executes. Import is a synchronous, depth-first descent. A pauses at its import B, B runs entirely, control returns to A, and A continues. Trace the interleaving once and it is never mysterious again:
# --- B.py ---
def setup():
print('B.setup ran')
return 42
# --- A.py ---
import B # B is found/compiled/exec'd fully before the next line
VALUE = B.setup() # prints 'B.setup ran', binds VALUE = 42 — all at A's import time
print('A finished, VALUE =', VALUE)Now tie it to the run-once law you'll formalize next. Because the finished module object is cached, this top-level execution — and every side effect it causes — happens exactly once per process, even if ten different files import the module. That is a gift and a trap. The gift: expensive setup runs once and is shared. The trap: heavy or side-effecting work at module top level — a network call, reading a large file, mutating shared state — fires at import time, in import order, often long before you meant for the program to "start." That fragility is precisely the pressure that produces the __main__ guard in the next section. It is a way to keep the runnable side effects out of the top-level path that import executes.
settings.json holding {"theme": "dark", "volume": 7, "shuffle": true}, then a config.py whose top level prints a line, reads that file with json.loads(Path("settings.json").read_text(encoding="utf-8")), binds the result to SETTINGS, and mutates it by adding SETTINGS["opened_by"] = __name__. Now write a report.py that prints report.py: starting, imports config, and prints the volume. Run it and read the order of the four lines out loud. Your file did disk I/O between two lines of a module that only wanted one integer. Second, name the bug precisely before you fix it. In one sentence, say which of the four import stages fired the read — and say why deleting __pycache__ would change nothing about it. Third, move the doing. Rewrite config.py so importing it binds names and does nothing else: keep PATH as a Path, add a module-level _cache = None, and put the read inside a settings() function that fills _cache on its first call only. Put the demo behind if __name__ == "__main__":. Re-run report.py and check three things: the import line is silent, the read happens where you asked for it, and a second config.settings() costs nothing. One hint and no more: global _cache is required inside settings(), because assigning to a name makes it local unless you say otherwise.show the solution
# ---------- settings.json ----------
{"theme": "dark", "volume": 7, "shuffle": true}
# ---------- config.py (BROKEN: it does its work at IMPORT time) ----------
import json
from pathlib import Path
print("config.py: opening settings.json ...") # fires during import
SETTINGS = json.loads(Path("settings.json").read_text(encoding="utf-8"))
SETTINGS["opened_by"] = __name__ # mutates on import, too
print("config.py: loaded", len(SETTINGS), "keys")
# ---------- report.py ----------
print("report.py: starting")
import config
print("report.py: volume is", config.SETTINGS["volume"])
# ------------- $ python report.py -------------
report.py: starting
config.py: opening settings.json ...
config.py: loaded 4 keys
report.py: volume is 7
# The bug, named: EXEC. Step 3 runs every top-level statement, so the open()
# happened inside the `import config` line. The .pyc caches step 2 (compile),
# never step 3 -- deleting __pycache__ would not silence a single one of these.
# ---------- config.py (FIXED: import binds names, nothing more) ----------
import json
from pathlib import Path
PATH = Path("settings.json")
_cache = None # nothing read yet
def settings():
"""Read settings.json once, on the FIRST call -- not on import."""
global _cache
if _cache is None:
print("config.py: opening settings.json ...")
_cache = json.loads(PATH.read_text(encoding="utf-8"))
return _cache
if __name__ == "__main__": # the demo, behind the guard
print("config.py self-test:", settings())
# ---------- report.py (same intent, now IT decides when to read) ----------
print("report.py: starting")
import config
print("report.py: import cost nothing")
print("report.py: volume is", config.settings()["volume"])
print("report.py: second ask is free:", config.settings()["theme"])
# ------------- $ python report.py -------------
report.py: starting
report.py: import cost nothing
config.py: opening settings.json ...
report.py: volume is 7
report.py: second ask is free: dark
# ------------- $ python config.py -------------
config.py: opening settings.json ...
config.py self-test: {'theme': 'dark', 'volume': 7, 'shuffle': True}
# The one honest sentence to take away:
# top level is for BINDING; a function body is for DOING.
# The lazy _cache keeps the run-once behaviour you liked and moves the
# moment it happens from "whenever someone imports me" to "when asked"."Exactly once per process" keeps coming up. What actually enforces it? One dict you can print — and it is the same dict step 1 checks first and step 4 writes last. →
05sys.modules is the run-once cache
You have heard "importing runs the code once" — but why once? If two different files both write import prices, what actually stops prices.py from running twice? And here's the flip side that bites every REPL user. You edit a module, re-type import prices in the same session, and your edits seem ignored. Why? Both answers live in a single dict you can print. It is called sys.modules, and seeing it turns two memorized rules into one mechanism you can inspect. sys.modules is an ordinary dict mapping import name → module object. And it is the very first thing import consults: the front of step 1 and the storage of step 4 are the two ends of this one cache.
Here is the real algorithm, in full. On import prices, check whether 'prices' is a key in sys.modules. If yes — skip find, compile, and exec entirely, and just bind the local name to the already-stored object. If no — run find → compile → create a blank module → insert it into sys.modules → exec its code → leave it cached. Two consequences fall straight out. First, run-once: the top-level code execs only on the cache-miss path. So no matter how many files import prices, or how many times, its code runs exactly once and every importer receives the same module object. Second, and equally important, the namespace is a process-wide singleton: because everyone holds the same object, if one file does prices.RATE = 0.2, every other importer sees 0.2. There is one __dict__, shared.
>>> import sys, prices
prices module loading... # cache MISS — top-level code ran
>>> 'prices' in sys.modules
True
>>> sys.modules['prices'] is prices # the cache entry and your name are ONE object
True
>>> import prices # cache HIT — no 'loading' print this time
>>> prices.RATE = 0.2 # mutate the shared singleton...
>>> sys.modules['prices'].RATE # ...and every importer sees it
0.2Now the edits-ignored mystery dissolves. In a long-lived REPL or server process, re-typing import prices after saving an edit is a cache hit. So it does nothing at all. You are handed the old object, and the new source on disk is never read. To force a re-exec you must reach past the cache. Either delete the key with del sys.modules['prices'] so the next import misses, or call importlib.reload(prices), which re-execs the source into the existing module object. Nothing was broken. The cache was doing its job perfectly.
Drag it to 1000. The green bar barely moves and the execution count stays welded to 1 — because after the first import, every import prices anywhere in the process is a single sys.modules dict lookup (~0.0002 ms), not a file load. The red bar is the intuition most people carry: “import loads the file,” done N times. That gap is the whole chapter.
import is a cached lookup keyed by name. The costs are modelled to order of magnitude (compile dominates the one-time work; a warm .pyc would shrink it further); the shape — one execution, then near-free lookups — is exactly real.The pre-exec insertion from step 3 is not a detail. It is what makes circular imports survivable. The blank module is placed in sys.modules before its top-level code runs. So if that code imports itself, directly or through another module, the second import finds the partially-built module already in the cache and returns it. It does not recurse forever into find/compile/exec. The cost is honest and worth knowing. The second importer may receive a half-populated module, with only the names bound above the import line existing yet. That is exactly why circular imports fail with "cannot import name X" when X is defined lower down. The cache trades infinite recursion for the possibility of a temporarily incomplete module.
Step back and the whole design snaps into focus. sys.modules is the registry of object identity for modules, and import is fundamentally a cached lookup keyed by name. This is the deepest connection to the thread of this volume. Volume 1 taught that a name points at one real object at one address. Here that idea scales up a level: an import name points at one real module object, shared across every file in the process. There are not "copies of json" floating around. There is one json object, and import json anywhere is a lookup that hands back that same one.
# ---------- boot.py ----------
# boot.py -- a module with a top-level print, so you can SEE it execute
print("boot.py: top level running")
COUNT = 1
def bump():
return COUNT + 1
# ---------- reload_reality.py ----------
# reload_reality.py -- import it three times, execute it once
import importlib
import sys
print("before :", "boot" in sys.modules)
import boot # MISS: find -> compile -> exec (the print fires)
print("after 1 :", "boot" in sys.modules)
import boot # HIT: one dict lookup, nothing runs
print("after 2 : still one execution, no second print")
import boot as b # HIT again, different local name
print("same object:", b is boot, "| is sys.modules entry:", b is sys.modules["boot"])
print()
boot.COUNT = 99 # mutate the shared singleton
print("b.COUNT :", b.COUNT, "- one module object, three names")
print()
print("now force the pipeline to run again:")
importlib.reload(boot) # re-exec the SAME module object
print("COUNT after reload:", boot.COUNT, "- the source's value is back")
print("still the same object:", boot is sys.modules["boot"])
before : False boot.py: top level running after 1 : True after 2 : still one execution, no second print same object: True | is sys.modules entry: True b.COUNT : 99 - one module object, three names now force the pipeline to run again: boot.py: top level running COUNT after reload: 1 - the source's value is back still the same object: True
python reload_reality.py. The whole lesson is in what is missing from the output. Three import statements, and boot.py: top level running appears once. The first one missed the cache and paid for find, compile and exec; the other two were a single dict lookup each. Notice that import boot as b did not build a second anything — b is boot is True, and both are sys.modules["boot"]. Three names, one object, one __dict__. That is why setting boot.COUNT = 99 through one name is visible through the other: you did not update two copies, because there were never two. Then watch importlib.reload, because it is more surgical than people expect. It does not make a new module. It re-execs the source into the existing object — which is why the print fires a second time, COUNT snaps back to 1, and boot is sys.modules["boot"] is still True. Every name anyone already holds keeps working, and quietly sees the new code. Two things to try. Add del sys.modules["boot"] before a fourth import boot and watch the print fire because you emptied the slot — but check boot is sys.modules["boot"] afterwards and find it False, because that route builds a second module object and strands the first. Then edit COUNT = 1 to COUNT = 7 on disk while the script is running — you cannot, in a script this short, and that is exactly the point: the reload is the only door.import name means: look name up in sys.modules; if present, hand back that object; if absent, build it, store it, then hand it back. Run-once, the shared singleton namespace, ignored REPL edits, and survivable circular imports are all one behavior of one dict — seen from four angles.You've seen that top-level code always runs on import. So how does a file act as a quiet library when imported but a runnable program when launched? One string, set by the interpreter, decides. →
06__name__ and the __main__ guard
Every real Python file you have ever read ends with the same incantation — if __name__ == '__main__': — and you have cargo-culted it faithfully. You did it without knowing what __name__ actually is or why the guard works. That ignorance has a concrete cost: you cannot decide what belongs inside the block and what belongs outside it. The felt need is real and specific. You want one file that is both a reusable library — safe for anyone to import for its functions — and a runnable program that does something when you launch it. And you want that without its "run the program" code firing every time someone merely imports it for a function. Section 4 showed the danger: top-level code runs on import. So an unguarded "start the menu loop" line at module scope would ambush every importer. The guard is the fix, and it is built from a single fact.
Every module has a __name__ entry in its __dict__, and the interpreter chooses its value at load time based on how the module was entered. When a module is imported, its __name__ is set to its import name — the string 'prices'. When a file is run directly as the program — python prices.py, or python -m prices — the interpreter runs it as the top-level module and sets its __name__ to the special string '__main__'. Same file. Same code object. Same bytecode. Different __name__, decided purely by which door the code was entered through.
__main__ guard — one file, two facesnot an incantation: __name__ is a string the interpreter sets, and if reads it like any other__name__ == '__main__'An ordinary comparison of two strings, evaluated when the line executes. There is no compiler magic and no special statement.import, at a moment you never chose.__name__ line, because it sits above the guard. Only the indented block is conditional.main.py is '__main__' tooWhichever file you launch gets the name. It is a property of the entry door, not of the file.'__main__' is a real keyPrint '__main__' in sys.modules from inside the guard and you get True. The program is a module like any other.you type
# ---------- tracks.py ----------
# tracks.py -- a module that can also be run
def as_clock(seconds):
minutes, secs = divmod(seconds, 60)
return f"{minutes}:{secs:02d}"
print("tracks.py top level ran, __name__ =", repr(__name__))
if __name__ == "__main__":
print(" demo:", as_clock(245))
# ---------- main.py ----------
# main.py -- imports it instead of running it
print("main.py top level ran, __name__ =", repr(__name__))
import tracks
print(" used as a library:", tracks.as_clock(200))
# ---------- both doors ----------
$ python tracks.py
$ python main.pyyou see
$ python tracks.py
tracks.py top level ran, __name__ = '__main__'
demo: 4:05
$ python main.py
main.py top level ran, __name__ = '__main__'
tracks.py top level ran, __name__ = 'tracks'
used as a library: 3:20if __name__ == "__main__"with one underscore, or'main'instead of'__main__', is silently False forever. No error, just a block that never runs.__name__is not the filename. Renametracks.pytodurations.pyand importers see'durations'.- The guard does not make code private. An importer can still call
tracks.as_clock— and can evenexecyour file to run the block. - Putting a
definside the guard hides it from every importer. Definitions belong above; only the doing goes below. - A module imported and run in the same process gets loaded twice under two names —
'__main__'and'tracks'— with two separate sets of globals. - The guard is not required to import a file. It is required to keep a file's program from ambushing the person importing it.
# ---------- tracks.py ----------
"""tracks.py -- the jukebox's duration helpers, importable by anything."""
LIBRARY = "Friday Night"
def as_clock(seconds):
"""245 -> '4:05'."""
minutes, secs = divmod(seconds, 60)
return f"{minutes}:{secs:02d}"
def total_seconds(playlist):
"""Sum the seconds of an iterable of (title, seconds) pairs."""
return sum(secs for _title, secs in playlist)
if __name__ == "__main__": # the PROGRAM face
demo = [("Blinding Lights", 200), ("Titanium", 245)]
print("tracks.py self-test")
print(" as_clock(245) ->", as_clock(245))
print(" total_seconds(demo)->", total_seconds(demo))
print(" as a clock ->", as_clock(total_seconds(demo)))
# ---------- main.py ----------
# main.py -- a different file, using tracks.py as a library
import tracks
night = [("Blinding Lights", 200), ("Titanium", 245), ("Levels", 340)]
print(tracks.LIBRARY, "-", len(night), "tracks")
for title, secs in night:
print(f" {title:<16} {tracks.as_clock(secs)}")
print("runtime:", tracks.as_clock(tracks.total_seconds(night)))
print("tracks.__name__ from in here:", tracks.__name__)
$ python tracks.py tracks.py self-test as_clock(245) -> 4:05 total_seconds(demo)-> 445 as a clock -> 7:25 $ python main.py Friday Night - 3 tracks Blinding Lights 3:20 Titanium 4:05 Levels 5:40 runtime: 13:05 tracks.__name__ from in here: tracks
tracks.py. It has a docstring, two functions with docstrings of their own, and a guarded block that tests itself. Run it directly and it is a program: it exercises both functions and prints what they gave back. That self-test is worth the four lines every time — it is the cheapest possible answer to “is this module still healthy?”, and it costs importers exactly nothing. Now look at main.py, and count what it did not have to do. No path. No copy of as_clock. No sys.path surgery. One line — import tracks — and the whole file is available under one name that cannot collide with anything of its own. The {title:<16} is chapter 2's format spec, unchanged, doing the column alignment. Then read the last line of the second run, because it is the chapter in miniature: tracks.__name__ is 'tracks', not '__main__', so the self-test stayed silent. Same file, same bytecode, different door. Two things to try. Add a longest(playlist) function to tracks.py, extend the self-test, and watch main.py pick it up with no edit at all. Then move the night list into tracks.py at top level, re-run main.py, and ask yourself whether data belongs in a module that other programs import.Once you know that, the guard is not clever — it is obvious. if __name__ == '__main__': is a plain runtime comparison of that one string. So the block beneath it runs only when the file was launched directly, and never when it was imported. That single test lets you give the file two faces. The library face — your defs, classes, and constants — goes at top level, where every importer receives it. The program face — argument parsing, the menu loop, a demo — goes inside the guard, where it fires only on direct launch:
def discount(price, pct): # LIBRARY FACE — importers get this, silently
return round(price * (1 - pct), 2)
if __name__ == '__main__': # PROGRAM FACE — runs ONLY on `python prices.py`
print('demo:', discount(100, 0.1)) # importers never see this fireImport this module from another file and you get discount quietly, with no demo output. Run python prices.py and it prints demo: 90.0. One file, two behaviors, chosen by the entry door. And the guard's value is exactly what sections 4 and 5 set up. Because import execs the whole top level, an unguarded "run the program" call at module scope would fire on every import. And because that exec is cached and run-once, the unwanted program-run would be smuggled into whichever file imported you first, then never repeat — a maddening, once-only ghost. The guard is the discipline that keeps import-time side effects out of the top-level path entirely.
__name__ — three launches, two answers, one surprisea.py prints "a.py top level, __name__ =", repr(__name__) at top level, defines hello() returning "hello from a", and ends with a guard that prints " a.py guard FIRED:", hello(). b.py prints its own top-level line the same way, then import a, then prints a.__name__, then has its own guard that calls a.hello(). Second, predict all three, on paper, before you run anything. For python a.py, python b.py and python -m a, write down every line that will print and in what order. Two of the three produce identical output — say which two, and say why the third differs. The one that catches people is python b.py: predict how many lines it prints, and be exact about whether a.py's guard fires. Third, run them and mark your answers. Then explain, in one sentence each: why b.py is '__main__' even though it imports something; why a is 'a' in that run and '__main__' in the other two; and why -m and the plain script form agree here. One hint and no more: __name__ is decided by how the module was entered, and there is exactly one entry-point module per process.show the solution
# ---------- a.py ----------
# a.py
print("a.py top level, __name__ =", repr(__name__))
def hello():
return "hello from a"
if __name__ == "__main__":
print(" a.py guard FIRED:", hello())
# ---------- b.py ----------
# b.py
print("b.py top level, __name__ =", repr(__name__))
import a
print(" a.__name__ seen from b:", repr(a.__name__))
if __name__ == "__main__":
print(" b.py guard FIRED:", a.hello())
# ------------- case 1: $ python a.py -------------
a.py top level, __name__ = '__main__'
a.py guard FIRED: hello from a
# ------------- case 2: $ python b.py -------------
b.py top level, __name__ = '__main__'
a.py top level, __name__ = 'a'
a.__name__ seen from b: 'a'
b.py guard FIRED: hello from a
# ------------- case 3: $ python -m a -------------
a.py top level, __name__ = '__main__'
a.py guard FIRED: hello from a
# Why, one sentence each:
# b.py is '__main__' because it is the file you launched -- importing
# something does not give the name away, there is one entry point.
# a is 'a' in case 2 because it arrived through import, which names a
# module after the name that was asked for.
# -m and the script form agree because both make the named module the
# entry point; what they disagree about is sys.path and __package__.
#
# The line people miss in case 2 is the second one. `import a` EXECUTES
# a.py -- so its top-level print fires, right there inside b.py's import
# statement, before b.py's own next line runs.Finally, dispel the last bit of magic. __main__ is not a keyword or a special construct — it is an ordinary module too. When you launch a file directly, the interpreter registers it in sys.modules under the key '__main__', exactly like any other module, and gives it the name '__main__'. You can prove it from inside the guard with print('__main__' in sys.modules) → True. There is no separate mechanism for "the main program." There is just the entry-point module, which happens to be assigned the reserved name '__main__' instead of a filename-derived one. The whole idiom, then, is two ordinary facts stacked: modules have a __name__, and the entry-point module's __name__ is '__main__'.
python file.py vs python -m package.modulethe guard fires either way — what changes is sys.path[0], __package__, and whether relative imports can work'__main__'The guard is not the difference. Whichever module you launch becomes the entry point and is given that name.sys.path[0]Script form puts the script's own folder first; -m puts your current directory first. That single row decides what import can find.__package__None for a lone script, 'app' under -m. A relative import needs a parent to be relative to.attempted relative import with no known parent package is not a bug in your code. It is Python saying the entry door threw the package away.sys.argvA list of strings the shell handed you. argv[0] is the script, argv[1:] is everything you typed after it — already split on spaces.strpython play.py 7 gives you '7', not 7. Convert at the edge, and let chapter 13's try handle a bad one.-m and the imports stop fighting you.you type
# ---------- the layout ----------
shop/
app/
__init__.py # this folder is a package (chapter 21's subject)
tracks.py # def as_clock(seconds): ...
play.py # the file below
# ---------- app/play.py ----------
# app/play.py -- ONE file, two launch doors
import os
import sys
print("__name__ :", repr(__name__))
print("__package__:", repr(__package__))
print("argv[0] :", os.path.relpath(sys.argv[0]).replace(os.sep, "/"))
print("argv[1:] :", sys.argv[1:])
print("sys.path[0]:", os.path.basename(sys.path[0]) or "(cwd)")
from . import tracks # a RELATIVE import: "my sibling, tracks"
print("relative import worked:", tracks.as_clock(245))
# ---------- both doors, run from inside shop/ ----------
$ python app/play.py Titanium --loud
$ python -m app.play Titanium --loudyou see
$ python app/play.py Titanium --loud
__name__ : '__main__'
__package__: None
argv[0] : app/play.py
argv[1:] : ['Titanium', '--loud']
sys.path[0]: app
Traceback (most recent call last):
File "shop/app/play.py", line 11, in <module>
from . import tracks # a RELATIVE import: "my sibling, tracks"
^^^^^^^^^^^^^^^^^^^^
ImportError: attempted relative import with no known parent package
$ python -m app.play Titanium --loud
__name__ : '__main__'
__package__: 'app'
argv[0] : app/play.py
argv[1:] : ['Titanium', '--loud']
sys.path[0]: shop
relative import worked: 4:05- Adding
sys.path.append("..")to make the script form work is the reflex to resist —-mis the fix, and it is one dash. python -m app.playuses a dot, not a slash, and no.py. It is an import name, so it obeys import's rules.-mimports the parent package first, so anything at the top level ofapp/__init__.pyruns before your module does.- Under
-mthe module is executed and named'__main__', so a sibling that doesimport app.playloads a second, separate copy. sys.argv[0]can be an empty string when Python is embedded or run with-c. Never assume it is a usable path.- Quoting matters at the shell, not in Python.
python play.py "Blinding Lights"is one argument; without the quotes it is two.
-m AND python file.py DIFFER SUBTLY__name__ to '__main__', so the guard fires either way. The difference is sys.path: python -m pkg.mod runs it as part of a package (relative imports work), while python path/to/mod.py runs it as a lone script (the script's own directory goes on sys.path, and package-relative imports may fail). Same guard, different find context.You now import cleanly. But the other import form — from config import SETTINGS — behaves differently in a way that silently breaks configs and monkeypatches. What exactly does it bind? →
Traceback (most recent call last):
File "tools/report.py", line 1, in <module>
import prices
ModuleNotFoundError: No module named 'prices'prices.py is right there — one folder up, in the same project, open in the next tab of your editor. Which is exactly why the first thought (“but the file exists”) is the wrong one. Read the message literally: it is about a name, and it never mentions a path, because step 1 was handed the string 'prices' and asked a list of directories whether any of them had it. None did. Import has no notion of “my project”; it has a list, and the first entry on that list is decided by how you launched Python, not by where you were standing when you typed the command. You ran python tools/report.py, so the search began in tools/, the folder of the file you named — and the project root, where prices.py lives, was never asked. Three causes account for nearly every instance, and they are worth ranking, because the fix is different for each. Most often the entry door is wrong, exactly as here, and the file is fine. Next most often the package is installed but not into the interpreter you just ran — a virtual environment you forgot to activate, or two Pythons on the machine and the shell picking the other one; import sys; print(sys.executable) settles that in one line. Least often, and most loudly assumed first, the name itself is not the filename minus .py, which section 1's trip list already catalogued. Chapter 20 opens up that list of directories and shows you exactly how it is built.python -m tools.report — same file, and section 6 just showed you exactly what that one dash movesTraceback (most recent call last):
File "worker.py", line 3, in <module>
q = queue.Queue()
^^^^^^^^^^^
AttributeError: module 'queue' has no attribute 'Queue'queue. The point is that queue here is not the standard library's queue at all — it is the queue.py you wrote yesterday, sitting beside worker.py, holding a JOBS list and an add function and no Queue whatsoever. Step 1 does not know the phrase “standard library.” It walks its list in order and takes the first directory that has a matching name, and the launched script's own folder is searched before the interpreter's own. So the import did not fail; it succeeded, at finding a different file. Notice that this is the same machinery as the row above, seen from the other side: there, the folder you were in was not searched, and here, it was searched first. The one-line diagnostic that ends every argument is print(queue.__file__) — section 3 called that field the honest answer to “which copy did I actually import?”, and this is the moment it earns the sentence. The trap wears many masks, all of them reasonable filenames: types.py, logging.py, select.py, email.py, string.py, test.py. One precise bound on the usual folk remedy, because half of it is wrong: renaming the source is enough — we checked, and an orphaned __pycache__/queue.cpython-312.pyc left behind after the rename does not keep shadowing, because a .pyc inside __pycache__ is not importable without its source beside it. A stray queue.pyc sitting loose in the folder itself is importable, and does keep shadowing. So sweep the loose ones; you can leave __pycache__ alone.jobqueue.py — and check module.__file__ the instant an attribute goes missingTraceback (most recent call last):
File "app.py", line 1, in <module>
import models
File "models.py", line 1, in <module>
from storage import save
File "storage.py", line 1, in <module>
from models import Track
ImportError: cannot import name 'Track' from partially initialized module 'models' (most likely due to a circular import) (models.py)app.py paused inside import models; models.py paused on its first line inside from storage import save; and storage.py, on its first line, asked models for Track. Section 5 already told you why this did not spin forever: step 3 puts a blank module into sys.modules before running a line, so models was found instantly in the cache. The price is written into the message. It was found partially initialized — at that instant models had executed exactly one statement, its own import, and class Track was still further down the file and had bound nothing. Notice which word the error chooses: it cannot import a name. The module was there. The failure is timing, not location — which is worth proving to yourself, because the obvious experiment misleads. We changed app.py to import storage first instead, and it fails just the same, only now the message reads cannot import name 'save' from partially initialized module 'storage'. Entering the ring from the other side only changes which of the two names gets reported, so chasing the name in the traceback is a dead end. What genuinely does change the outcome is where the import sits relative to the name being asked for: move from storage import save to the bottom of models.py, below class Track, and the identical two files load without complaint, because by the time storage asks, Track has been bound. That works, and we ran it — and it is still the worst of the fixes, since it makes correctness depend on line order in a file someone will tidy up next month. Two better cures, both verified, and they are one trick from two angles: make the lookup happen later, once both modules have finished. Move the from storage import save down into the method that actually calls save — legal, and not a hack, because the misconception check back in section 2 showed the compiler emits an ordinary IMPORT_NAME wherever that line happens to sit. Or have both files use plain import models / import storage at the top and reach through the module at call time, which works for the reason the next section is about: the dot is looked up fresh, every single time.07from X import Y copies the reference
You wrote from config import SETTINGS at the top of five files, because it read better than config.SETTINGS everywhere. Weeks later, some initialization code reassigns config.SETTINGS to a fresh dict at runtime. And your five files keep seeing the old one. Or you write from utils import helper, a test monkeypatches utils.helper to a stub, and your imported name blithely runs the real function, unpatched. This aliasing surprise silently defeats configs, test patches, and hot-swaps. The only cure is to know exactly what from X import Y binds. So state it precisely: from X import Y is two operations, and the second is a reference copy.
Operation one is a full, ordinary import of X — find, compile, exec, cache, identical to import X. X's top-level code runs, and X ends up in sys.modules. The only difference from plain import X is that X's own name may not be left bound in your namespace. Operation two looks up the attribute Y on the module object — a lookup of 'Y' in X.__dict__, the dict lookup from section 3. Then it binds that value to a name Y in your namespace. The crucial word is value. It copies the reference — the object Y points at right now — not a live link to X's slot. After from config import SETTINGS, your SETTINGS and config.SETTINGS are two names in two different namespaces, both pointing at the same object at this instant. And from this instant on, the two names have independent futures.
tracks is tk is True — renaming binds, it never copies.f pointed at during the import. Cheaper to type, and frozen at that instant.load.from m import f does a full find/compile/exec of m and leaves it in sys.modules. You just did not keep a name for it.total came from, and neither can your editor.* is a smellWatch total in the run below go from 999 to 0. No error, no warning — a name you owned was silently overwritten.__all__A module can set __all__ = ["as_clock"] to limit what * exports. It is a courtesy, and it is not a defence — the others are still importable by name.you type
# ---------- tracks.py ----------
LIBRARY = "Friday Night"
total = 0 # a perfectly innocent module-level name
def as_clock(seconds):
minutes, secs = divmod(seconds, 60)
return f"{minutes}:{secs:02d}"
def _internal(): # leading underscore: not exported by *
return "private"
# ---------- forms.py ----------
# forms.py -- four import forms, and what each leaves in YOUR namespace
total = 999 # YOUR name, bound before any import
before = set(globals())
import tracks # 1: bind the MODULE object
import tracks as tk # 2: same object, your choice of name
from tracks import as_clock # 3: one ATTRIBUTE, copied out
from tracks import as_clock as clock # 4: same attribute, renamed
print("names added:", sorted(set(globals()) - before - {"before"}))
print("tracks is tk :", tracks is tk)
print("as_clock is tracks.as_clock:", as_clock is tracks.as_clock)
print("clock is as_clock :", clock is as_clock)
print("tracks ->", type(tracks).__name__, "| as_clock ->", type(as_clock).__name__)
print("your total is still :", total)
from tracks import * # 5: the smell -- names you did not name
print()
print("after import * :", sorted(set(globals()) - before - {"before"}))
print("your total is now :", total, "<- tracks.total landed on top of it")
print("_internal came along? :", "_internal" in globals())you see
names added: ['as_clock', 'clock', 'tk', 'tracks']
tracks is tk : True
as_clock is tracks.as_clock: True
clock is as_clock : True
tracks -> module | as_clock -> function
your total is still : 999
after import * : ['LIBRARY', 'as_clock', 'clock', 'tk', 'tracks']
your total is now : 0 <- tracks.total landed on top of it
_internal came along? : Falseimport a.bbindsa, notb. After it,balone is aNameError— you reach it asa.b.import tracks as tkleavestracksunbound. The run above only has both names because line 1 imported it plainly first.from m import *at function scope is aSyntaxError. Python needs the names known at compile time inside a function.- A second
from m import *after you rebind a clashing name will clobber it again. The last import wins, and nothing says so. from m import fthenm.f = otherleaves yourfon the old function — the stale-reference bug this section is about.- The underscore rule only applies to
*.from tracks import _internalworks perfectly; the underscore is a convention, not a lock. - In the REPL,
from m import *is fine and often useful. In a file someone else will read, it costs them the ability to answer “where is this from?”
Two consequences follow, and the difference between them is the entire lesson. It is the same names-versus-objects model from the binding chapters, now stretched across a module boundary. Consequence one: rebinding does not reach you. If someone later does config.SETTINGS = {new dict}, that only changes config.__dict__['SETTINGS'] to point at a new object. Your name still points at the old one. Your Y is now stale. This is precisely why monkeypatching module.func fails to affect callers who did from module import func. Consequence two: mutation does reach you. If instead someone mutates the shared object in place — config.SETTINGS['debug'] = True — both names still point at that same object, so you do see the change. Rebind versus mutate is the whole story:
from config import SETTINGS # copy the reference: your SETTINGS -> Object#1
import config
config.SETTINGS = {'v': 2} # REBIND: config.SETTINGS -> a NEW Object#2
print(SETTINGS) # {'v': 1} — your name still on Object#1 (STALE)
print(config.SETTINGS) # {'v': 2} — the module moved on without you
config.SETTINGS['debug'] = True # MUTATE the object both names still share
print(SETTINGS) # {'v': 2, 'debug': True} — seen, because same objectContrast this with plain import config. You kept the module object, and you reach the value fresh each time through config.SETTINGS — a live __dict__ lookup performed at the moment of use. So you always observe the current binding. That is exactly why import module then module.thing is more patch-friendly than from module import thing. The first re-reads the slot every time. The second froze a snapshot of it at import. Neither is "correct" — they answer different questions. from module import thing asks "what object is thing right now?" and remembers the answer. import module; module.thing asks the question again on every use.
# ---------- tracks.py ----------
"""tracks.py -- what a track and a playlist ARE. No I/O, no menu."""
from dataclasses import dataclass
@dataclass
class Track: # ch 12's dataclass, unmoved
title: str
artist: str
seconds: int
plays: int = 0
def __str__(self):
minutes, secs = divmod(self.seconds, 60)
return f"{self.title} - {self.artist} ({minutes}:{secs:02d})"
class Playlist:
def __init__(self, name, tracks):
self.name, self.tracks = name, list(tracks)
def __len__(self):
return len(self.tracks)
def __iter__(self):
return iter(self.tracks)
def __repr__(self):
minutes, secs = divmod(sum(t.seconds for t in self), 60)
return f"Playlist({self.name!r}, {len(self)} tracks, {minutes}:{secs:02d})"
def play(self, number):
track = self.tracks[number - 1]
track.plays += 1
return track
if __name__ == "__main__":
print("tracks.py self-test:", Track("Titanium", "David Guetta", 245))
# ---------- storage.py ----------
"""storage.py -- the only file that knows what JSON is. Chapter 18, boxed."""
import json
from pathlib import Path
from tracks import Track, Playlist
DB = Path("library.json")
def save(playlist, path=DB):
payload = {"version": 1, "name": playlist.name,
"tracks": [vars(t) for t in playlist]}
with open(path, "w", encoding="utf-8", newline="\n") as f:
json.dump(payload, f, indent=2)
return path.stat().st_size
def load(path=DB):
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
return Playlist(data["name"], [Track(**row) for row in data["tracks"]])
if __name__ == "__main__":
print("storage.py self-test: DB is", DB)
# ---------- jukebox.py ----------
"""jukebox.py -- the program. Imports the other two; owns nobody's internals."""
import storage
from tracks import Track, Playlist
def seed():
return Playlist("Friday Night", [
Track("Blinding Lights", "The Weeknd", 200),
Track("Titanium", "David Guetta", 245),
Track("Levels", "Avicii", 340),
])
def report(playlist):
print(repr(playlist))
for i, track in enumerate(playlist, start=1):
print(f" {i}. {track} played {track.plays}x")
def main():
night = storage.load() if storage.DB.exists() else seed()
print("loaded from disk:", storage.DB.exists())
report(night)
print()
for number in (2, 1, 2):
print(" now playing", night.play(number))
size = storage.save(night)
print()
print(f"saved {size} bytes to {storage.DB.name}")
again = storage.load() # prove the round trip, same process
report(again)
print("counts survived the file:", [t.plays for t in again])
if __name__ == "__main__":
main()
$ python jukebox.py
loaded from disk: False
Playlist('Friday Night', 3 tracks, 13:05)
1. Blinding Lights - The Weeknd (3:20) played 0x
2. Titanium - David Guetta (4:05) played 0x
3. Levels - Avicii (5:40) played 0x
now playing Titanium - David Guetta (4:05)
now playing Blinding Lights - The Weeknd (3:20)
now playing Titanium - David Guetta (4:05)
saved 392 bytes to library.json
Playlist('Friday Night', 3 tracks, 13:05)
1. Blinding Lights - The Weeknd (3:20) played 1x
2. Titanium - David Guetta (4:05) played 2x
3. Levels - Avicii (5:40) played 0x
counts survived the file: [1, 2, 0]
python jukebox.py. Nothing in this program is new. Track and Playlist are chapter 12's classes, unedited. save and load are chapter 18's json.dump and json.load, unedited. What changed is the shape: the course's program is now three files instead of one, and that is the payoff worth naming — this is how real code is organised. Look at what each file is allowed to know. tracks.py knows what a track is and has never heard of JSON or of a disk. storage.py is the only file in the project that imports json; move to CSV or SQLite tomorrow and exactly one file changes. jukebox.py knows the story — load, play, save, report — and reaches into nobody's internals. That is not tidiness for its own sake. It is the direct answer to section 1: the price math and the menu loop stopped sharing one flat global namespace. Two mechanisms from this chapter are doing quiet work here. storage.py does from tracks import Track, Playlist and jukebox.py does import storage, which imports tracks in turn — and tracks.py executes exactly once, because the second request was a sys.modules hit. Both files hold names for the same Track class, which is why a Track built in jukebox.py is the same type storage.py reconstructs. And all three files end in a guard, so each one can be run directly for a self-test without any of them ambushing the others on import. Now run it a second time. loaded from disk is True, and the play counts come back 2, 4, 0 — the first run's plays survived the process that made them. Two things to try. Add a shuffle() to Playlist and notice that neither other file needs an edit. Then delete library.json and watch the seed path run again, which is the only branch in main.The reference-copy semantics are the same no matter how you dress the syntax. import numpy as np and from utils import helper as h only change which local name you bind. They do not change the fact that you bound a reference to whatever object was there at import time. And the mechanism is nothing exotic — it is the plainest possible reference copy. Import a name bound to a shared list, append to the list through either name, and both the module's name and your imported name see the new element, because there is one list and two names for it. That single object with two names, reachable from two files, is the through-line of this whole chapter. import does not copy objects between files. It gives many files names for the same objects — which is exactly the law the rest of Volume 2 will test against every harder boundary, where objects genuinely cannot cross and only bytes can.
tracks.py holding LIBRARY, as_clock(seconds) and total_seconds(playlist), where a playlist is a list of (title, seconds) pairs. Now add a third file, stats.py, that import tracks and exposes avg_seconds(playlist) returning the mean duration — and 0.0 for an empty playlist, because a module that raises ZeroDivisionError on a legitimate input is not finished. Build it on top of tracks.total_seconds; do not re-implement the sum. Second, add a second function and a self-test. Give stats.py a longest(playlist) that returns the (title, seconds) pair with the most seconds, using max with a key=. Then put a guarded self-test at the bottom that exercises avg_seconds on a two-track demo, so python stats.py is a useful thing to type. Third, use it from main, then prove the sharing. Write main.py that imports both modules and prints the total, the mean in seconds, the mean as a clock, and the longest title. Finish with print(stats.tracks is tracks) and explain the answer in one sentence — that single True is the whole chapter. One hint and no more: tracks.as_clock wants a whole number, so round() the mean before you format it.show the solution
# ---------- tracks.py ----------
"""tracks.py -- durations."""
LIBRARY = "Friday Night"
def as_clock(seconds):
minutes, secs = divmod(seconds, 60)
return f"{minutes}:{secs:02d}"
def total_seconds(playlist):
return sum(secs for _title, secs in playlist)
# ---------- stats.py ----------
"""stats.py -- numbers about a playlist. Imports tracks; imported by main."""
import tracks
def avg_seconds(playlist):
"""Mean duration in seconds; 0.0 for an empty playlist."""
if not playlist:
return 0.0
return tracks.total_seconds(playlist) / len(playlist)
def longest(playlist):
"""The (title, seconds) pair with the most seconds."""
return max(playlist, key=lambda pair: pair[1])
if __name__ == "__main__":
demo = [("A", 100), ("B", 200)]
print("stats.py self-test: avg_seconds ->", avg_seconds(demo))
# ---------- main.py ----------
import stats
import tracks
night = [("Blinding Lights", 200), ("Titanium", 245), ("Levels", 340)]
print(tracks.LIBRARY, "-", len(night), "tracks")
print("total :", tracks.as_clock(tracks.total_seconds(night)))
print("mean :", stats.avg_seconds(night), "seconds")
print("mean :", tracks.as_clock(round(stats.avg_seconds(night))))
print("longest :", stats.longest(night)[0])
print()
print("main is :", __name__)
print("its imports are:", stats.__name__, "and", tracks.__name__)
print("stats sees the same tracks:", stats.tracks is tracks)
# ------------- $ python stats.py -------------
stats.py self-test: avg_seconds -> 150.0
# ------------- $ python main.py -------------
Friday Night - 3 tracks
total : 13:05
mean : 261.6666666666667 seconds
mean : 4:22
longest : Levels
main is : __main__
its imports are: stats and tracks
stats sees the same tracks: True
# Why that last True matters:
# main.py imported tracks and so did stats.py -- but tracks.py ran ONCE.
# The second request was a sys.modules hit, so both files hold a name for
# the same module object. There is no "stats' copy of tracks". Grow the
# project to twenty files and the count stays at one.from X import Y copies the reference Y holds now; import X; X.Y re-reads it every time. So mutation of the shared object is always seen through either; rebinding the name in X is seen only by import X; X.Y. When patchability matters — configs, tests, hot-swaps — import the module and reach through it.✗ The myth
"from config import SETTINGS gives me a live link to config's setting — if config changes it, I'll see the new value."
✓ The reality
It gives you a snapshot reference to the object that was there at import time. If config rebinds SETTINGS to a new object, your name is stranded on the old one. You only track config's changes if you kept the module and read config.SETTINGS fresh each time.
One object, many names, all inside one process — the easy case. Next volume-step: what happens when the boundary is real, an object cannot cross it, and only a byte stream can. That is where import's cousins — encoding, files, and pickle — take over. →
import pricesis not “load a file” but four steps in a fixed order — find turns a name into a file and a loader, compile turns source text into a code object and stashes it in__pycache__so the next process can skip that one step, exec runs the code with a fresh module's__dict__serving as its globals, and cache leaves the finished module insys.modules— which means every strange symptom you will ever debug belongs to exactly one of the four, and naming the stage is most of the fix.- A module is not a construct the language keeps in a special place but an ordinary object, an instance of
types.ModuleType, whose__dict__is the namespace its top-level code executed in — sojson.dumpsis nothing more exotic than the key'dumps'read out ofjson.__dict__,vars(m) is m.__dict__comes backTrue, and you can alias a module, put it in a list, orsetattra new name onto it, because there is nothing special there to violate. - Importing a module runs it, top to bottom, inside the
importstatement and before the importing file's next line executes — a top-leveldefbuilds a function object and binds a name without running the body, and a top-levelclassdoes run its body, though that body is normally nothing but moredefs and constants, while every other top-level line does its work right then, at import time and in import order, which is why the discipline that survives contact with a real project is top level for binding, function bodies for doing. sys.modulesis the entire mechanism in one printable dict, consulted first and written early — the blank module goes in before a line of its code runs, and only the finish is deferred — keyed by the name you asked for, and that single fact buys all four of the behaviours people memorise separately: code that runs exactly once per process, one shared module object that every importer holds a name for, saved edits that seem ignored in a live session until you reach past the cache withimportlib.reload, and circular imports that fail with a readable message instead of recursing forever, because the blank module is inserted before its code is allowed to run.- One file can wear two faces, and neither of them is magic:
__name__is a plain string the interpreter sets from the door the code was entered through — the import name, or'__main__'for the file you launched — so the guard is an ordinary comparison of two strings; andfrom X import Ycopies the referenceYholds at that instant, which is why later mutation of the shared object still reaches you and later rebinding insideXnever will.
sys.modules and from X import Y have ever done, only now stretched across a file boundary; chapter 7 gave you the dict and its one-hash lookup, which is exactly what “runs once per process” costs on every import after the first; chapter 9 gave you def as a statement that builds a function object and binds a name without running the body, which is the whole reason importing a library of functions is quiet; chapter 13 gave you the traceback as a flight log, oldest call first, which is the only reason the error clinic's three stacked half-finished files read as a story instead of a wall; and chapters 12 and 18 gave you the dataclass and the json.dump that the three-file jukebox re-used without a single edit — because splitting a program into modules changes its shape, never its code.Chapter 19 is where "one big script" becomes a program built from modules. These twelve steppers make the import machinery physical: import is not magic, it is find-the-file, compile-it, exec-it top-to-bottom, and cache-the-result. Along the way a module turns out to be an ordinary object whose attributes are just dictionary keys, the cache turns out to be one shared copy in sys.modules, and the famous `from x import y` staleness bug turns out to be nothing but a copied reference.