26Threads & the GIL — derived from refcounting

You have heard both halves of the folk wisdom, usually in the same breath: threads make Python faster, and threads don't help in Python. Both are true. The gap between them is where a beginner loses a weekend and blames the language. Back in Chapter 3 we gave every object a header carrying an ob_refcnt counter that ticks up and down as names come and go. It looked like private bookkeeping. In this chapter we take that one counter seriously across two cores, and it turns out to decide whether your program can use a second core at all. Here's the plan. We point two threads at the same heap, then watch a refcount race quietly corrupt memory. Out of the wreckage falls CPython's answer, the Global Interpreter Lock — not a wart someone forgot to delete, but a theorem. And the whole way through we keep asking the one question that settles the folklore: given refcounting, could Python ever safely let two threads run bytecode at once? By the end we can look at any slow program and say, in a single measurement, whether threads will rescue it or do nothing at all. The same lock that makes threads useless for computing leaves them untouched for waiting, and we'll know exactly why.

01The one-question diagnosis

Let's start with the trap, because almost everyone falls into it once. You read that threads make programs faster. So you wrap a loop in a ThreadPoolExecutor, run it, and watch: it comes back slower. Not a little slower. Measurably, embarrassingly slower, fans spinning up for nothing. The instinct is to decide Python is broken, or that you held it wrong. Neither is true. You reached for the right tool on the wrong workload. The difference between the two workloads isn't a matter of taste. It's physical: it lives in what the CPU is actually doing while the wall clock ticks. So before we write a single thread, we need the one question that sorts any program cleanly into one bucket or the other. Every later decision in this chapter falls straight out of the answer.

Here's the fork. A running program spends its wall-clock time in exactly one of two states, and down at the silicon they could not be more different. In the first, the core is busy computing. The instruction pointer never leaves your code. The pipeline retires an arithmetic, logic, or branch instruction essentially every cycle, and the ALU is hot. Summing a hundred million integers, resizing an image, hashing a password — this is CPU-bound work. The only way to finish sooner is more cycles per second, or more cores working the problem at the same time. In the second state, the core is waiting. Your thread issued a system call — read(), recv(), connect() — and the kernel put it to sleep in a wait queue. Linux calls this the uninterruptible-sleep, or D-state. The CPU is not slow here. It is idle for this task, descheduled, doing other people's work or nothing at all. It wakes your thread only when an interrupt fires: a packet arrives, a disk DMA transfer completes. Downloading fifty pages, querying a database, reading files — this is I/O-bound work, and the bottleneck is not computation at all. It is the waiting.

So here's the diagnostic question — call it the crunch test: if the network and the disk were infinitely fast, would this program finish instantly? If yes, the time was spent waiting, the work is I/O-bound, and the win is to overlap the waits. You keep many requests in flight so their idle stretches stack on top of each other instead of end to end. If instead the only cure is a faster CPU, the work is CPU-bound. No amount of overlapping helps. Only true parallelism — more cores actually executing — moves the needle. That single fork decides everything: threads, async, or processes.

You do not have to guess which state you are in. You can measure it, because the operating system already keeps two different clocks. time.perf_counter() measures wall time — elapsed reality, including every second spent asleep in a kernel queue. time.process_time() measures only CPU time actually charged to your process, the cycles you truly burned. Hold them up against the same function, and the ratio confesses the workload:

import time

def fetch_url(url):            # pretend network: the CPU does almost nothing
    time.sleep(2)              # blocked in the kernel — not computing
    return f"<html>{url}</html>"

def crunch(n):                 # pure arithmetic: the ALU never rests
    return sum(i * i for i in range(n))

def classify(fn, *args):
    wall0, cpu0 = time.perf_counter(), time.process_time()
    fn(*args)
    wall = time.perf_counter() - wall0
    cpu  = time.process_time()  - cpu0
    verdict = "CPU-bound" if cpu / wall > 0.5 else "I/O-bound"
    print(f"{fn.__name__:9} wall={wall:5.2f}s cpu={cpu:5.2f}s  {verdict}")

classify(fetch_url, "example.com")   # wall=2.00s cpu=0.00s  I/O-bound
classify(crunch, 10_000_000)         # wall=0.80s cpu=0.80s  CPU-bound

Read the two comment lines like an X-ray. The fetch burned two full seconds of wall time and essentially zero CPU — the process was asleep, the core free. The crunch spent its wall time and its CPU time in lockstep. Every second of clock was a second of computation. That cpu / wall ratio is the whole diagnosis in one number: near 1 means you were computing, near 0 means you were waiting. Everything the rest of this chapter says about the GIL is downstream of this split, because the GIL punishes the crunch and leaves the waiting untouched.

The classifier says overlap the waits for I/O and use more cores for compute. But to see why one works in Python and the other refuses to, you first have to know what a thread actually is — and what, alarmingly, it shares. →

02A thread is a flow sharing the heap

To predict what threads do, you have to know precisely what a thread is — and, more sharply, what it is not. The dangerous half is what a thread does not get: its own copy of your data. In Volume 1 every object lived on the heap behind a header — ob_refcnt, ob_type, then the payload — at a real address in RAM, and every name was a pointer to it. When you spin up a second thread, none of that is duplicated. The list is not cloned. The dict is not cloned. Your module globals are not cloned. Both threads reach into the same heap and touch the same objects.

Formally, a process is an address space plus one or more threads of execution. Each thread is scheduled independently by the OS kernel, and each owns exactly three private things. It has its own call stack — its chain of frame objects, its local variables. It has its own instruction pointer, which line it is on. And it has its own register set, the handful of CPU registers the kernel saves and restores every time it switches threads. That is the entire list of what is private. Everything else is common property: the heap and every Python object on it, the loaded bytecode, the open file descriptors, the interpreter's own state. So when thread A and thread B both name results, they are not holding two similar lists. They hold the identical 8-byte pointer to the identical header at the identical address. One list, two arrows.

Now add the part that turns “shared” into “dangerous.” The kernel does not switch threads politely at the end of your Python statements. It switches on a timer interrupt, at an arbitrary machine instruction — possibly halfway through evaluating a single expression — saving A's registers and restoring B's. This interleaving is invisible from your source code and non-deterministic run to run. A thread can be frozen mid-reach, its half-finished work exposed, while another thread walks up to the very same object and starts changing it. Hold that image: two arrows into one list on one heap, and a scheduler that can pause either arrow between any two instructions. Every race condition in the chapter is already implied by that picture.

You can prove the sharing is literal, not a metaphor, in five lines. Use id(), which in CPython returns the object's actual heap address, and threading.get_ident(), which returns the OS-level identity of the running thread:

import threading

shared   = []            # one list object, at one address on the heap
seen_ids = []

def worker():
    shared.append(threading.get_ident())   # which thread am I?
    seen_ids.append(id(shared))            # what address is the list at?

t1 = threading.Thread(target=worker)
t2 = threading.Thread(target=worker)
t1.start(); t2.start()
t1.join();  t2.join()

print(shared)      # [140192..., 140188...]  — two DIFFERENT thread ids
print(seen_ids)    # [4501...840, 4501...840] — ONE identical address

Two different thread identities, appended into one list, which both threads saw at the same address. That is the whole point compressed into two numbers: the flows of control are separate, but the data underneath them is one and the same. Notice too that neither thread had to be handed the list — shared was simply a global, and globals are just names in the module object, which lives on the shared heap. There is no per-thread copy of your module. Everyone reads and writes the one original.

you type
# ---------- skeleton.py ----------
import threading
import time


def tick(label, n):
    for i in range(n):
        print(f"{label}-{i}  ({threading.current_thread().name})", flush=True)
        time.sleep(0.05)


# 1. CALLING it: runs here, in this thread, now, to completion.
tick("direct", 2)

# 2. STARTING it: a real OS thread runs it; start() returns immediately.
t1 = threading.Thread(target=tick, args=("A", 3))
t2 = threading.Thread(target=tick, args=("B", 3))

t1.start()
t2.start()
print("main is free", flush=True)

t1.join()
t2.join()
print("both joined")

you see
$ python skeleton.py
direct-0  (MainThread)
direct-1  (MainThread)
A-0  (Thread-1 (tick))
B-0  (Thread-2 (tick))
main is free
A-1  (Thread-1 (tick))
B-1  (Thread-2 (tick))
B-2  (Thread-2 (tick))
A-2  (Thread-1 (tick))
both joined

$ python skeleton.py          # the very same file, run again
direct-0  (MainThread)
direct-1  (MainThread)
A-0  (Thread-1 (tick))
B-0  (Thread-2 (tick))
main is free
B-1  (Thread-2 (tick))
A-1  (Thread-1 (tick))
A-2  (Thread-1 (tick))
B-2  (Thread-2 (tick))
both joined

This is exactly where the metal-up thread of Volume 2 splits in two. When you cross a process boundary — the subject of chapter 28 — fork() copies the entire address space. So processes share nothing by default and can never corrupt each other's objects. What passes between them is bytes, never live objects, which is precisely why processes escape the GIL. Threads are the opposite border: nothing is copied, everything is shared, live objects are touched directly by two cores at once. That single structural difference — copy-everything versus share-everything — is the root of every behavior in this chapter, including the lock we are about to prove must exist.

you type
# ---------- daemon_box.py ----------
import threading
import time


def job(label, secs):
    me = threading.current_thread().name
    try:
        print(f"  [{me}] {label} working", flush=True)
        time.sleep(secs)
        print(f"  [{me}] {label} finished", flush=True)
    finally:
        print(f"  [{me}] {label} cleanup", flush=True)


keeper = threading.Thread(target=job, args=("short", 0.4), name="keeper")
ghost  = threading.Thread(target=job, args=("long", 5.0), name="ghost", daemon=True)

keeper.start()
ghost.start()

print("alive       :", [t.name for t in threading.enumerate()], flush=True)
print("daemon flags:", keeper.daemon, ghost.daemon, flush=True)

keeper.join()                 # we wait for this one
print("main: returning -- nobody waits for a daemon", flush=True)

you see
$ python daemon_box.py
  [keeper] short working
  [ghost] long working
alive       : ['MainThread', 'keeper', 'ghost']
daemon flags: False True
  [keeper] short finished
  [keeper] short cleanup
main: returning -- nobody waits for a daemon

# and that is the entire output. the process exits after 0.4s,
# not 5s. "ghost long finished" never prints -- fine, it was cut short.
# but "ghost long cleanup" never prints either, and that one is a promise
# the language just broke. finally: does not run on a killed daemon.

Two threads, one heap, a scheduler that pauses either one between any two instructions. Now aim both of them at the one field every object carries — ob_refcnt — and watch the crash assemble itself. →

03Why the GIL must exist

This is the spine of the chapter, and its payoff. You have been told the GIL is an embarrassment, a bottleneck someone should simply delete. By the end of this section you will see it is nothing of the kind. It falls straight out of Volume 1's refcounting as an inevitability. Take refcounting seriously across the two shared-heap threads you just met, and a global lock — or a lock on every single object — stops being a design choice. It becomes forced.

Recall the mechanism from chapter 3. Every object carries ob_refcnt, a count of how many references point at it. Every time a reference is created or destroyed — an assignment, passing an argument, returning a value, a name going out of scope — CPython runs Py_INCREF or Py_DECREF. And here is the fact that detonates everything: at the machine level, incrementing a counter in memory is not one instruction. It is three. LOAD the current count from RAM into a CPU register. ADD one to the register. STORE the register back to RAM. A read, a modify, a write — and, as you learned last section, the scheduler can fire a timer interrupt between any two of them.

Now interleave two threads over one object whose count is 2. Thread A executes LOAD and pulls 2 into its register. The timer fires. The kernel freezes A, its register still privately holding 2, and runs Thread B. B does the whole tick cleanly: LOAD 2, ADD to 3, STORE 3. The count in RAM is now correctly 3. The kernel switches back to A, which has no idea anything happened. It resumes from where it stood, with its stale 2, ADDs to get 3, and STOREs 3. Two increments occurred. The count should read 4. It reads 3. One increment vanished into the gap — a classic lost update.

A lost increment is bad. The mirror case is catastrophic. Run the same race on Py_DECREF and you can lose a decrement the same way — but worse, an interleaving can drive the count to 0 while a live reference still exists. Reaching 0 is CPython's signal to free the object: call its destructor, return its memory to the allocator. So the object is torn down and its bytes recycled while another thread is still holding a pointer into it. The next time that thread dereferences its now-dangling pointer, it reads freed memory — a use-after-free. That is not a wrong answer; that is a segfault, or worse, silent corruption. The very mechanism that makes CPython's memory management deterministic becomes, under naive threading, a machine for crashing.

You can watch the lost-update half happen in pure Python, no C required. Spell the read-modify-write out by hand, so the interpreter can be preempted between the pieces:

import threading

counter = 0

def bump():
    global counter
    for _ in range(1_000_000):
        tmp = counter        # LOAD  — read the count into a local
        tmp = tmp + 1        # ADD   — bump the private copy
        counter = tmp        # STORE — write it back; preemptible before this

a = threading.Thread(target=bump)
b = threading.Thread(target=bump)
a.start(); b.start(); a.join(); b.join()

print(counter)   # the race in principle — see the honesty note below

Here is the honest 3.12 wrinkle. This exact loop happens to print 2,000,000 on current CPython, because since 3.10 the interpreter only offers to switch threads at a few instructions, and none of them fall inside this tiny window. The race is real, but it needs a genuine preemption point between the LOAD and the STORE — put a function call in that window and updates are lost by the hundreds of thousands, as the lock box below shows with captured numbers. When the race does fire, every shortfall is one thread's LOAD stomped by the other's STORE — the exact race, made visible in your own code. And ob_refcnt is bumped this way not once per million iterations but on every reference operation the interpreter performs, millions of times a second, on every object alive. You can even watch the count move:

>>> import sys
>>> x = object()
>>> sys.getrefcount(x)     # 2: your name x, plus getrefcount's own argument
2
>>> y = x                    # a second name — the count ticks up
>>> sys.getrefcount(x)
3
>>> del y                     # drop it — the count ticks down
>>> sys.getrefcount(x)
2

So the fix space has exactly two shapes. Option one: make every refcount tick atomic, or give every object its own lock. Correct — and ruinous. There are millions of tiny objects, and a per-object lock on each is enormous overhead. Atomic increments force the CPU to serialize the cache line on every tick, which murders performance on the single-threaded programs that are the overwhelming majority. Option two: a single Global Interpreter Lock that a thread must hold to touch any Python object. Then refcount ticks are never concurrent, because only one thread runs at a time. CPython chose option two. One mutex. Near-zero cost in the common single-threaded case, and refcounting stays sane — hence deallocation, hence all of C-level memory safety. The GIL is the price of refcounting without per-object locks.

you type
# ---------- lock_box.py ----------
import threading
import time

TIMES = 1_000_000
total = 0
lock  = threading.Lock()


def step(n):                 # a call, so the GIL really can change hands here
    return n + 1


def bump():                  # the race
    global total
    for _ in range(TIMES):
        seen  = total        # LOAD
        seen  = step(seen)   # <-- handoff point
        total = seen         # STORE


def bump_safe():             # the same three lines, made indivisible
    global total
    for _ in range(TIMES):
        with lock:
            seen  = total
            seen  = step(seen)
            total = seen


def run(fn, label):
    global total
    total = 0
    t0 = time.perf_counter()
    ts = [threading.Thread(target=fn) for _ in range(2)]
    for t in ts: t.start()
    for t in ts: t.join()
    print(f"{label:9} {total:>9,} of 2,000,000   {time.perf_counter() - t0:5.2f}s")


run(bump,      "no lock")
run(bump,      "no lock")
run(bump_safe, "with lock")
run(bump_safe, "with lock")

you see
$ python lock_box.py
no lock   1,371,227 of 2,000,000    0.14s
no lock   1,616,963 of 2,000,000    0.15s
with lock 2,000,000 of 2,000,000    0.95s
with lock 2,000,000 of 2,000,000    0.98s

$ python lock_box.py
no lock   1,784,646 of 2,000,000    0.16s
no lock   1,841,344 of 2,000,000    0.15s
with lock 2,000,000 of 2,000,000    1.00s
with lock 2,000,000 of 2,000,000    0.90s

# four unguarded runs, four different wrong answers.
# four guarded runs, one right answer, every time, at 6x the cost.

The lock must exist — accepted. But “the GIL” is still a fog until you know the exact rule for who holds it and when they hand it off. That rule explains every surprising thing threads do next. →

04One lock, one runner at a time

You have accepted the lock. Now you need its operating manual, because without the precise handoff rule “the GIL” stays a vague fog and threads keep surprising you. The confusion to kill first: are Python threads even real? Yes. Emphatically. threading.Thread creates a genuine operating-system thread — a pthread on Linux and macOS, a native thread on Windows. The kernel schedules them, they can be placed on different physical cores, they are not a simulation. What they cannot do is run the interpreter simultaneously, because advancing the bytecode evaluation loop requires holding the GIL, and there is one GIL. Real threads; one baton; they take turns carrying it.

The rule in a single line: a thread must hold the GIL to execute Python bytecode. Everything else is the mechanics of how the baton gets passed, and it passes in exactly two ways.

The first is voluntary release. Whenever a thread is about to make a blocking call into C — a syscall, time.sleep, a socket recv, a file read — CPython wraps the call in the macro pair Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS. The first macro drops the GIL right before the thread goes to sleep in the kernel. The second reacquires it when the call returns. The logic is obvious once you see it. A thread that is about to block for two seconds has no use for the interpreter lock during those two seconds, so it lets go, and some other thread gets to run. This one mechanism is the entire reason I/O-bound threading works — hold that thought for the next section.

The second is involuntary preemption. A thread doing pure arithmetic never makes a blocking call, so it would never voluntarily let go. It could hog the baton forever and starve every other thread. To prevent that, the interpreter forces a checkpoint. Since Antoine Pitrou's 3.2 redesign, CPython uses a time-based switch interval, default 5 milliseconds, which you can read with sys.getswitchinterval(). A background timer sets a flag — the eval_breaker — when the interval elapses. The running thread checks that flag at the boundary between bytecode instructions. When it sees the flag set, it releases the GIL and signals a waiting thread to take it. And here is the load-bearing detail: the release happens only between bytecode instructions, never in the middle of one.

>>> import sys
>>> sys.getswitchinterval()      # seconds a thread may hold the GIL before a forced checkpoint
0.005
>>> import dis
>>> dis.dis("x += 1")          # one Python line — several bytecodes
  LOAD_NAME     0 (x)
  LOAD_CONST    0 (1)
  BINARY_OP    13 (+=)
  STORE_NAME    0 (x)

Now the two halves click together. Because the GIL is released only between bytecodes, any operation that is a single bytecode is effectively atomic. No other thread can wedge itself into the middle of it. A dict[key] = value insert compiles to one STORE_SUBSCR. It cannot be interrupted mid-store, so concurrent inserts into one dict never corrupt the dict's internal structure. But x += 1, as the disassembly shows, is four bytecodes: load, load, add, store. The scheduler can preempt between the ADD and the STORE, and you are back in the exact lost-update race from section 3, on your own variable this time. This is the single most misunderstood thing about the GIL. It protects ob_refcnt, the interpreter's private bookkeeping, so the interpreter never crashes. It does not protect the logical invariants of your program. Your counter, your list-of-results, your two-line “read balance, then write balance” — those span multiple bytecodes and remain fully racy.

"""race_and_lock.py -- one job, run twice: unguarded, then under a Lock."""
import sys
import threading
import time

ROUNDS = 2_000_000
WORKERS = 2

total = 0
lock = threading.Lock()


def charge(seen):
    """One step of work. Being a call, it is also a real preemption point."""
    return seen + 1


def unguarded():
    global total
    for _ in range(ROUNDS):
        seen = total            # LOAD  -- read the shared value
        seen = charge(seen)     # the GIL can change hands right here
        total = seen            # STORE -- write it back, stale or not


def guarded():
    global total
    for _ in range(ROUNDS):
        with lock:              # nobody can sit between LOAD and STORE
            seen = total
            seen = charge(seen)
            total = seen


def run(fn, label):
    global total
    total = 0
    t0 = time.perf_counter()
    threads = [threading.Thread(target=fn) for _ in range(WORKERS)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
    secs = time.perf_counter() - t0
    want = ROUNDS * WORKERS
    print(f"{label:9} total={total:>9,}  want={want:,}  lost={want - total:>9,}  {secs:5.2f}s")


print("switch interval:", sys.getswitchinterval(), "s   workers:", WORKERS)
for _ in range(3):
    run(unguarded, "no lock")
for _ in range(3):
    run(guarded, "with lock")

$ python race_and_lock.py
switch interval: 0.005 s   workers: 2
no lock   total=2,682,966  want=4,000,000  lost=1,317,034   0.28s
no lock   total=2,882,105  want=4,000,000  lost=1,117,895   0.28s
no lock   total=3,235,045  want=4,000,000  lost=  764,955   0.28s
with lock total=4,000,000  want=4,000,000  lost=        0   1.84s
with lock total=4,000,000  want=4,000,000  lost=        0   5.02s
with lock total=4,000,000  want=4,000,000  lost=        0   5.62s

One more confirmation that these are real kernel threads and not green threads. They share a process id, but each has its own OS-level identity. And two of them pinned to pure computation still deliver only one core's worth of throughput — because the baton, not the cores, is the bottleneck.

import os, threading

def who():
    print(f"pid={os.getpid()}  native_id={threading.get_native_id()}")

who()
t = threading.Thread(target=who); t.start(); t.join()
# same pid (one process), different native_id (two real OS threads)

# ---------- tally.py ----------
"""Is list.append atomic? Yes. Is a read-modify-write? No."""
import sys
import threading
import time

sys.setswitchinterval(1e-6)          # force frequent handoffs, expose the bug fast
PER, WORKERS = 100_000, 4
WANT = PER * WORKERS

lock = threading.Lock()


def drive(body, label):
    seen, counts = [], {"total": 0}
    t0 = time.perf_counter()
    ts = [threading.Thread(target=body, args=(k, seen, counts)) for k in range(WORKERS)]
    for t in ts:
        t.start()
    for t in ts:
        t.join()
    secs = time.perf_counter() - t0
    print(f"{label:22} len(seen)={len(seen):>7,}  counts={counts['total']:>7,}"
          f"  of {WANT:,}  {secs:5.2f}s")


def broken(tag, seen, counts):
    for _ in range(PER):
        seen.append(tag)                                  # (A) one C call
        counts["total"] = counts.get("total", 0) + 1      # (B) get, add, store


def fix_lock(tag, seen, counts):
    for _ in range(PER):
        seen.append(tag)
        with lock:                                        # make (B) one step
            counts["total"] = counts.get("total", 0) + 1


def fix_local(tag, seen, counts):
    mine = 0
    for _ in range(PER):
        seen.append(tag)
        mine += 1                                         # nothing shared: no race
    with lock:                                            # merge once, at the end
        counts["total"] = counts.get("total", 0) + mine


drive(broken,    "broken")
drive(broken,    "broken")
drive(fix_lock,  "fix 1: lock each tick")
drive(fix_local, "fix 2: count locally")


# ------------- $ python tally.py -------------
broken                 len(seen)=400,000  counts=151,292  of 400,000   0.41s
broken                 len(seen)=400,000  counts=146,577  of 400,000   0.55s
fix 1: lock each tick  len(seen)=400,000  counts=400,000  of 400,000   1.57s
fix 2: count locally   len(seen)=400,000  counts=400,000  of 400,000   0.23s

# ------------- $ python tally.py  (again) -------------
broken                 len(seen)=400,000  counts=148,513  of 400,000   0.52s
broken                 len(seen)=400,000  counts=144,148  of 400,000   0.57s
fix 1: lock each tick  len(seen)=400,000  counts=400,000  of 400,000   1.48s
fix 2: count locally   len(seen)=400,000  counts=400,000  of 400,000   0.23s


# ---------- the machine's answer: dis, not folklore ----------
>>> import dis
>>> def f(bag, counts, tag):
...     bag.append(tag)
...     counts['n'] = counts.get('n', 0) + 1
>>> dis.dis(f)
  4    LOAD_FAST     0 (bag)
       LOAD_ATTR     1 (NULL|self + append)
       LOAD_FAST     2 (tag)
       CALL          1                 <-- the ENTIRE mutation is inside this one op
       POP_TOP

  5    LOAD_FAST     1 (counts)
       LOAD_ATTR     3 (NULL|self + get)
       LOAD_CONST    1 ('n')
       LOAD_CONST    2 (0)
       CALL          2                 <-- READ happens here...
       LOAD_CONST    3 (1)
       BINARY_OP     0 (+)
       LOAD_FAST     1 (counts)
       LOAD_CONST    1 ('n')
       STORE_SUBSCR                    <-- ...and WRITE happens here. a gap.


# The three answers.
#
# 1. len(seen) is 400,000 every run. list.append IS atomic -- but not because
#    "lists are thread-safe." It is atomic because the whole mutation lives
#    inside a single CALL bytecode, executing C code that holds the GIL for its
#    entire duration. There is no instant at which another thread can observe a
#    half-appended list, so there is nothing to interleave with.
#
# 2. counts['total'] loses ~60% of its updates. The read (CALL to .get) and the
#    write (STORE_SUBSCR) are separate bytecodes with a gap between them, and a
#    CALL is one of the few instructions where CPython checks whether to hand
#    the GIL over. So the handoff lands exactly in the gap. A thread wakes with
#    a value it read a timeslice ago and stomps it back over everyone's work --
#    which is why the loss is hundreds of thousands, not a handful.
#
# 3. The switch interval only changes how OFTEN a handoff is offered. It cannot
#    make append racy (there is no gap to land in) and it cannot make the
#    counter safe (the gap is structural). Shrinking it just converts a bug you
#    would meet in production into one you meet in 0.4 seconds.
#
#
# Fix 2 wins, and by a lot: 0.23s against 1.57s -- and it beats the BROKEN
# version too. Fix 1 acquires and releases the lock 400,000 times, and every
# acquisition that finds the lock held parks a thread in the kernel and wakes
# it again. That is contention, and contention is the cost. Fix 2 acquires it
# 4 times, once per thread. The general rule this is teaching: the cheapest
# lock is the one you do not take. Keep per-thread work per-thread, and share
# only the summary.

You now hold the two release rules — voluntary on blocking, forced every 5ms. Point them at the two workloads from section 1 and the folklore finally resolves into arithmetic: why compute threads flatline and wait threads fly. →

05The crunch test: I/O overlaps, CPU doesn't

Theory earns its keep only when it predicts a measurement you can run yourself. So take the two release rules from the last section — voluntary drop on blocking, forced 5ms handoff otherwise — and push both workloads from section 1 through them. This is where “I added threads and it got slower” finally gets its full explanation. And this is where you get to watch the two outcomes diverge.

Start with CPU-bound. Two threads each summing a hundred million integers. Neither ever makes a blocking call, so neither ever voluntarily releases the GIL. The only handoff is the forced 5ms checkpoint, and while one thread holds the baton, the other is parked, not computing. So the total arithmetic is unchanged and now serialized through one lock. Two threads do not split the work. They take turns doing all of it. Wall time comes out roughly equal to the single-threaded time, plus the overhead of thousands of GIL acquire/release handoffs and context switches. That is why it often runs slightly slower, and never near half. Run it on 1, 2, 4 threads and the wall-time line stays flat, or drifts upward. That flat line is the definitive fingerprint of the GIL.

you type
# ---------- pool_box.py ----------
import time
from concurrent.futures import ThreadPoolExecutor, as_completed


def fetch(name):
    time.sleep(0.3)
    if name == "bad":
        raise ValueError(f"no such track: {name}")
    return f"{name}:{len(name)}"


names = ["levels", "titanium", "alive", "bad"]

# 1. map -- results in ARGUMENT order; an exception surfaces when you read it
t0 = time.perf_counter()
with ThreadPoolExecutor(max_workers=4) as pool:
    it = pool.map(fetch, names[:3])
    print("map      ", list(it), f"{time.perf_counter()-t0:.2f}s")

# 2. submit -> Future -- results in COMPLETION order, one failure isolated
t0 = time.perf_counter()
with ThreadPoolExecutor(max_workers=4) as pool:
    futures = {pool.submit(fetch, n): n for n in names}
    for fut in as_completed(futures):
        name = futures[fut]
        try:
            print("  done  ", name, "->", fut.result())
        except ValueError as exc:
            print("  failed", name, "->", type(exc).__name__, exc)
print(f"submit    {time.perf_counter()-t0:.2f}s")

you see
$ python pool_box.py
map       ['levels:6', 'titanium:8', 'alive:5'] 0.30s
  done   levels -> levels:6
  done   titanium -> titanium:8
  failed bad -> ValueError no such track: bad
  done   alive -> alive:5
submit    0.30s

# four 0.3s waits finished in 0.30s, and the one failure
# stayed inside its own Future instead of killing the batch.

import time
from concurrent.futures import ThreadPoolExecutor

def crunch(n):
    return sum(i * i for i in range(n))

N = 30_000_000

t = time.perf_counter()                       # sequential: do it twice, back to back
crunch(N); crunch(N)
print("2x sequential:", round(time.perf_counter() - t, 2), "s")

t = time.perf_counter()                       # two threads: hoping for half the time
with ThreadPoolExecutor(max_workers=2) as pool:
    list(pool.map(crunch, [N, N]))
print("2 threads    :", round(time.perf_counter() - t, 2), "s")
# 2x sequential: 4.10 s      2 threads: 4.30 s   — no speedup, a little worse

Now I/O-bound, same harness shape, but the work is a blocking wait instead of arithmetic. Each thread's time.sleep(2) — physically identical to a socket wait — hits Py_BEGIN_ALLOW_THREADS and drops the GIL before parking in the kernel. So while thread A sleeps, the baton is free. Thread B grabs it and starts its own sleep, dropping the baton again for C, and so on. All the waits are in flight at once. Four two-second sleeps that took eight seconds end to end now finish in about two, because the idle time — which was the entire cost — overlaps for free.

import time
from concurrent.futures import ThreadPoolExecutor

def fetch(url):
    time.sleep(2)          # blocking wait — GIL released the whole time
    return url

urls = ["a", "b", "c", "d"]

t = time.perf_counter()
for u in urls: fetch(u)              # sequential: waits stack end to end
print("sequential:", round(time.perf_counter() - t, 2), "s")   # ~8.0 s

t = time.perf_counter()
with ThreadPoolExecutor(max_workers=4) as pool:
    list(pool.map(fetch, urls))       # 4 threads: waits overlap
print("4 threads :", round(time.perf_counter() - t, 2), "s")   # ~2.0 s

Put the two results side by side and the whole chapter resolves into a single sentence. The GIL serializes bytecode execution; it does not serialize kernel waiting. CPU work is bytecode execution, so it serializes — flat line, no speedup. I/O work is kernel waiting with the GIL released, so it overlaps — near-linear speedup until the network or disk saturates. And now the folklore is finally exact. Threads win precisely to the degree your program is waiting rather than computing. That is nothing but section one's cpu/wall ratio read a second time. A ratio near 0 means all waiting, so threads overlap it away. A ratio near 1 means all computing, so the GIL turns your threads into a queue.

"""download_simulation.py -- eight metadata lookups, sequential then pooled."""
import time
from concurrent.futures import ThreadPoolExecutor

TRACKS = ["Blinding Lights", "Titanium", "Levels", "Midnight City",
          "Instant Crush", "Rather Be", "Alive", "Wake Me Up"]

LATENCY = 0.40      # stand-in for one blocking HTTP round trip


def fetch_meta(title):
    """Pretend network call. sleep() really does park the thread and drop the GIL."""
    time.sleep(LATENCY)
    return {"title": title, "seconds": 120 + 7 * len(title)}


def timed(label, fn):
    t0 = time.perf_counter()
    rows = fn()
    secs = time.perf_counter() - t0
    print(f"{label:12} {len(rows)} tracks in {secs:5.2f}s")
    return secs, rows


seq_s, rows = timed("sequential", lambda: [fetch_meta(t) for t in TRACKS])


def pooled():
    with ThreadPoolExecutor(max_workers=8) as pool:
        return list(pool.map(fetch_meta, TRACKS))


par_s, rows = timed("8 threads", pooled)

print()
print(f"speedup      {seq_s / par_s:.2f}x   (perfect overlap would be {len(TRACKS)}.00x)")
print(f"cpu time     {time.process_time():.3f}s of {seq_s + par_s:.2f}s wall -- almost none of it was work")
print(f"first row    {rows[0]}")

$ python download_simulation.py
sequential   8 tracks in  3.21s
8 threads    8 tracks in  0.41s

speedup      7.86x   (perfect overlap would be 8.00x)
cpu time     0.062s of 3.61s wall -- almost none of it was work
first row    {'title': 'Blinding Lights', 'seconds': 225}

$ python download_simulation.py
sequential   8 tracks in  3.21s
8 threads    8 tracks in  0.41s

speedup      7.83x   (perfect overlap would be 8.00x)
cpu time     0.016s of 3.61s wall -- almost none of it was work
first row    {'title': 'Blinding Lights', 'seconds': 225}

So threads are a scalpel, not a hammer — useless on compute, decisive on waiting. The last question is the positive one: exactly when do you reach for them, and how do you share results without re-opening the race? →

06When threads still win

The wrong lesson to walk away with is “threads are useless in Python.” They are not. They are specialized, and knowing their exact envelope means you reach for them with confidence, instead of over-engineering an async rewrite or spinning up processes you don't need. Threads win wherever the bottleneck is waiting — which, thanks to the GIL-release-on-blocking rule, is exactly the work the GIL lets overlap. Two canonical patterns cover almost every real case.

The first is I/O concurrency: many blocking calls in flight at once — dozens of HTTP requests, database queries, file reads. Each spends nearly all its time asleep in a kernel wait with the GIL released. So a ThreadPoolExecutor of N workers gives roughly N-way overlap, right up until the network, disk, or remote service saturates. This is the pragmatic, drop-in win. It needs no async rewrite and works with any ordinary blocking library — requests, a database driver, open(). If your code is already written in plain synchronous style and it is I/O-bound, a thread pool is very often the shortest path from slow to fast.

import time
from concurrent.futures import ThreadPoolExecutor

def fetch(url):
    time.sleep(1)                 # stand-in for a real blocking request
    return len(url)

urls = [f"https://example.com/{i}" for i in range(20)]

t = time.perf_counter()
with ThreadPoolExecutor(max_workers=20) as pool:
    results = list(pool.map(fetch, urls))
print("20 requests in", round(time.perf_counter() - t, 2), "s")
# ~1.0 s instead of ~20 s — the twenty waits overlapped

The second is responsiveness: a GUI event loop or a CLI must keep answering input while a slow operation runs. Run the slow blocking call on a worker thread. Because it releases the GIL while it waits, the main thread keeps getting the baton, and it keeps repainting the window or reading keystrokes. The app stays alive instead of freezing. The classic “UI hangs while a download runs” bug is exactly a blocking call left on the main thread. Moving it to a worker fixes it, precisely because of the release rule you now understand.

import time, threading

def slow_job():
    time.sleep(3)                     # blocking work, off the main thread

worker = threading.Thread(target=slow_job)
worker.start()

while worker.is_alive():           # the 'UI loop' keeps running
    print("still responsive…", flush=True)
    time.sleep(0.5)
worker.join()
print("done")              # the ticks kept printing the whole 3 seconds

Now the honest boundary, because this chapter promised no half-truths. Threads share the heap — that was section 2. So the moment two threads write the same mutable object, your invariants are back in play, and the GIL will not save them. The GIL protects ob_refcnt. It does not protect your counter, your growing list of results, your “read-then-write.” The disciplined fix is to hand data between threads rather than share it. queue.Queue is thread-safe by construction: workers put results, the main thread gets them, so you never touch one object from two threads at once. When you truly must share, wrap the critical section in a Lock.

you type
# ---------- queue_box.py ----------
import queue
import threading
import time

jobs = queue.Queue()          # the hand-off: thread-safe by construction
results = queue.Queue()
DONE = object()               # the poison pill


def consumer():
    while True:
        job = jobs.get()               # blocks until something arrives
        try:
            if job is DONE:            # time to go home
                return
            time.sleep(0.1)            # the blocking part (GIL released)
            results.put((job, job * job))
        finally:
            jobs.task_done()           # one item accounted for


workers = [threading.Thread(target=consumer, name=f"w{i}", daemon=True)
           for i in range(4)]
for w in workers:
    w.start()

t0 = time.perf_counter()
for n in range(12):               # producer: the main thread
    jobs.put(n)

jobs.join()                       # wait until every put() has a task_done()
print(f"12 jobs, 4 workers: {time.perf_counter() - t0:.2f}s")

for _ in workers:                 # one pill per worker, then reap them
    jobs.put(DONE)
for w in workers:
    w.join()

collected = []
while not results.empty():
    collected.append(results.get())
collected.sort()

print("results :", collected[:3], "...", collected[-2:])
print("count   :", len(collected), " threads still alive:", threading.active_count())

you see
$ python queue_box.py
12 jobs, 4 workers: 0.30s
results : [(0, 0), (1, 1), (2, 4)] ... [(10, 100), (11, 121)]
count   : 12  threads still alive: 1

# 12 jobs x 0.1s = 1.2s of waiting, done in 0.30s by 4 workers.
# all 12 results present, and every worker reaped: only MainThread left.

import time, queue, threading

results = queue.Queue()          # thread-safe mailbox — no Lock needed

def worker(job):
    time.sleep(0.5)               # the blocking part (GIL released)
    results.put(job * job)         # hand the result back safely

threads = [threading.Thread(target=worker, args=(i,)) for i in range(10)]
for t in threads: t.start()
for t in threads: t.join()

total = 0
while not results.empty():
    total += results.get()
print("collected", total)      # correct every run — no shared mutable object was raced

Finally, close the loop back to section 1's fork, because the whole chapter is one decision tree. Ask the question: faster network or faster CPU? If the answer is network, the work is I/O-bound. Threads are your tool for moderate fan-out and any blocking library. For very high fan-out, thousands of native thread stacks and their context-switch cost become the ceiling, and async's cooperative single-thread model wins instead (chapter 27). If the answer is CPU, the work is CPU-bound, and threads are the wrong tool entirely. Use processes to get real parallelism, or drop into a C extension that releases the GIL while it computes (chapter 28). Threads are not the general-purpose speed button the folklore sold you. They are the precise instrument for making a program that waits stop waiting one thing at a time — and now you know exactly why.

you type
# ---------- when_box.py ----------
import time
from concurrent.futures import ThreadPoolExecutor

N = 8_000_000


def wait_one(_):                 # I/O-bound: one blocking wait
    time.sleep(0.5)


def crunch(_):                   # CPU-bound: pure arithmetic, no syscall
    return sum(i * i for i in range(N))


def timed(fn, workers, tasks=4):
    t0 = time.perf_counter()
    with ThreadPoolExecutor(max_workers=workers) as pool:
        list(pool.map(fn, range(tasks)))
    return time.perf_counter() - t0


for fn, label in ((wait_one, "sleep(0.5) x4"), (crunch, "sum(i*i) x4  ")):
    one = timed(fn, 1)
    four = timed(fn, 4)
    print(f"{label}  1 worker {one:5.2f}s   4 workers {four:5.2f}s   speedup {one/four:4.2f}x")

you see
$ python when_box.py
sleep(0.5) x4  1 worker  2.00s   4 workers  0.50s   speedup 3.98x
sum(i*i) x4    1 worker  6.86s   4 workers  8.25s   speedup 0.83x

$ python when_box.py
sleep(0.5) x4  1 worker  2.00s   4 workers  0.51s   speedup 3.95x
sum(i*i) x4    1 worker  7.02s   4 workers  8.19s   speedup 0.86x

# one file, one executor, one machine with 32 idle cores.
# the workload picked the outcome, and it picked opposite ones.

# ---------- DRILL A: pages.py ----------
"""A producer/consumer word-count over 15 'pages', 4 worker threads."""
import queue
import threading
import time

PAGES = [f"page-{i:02d}" for i in range(15)]

work = queue.Queue()
done = queue.Queue()
STOP = object()


def fetch_and_count(page):
    time.sleep(0.15)                       # the blocking part -- GIL released
    return len(page) * 3                   # pretend word count


def consumer():
    me = threading.current_thread().name
    while True:
        item = work.get()
        try:
            if item is STOP:
                return
            done.put((me, item, fetch_and_count(item)))
        finally:
            work.task_done()               # every get() gets exactly one


workers = [threading.Thread(target=consumer, name=f"w{i}") for i in range(4)]
for w in workers:
    w.start()

t0 = time.perf_counter()
for page in PAGES:                         # the producer is the main thread
    work.put(page)
work.join()                                # every item accounted for
elapsed = time.perf_counter() - t0

for _ in workers:
    work.put(STOP)
for w in workers:
    w.join()

rows = []
while not done.empty():
    rows.append(done.get())

per_worker = {}
for who, _page, _n in rows:
    per_worker[who] = per_worker.get(who, 0) + 1

print(f"15 pages, 4 workers : {elapsed:.2f}s   (sequential would be 2.25s)")
print("total words         :", sum(n for _w, _p, n in rows))
print("pages per worker    :", dict(sorted(per_worker.items())))
print("rows collected      :", len(rows), " threads alive:", threading.active_count())


# ------------- $ python pages.py -------------
15 pages, 4 workers : 0.60s   (sequential would be 2.25s)
total words         : 315
pages per worker    : {'w0': 4, 'w1': 3, 'w2': 4, 'w3': 4}
rows collected      : 15  threads alive: 1

# ------------- $ python pages.py  (again) -------------
15 pages, 4 workers : 0.60s   (sequential would be 2.25s)
total words         : 315
pages per worker    : {'w0': 4, 'w1': 4, 'w2': 3, 'w3': 4}
rows collected      : 15  threads alive: 1

# 0.60s is 15 pages in 4 rounds of 0.15s -- 4 rounds because 15 does not
# divide by 4. The per-worker split moves between runs (3+4+4+4, in some
# order) and that is correct: nothing assigns pages, the workers race for
# them, and whoever finishes a sleep first grabs the next one.


# ---------- the two deliberate breaks ----------
#
# 1. Delete work.task_done(): the program hangs at work.join(), forever.
#    join() does not watch the queue for emptiness -- it watches an
#    unfinished-task counter that put() raises and task_done() lowers.
#    All 15 pages get fetched and all 15 results arrive in `done`, and the
#    counter still reads 15, so join() never returns. The work being finished
#    and the queue reporting it finished are two different facts.
#
# 2. Send one STOP instead of four: the first worker to reach get() takes it
#    and returns. The other three are blocked inside work.get() on an empty
#    queue with nothing left to arrive. Their w.join() calls never return, so
#    main hangs too -- and being non-daemon threads, they would have kept the
#    process alive even if main had finished. One pill per worker, always.


# ---------- DRILL B: threads or not ----------
#
# (1) 200 REST endpoints at ~300ms each  ->  THREADS.
#     Each request is ~0.3s parked in a kernel wait with the GIL released, and
#     essentially no bytecode. cpu/wall is near 0. A pool of, say, 20 workers
#     turns 60s into ~3s. The ceiling is the remote service, not Python. (Past
#     a few thousand in flight, the per-thread stack cost wins and async is the
#     answer instead -- that is chapter 27.)
#
# (2) SHA-256 of four 64MB blobs  ->  THREADS. This is the surprise.
#     It looks purely CPU-bound, and it is -- but hashlib releases the GIL
#     around the digest of any sizeable buffer, because that work happens in C
#     and touches no Python object. So the threads genuinely run on separate
#     cores. Measured, not guessed:
#
#         sha256 x4, 1 worker(s):  0.12s
#         sha256 x4, 4 worker(s):  0.03s        -- 4.0x, real parallelism
#
#     Same story for zlib, and for most of NumPy. The rule survives intact:
#     threads help while the GIL is not held, and a C extension that drops it
#     is just another way of not holding it.
#
# (3) Summing a 50-million-element list of ints  ->  NEITHER (use processes,
#     or do not thread it). Every add is interpreter bytecode, so the GIL is
#     held from the first element to the last. Two threads take turns and cost
#     you handoffs; expect around 0.85x, i.e. slower. ProcessPoolExecutor over
#     slices gives you real cores, at the price of pickling the slices across
#     the process boundary -- which is chapter 28's whole subject.

Threads overlap waits but hit a per-thread memory wall at massive fan-out. Chapter 27 keeps the overlap and removes the wall — one thread, one event loop, thousands of suspended coroutines. That is async. →