10Conveyors & generators

In Chapter 9 we turned functions into values you can hand around. Now we cash that in. We pour a whole playlist down a conveyor and let machines do the walking. Here's the plan. We map every track through a function, filter for the long ones, then reduce a column of runtimes down to a single total. After that we write the same work as one readable comprehension instead of a hand-cranked loop. And the whole way through, we keep asking the one thing that actually matters: when does the work really happen? The answer is almost never the moment you'd guess. That question is the thread straight into generators. A generator is a function that pauses, a sequence that doesn't exist until you ask for the next item, a million songs modelled in a few hundred bytes. By the end, one thing is obvious: why sorted() hands you a fresh list while .sort() quietly rewrites the old one. And adding "Blinding Lights" at 200s and "Titanium" at 245s, then totalling their runtime, is just two plain lines.

01map and filter: two machines on a conveyor

Let's start with the loop you'd write by hand. You open an empty list, walk the playlist, and append each reshaped item. You did exactly that in Chapter 9, and there's nothing wrong with it. But Python ships two machines that do the walking for you. Picture them bolted over a conveyor belt: items roll past, a machine acts on each one, a new stream flows out the far end. map(fn, stream) takes a function reference — the bare name, no parentheses, Chapter 9's rule. It promises to run every item that rolls past through that function, emitting the reshaped stream. filter(pred, stream) takes a predicate, which is just a function that answers True or False about one item. It keeps only the ones that answer True. Everything else falls off the belt.

Now point them at our playlist and watch what they do. map with an upper-casing function turns every title LOUD; filter with a "longer than five minutes" test keeps only the marathon tracks and lets the radio-length ones drop away. Same conveyor, two different jobs: one machine reshapes every item, the other waves items through or off. And notice — neither one touches the songs on disk. Each spins up a fresh stream flowing out the far end, and the original list just sits there, untouched.

you type
double = lambda n: n * 2
print(double(21))
print(type(double).__name__, "|", double.__name__)

pair = lambda a, b: a + b
print(pair(200, 245))

songs = [("Titanium", 245), ("Blinding Lights", 200), ("Strobe", 607)]

print(sorted(songs, key=lambda s: s[1]))
print(max(songs, key=lambda s: s[1])[0])
print(min(songs, key=lambda s: s[1])[0])
print(sorted(songs, key=lambda s: (len(s[0]), s[0])))

def seconds_of(song):
    """Pull the length out of one (title, seconds) pair."""
    return song[1]

print(sorted(songs, key=seconds_of)[0][0], "|", seconds_of.__name__)

zero = lambda: "no slots at all"
print(zero())
print((lambda a, b=100: a + b)(5))
print(list(filter(lambda s: s[1] > 210, songs)))

you see
42
function | <lambda>
445
[('Blinding Lights', 200), ('Titanium', 245), ('Strobe', 607)]
Strobe
Blinding Lights
[('Strobe', 607), ('Titanium', 245), ('Blinding Lights', 200)]
Blinding Lights | seconds_of
no slots at all
105
[('Titanium', 245), ('Strobe', 607)]

# a stream of songs — the belt everything rides on
playlist = [
    {"title": "Blinding Lights", "artist": "The Weeknd",           "seconds": 200},
    {"title": "Titanium",        "artist": "David Guetta ft. Sia", "seconds": 245},
    {"title": "One More Time",   "artist": "Daft Punk",            "seconds": 320},
    {"title": "Strobe",          "artist": "deadmau5",             "seconds": 634},
]

# map: a function REFERENCE (no parens!) + the belt to feed it
loud = map(lambda s: s["title"].upper(), playlist)
print(loud)              # <map object at 0x...>  — an IOU, not the titles
print(list(loud))        # ['BLINDING LIGHTS', 'TITANIUM', 'ONE MORE TIME', 'STROBE']

# filter: a predicate (True/False for ONE item) + the same belt
long = filter(lambda s: s["seconds"] > 300, playlist)
for s in long:
    print(s["title"])    # One More Time / Strobe  — the rest fell off

One catch, and it's the whole personality of this chapter: neither machine has done a scrap of work yet. Both return a lazy object. print(loud) doesn't show your titles. It shows <map object at 0x...>, a machine sitting idle. Nothing has been computed. It's a promise, not a result. The work happens only when something consumes the stream — list() around it, a for-loop over it, a sum() pulling on it. And even then it pays up one item at a time, computing each only at the instant it's asked for. That sounds like a mere quirk until you meet the job that needs it. Picture a stream you could never hold all at once: a 50 GB log on an 8 GB laptop, or a sequence with no end. An eager list dies on both. A machine that computes one item on demand sails straight through. That gap is the felt need this whole chapter is built to answer.

>>> lengths = map(int, ["200", "245", "607"])
>>> 245 in lengths
>>> 200 in lengths
>>> list(lengths)

True
False
[]

map keeps one more trick up its sleeve. Hand it several streams and it walks them in lockstep, plucking one item from each per step and handing the whole bunch to your function. map(min, radio, extended) pairs each track's radio-edit length with its extended-mix length and keeps the shorter — [200, 210, 190, 200]. It stops the instant the shortest stream runs dry, so feeding it mismatched lengths never throws. And one honesty note on filter: it keeps every item whose predicate comes back truthy, not literally True. Truthy is simply what you get when Python runs bool() on a value. A short, fixed list of things count as falsy: 0, 0.0, "", None, and any empty container ([], {}). Everything else is truthy. That's why the barest predicate of all, filter(None, stream), needs no function at all. With None in the predicate slot, filter tests each item's own truthiness, drops every falsy one, and keeps the rest.

you type
titles = ["Blinding Lights", "Titanium", "Strobe", "Levels"]
seconds = [200, 245, 607, 203]

loud = map(str.upper, titles)
print(type(loud).__name__, "- built, and nothing computed yet")
print(list(loud))
print(list(loud), "- the same machine, second pass")

print(list(map(len, titles)))

radio    = [200, 245, 607, 203]
extended = [210, 240, 634, 190]
print(list(map(min, radio, extended)))

print(list(filter(lambda s: s > 300, seconds)))
print(list(filter(None, [0, 200, "", "Titanium", None, [], 245])))
print(sum(map(lambda s: s // 60, seconds)))

import sys
print(sys.getsizeof(map(str.upper, titles)),
      sys.getsizeof(filter(None, seconds)), "bytes, whatever they wrap")

try:
    list(map(str.upper(), titles))
except TypeError as e:
    print("TypeError:", e)

you see
map - built, and nothing computed yet
['BLINDING LIGHTS', 'TITANIUM', 'STROBE', 'LEVELS']
[] - the same machine, second pass
[15, 8, 6, 6]
[200, 240, 607, 190]
[607]
[200, 'Titanium', 245]
20
48 48 bytes, whatever they wrap
TypeError: unbound method str.upper() needs an argument

map turned four songs into four titles; filter kept two survivors. But the playlist's total runtime is one number squeezed out of all four lengths at once — and neither machine folds a stream down to a single value. So which one does? →

02reduce: fold many values into one

map hands back one new item for every item you gave it — many to many. filter keeps a subset — many to some. reduce does the third thing, and the most radical. It takes a whole sequence and folds it down to a single value — many to one. Total runtime of a playlist. The longest track. A merged index. Drop beneath the song titles to the metal, too. A three-minute track is a river of raw audio samples: millions of little integers, each one a byte pattern of the kind chapter 0 first switched on. And reduce is exactly what folds that whole wall of numbers down to the single loudest one, the peak. Any time a pile of things has to collapse into one answer, you are looking at a fold.

It's the only one of the three that Python doesn't hand you for free. You import it: from functools import reduce. And it works by carrying a single running result called the accumulator. Your function takes exactly two parameters, the accumulator always first and the next item second, and returns the new accumulator. reduce seeds that accumulator with a starting value (here 0, for "no seconds counted yet"). Then it feeds the playlist through one song at a time, each step rolling the last result forward: 0+200→200, 200+245→445, 445+203→648, 648+215→863. The last accumulator standing is your answer.

from functools import reduce

total = reduce(lambda acc, song: acc + song["seconds"], playlist, 0)
print(total)          # 863
# fold: 0+200 → 200 | +245 → 445 | +203 → 648 | +215 → 863

Five moving parts — reduce, map, filter, lambda, and a list() to catch it all — is a lot of machinery for a one-line job. Python looked at map-then-filter and decided most of it was ceremony. Next: the comprehension, which folds a map and a filter into one line you can read like a sentence →

03Comprehensions: map + filter in one readable line

You have a list of songs and you want a new list from it: just the long ones, or every title upper-cased, or each length converted to minutes. In most languages that is a little ritual. You set up an empty accumulator, loop, test, append, then read the accumulator back out. Python has a purpose-built spelling that states the whole intent on one line you can read aloud — the comprehension. Its shape is [expr for x in xs if cond]. It fuses the two oldest moves in all of programming into a single expression that evaluates to a finished container: map (turn each item into something else) and filter (throw some items away).

Read it in three slots, front to back. The expression up front says what each kept item becomes. The for clause names which items you visit. The trailing if is the gate: an item only survives into the result if that condition is True. Drop the if entirely and nothing is filtered — you keep every item, merely reshaped by the expression. That is the whole grammar; everything else is variations on those three slots.

you type
playlist = [
    {"title": "Blinding Lights", "artist": "The Weeknd",   "seconds": 200},
    {"title": "Titanium",        "artist": "David Guetta", "seconds": 245},
    {"title": "Save Your Tears", "artist": "The Weeknd",   "seconds": 215},
    {"title": "One More Time",   "artist": "Daft Punk",    "seconds": 320},
    {"title": "Strobe",          "artist": "deadmau5",     "seconds": 607},
]

# TRANSFORM -- no gate, so every song comes out the far end
print([s["seconds"] // 60 for s in playlist])

# FILTER -- the if goes AFTER the for, and it has no else
print([s["title"] for s in playlist if s["seconds"] > 300])

# BOTH -- the gate runs first, the expression only for survivors
print([s["title"].upper() for s in playlist if s["artist"] == "The Weeknd"])

# TERNARY -- lives in the result slot, and it must have an else
print(["long" if s["seconds"] > 300 else "short" for s in playlist])

# NESTED -- read left to right, exactly as you would stack the loops
grid = [[200, 245], [215, 320], [607]]
print([x for row in grid for x in row])

# TWO GATES -- they simply and together
print([s["title"] for s in playlist if s["seconds"] > 200 if s["seconds"] < 400])

# the loop name never escapes the brackets
squares = [n * n for n in range(4)]
print(squares)
try:
    print(n)
except NameError as e:
    print("NameError:", e)

you see
[3, 4, 3, 5, 10]
['One More Time', 'Strobe']
['BLINDING LIGHTS', 'SAVE YOUR TEARS']
['short', 'short', 'short', 'long', 'long']
[200, 245, 215, 320, 607]
['Titanium', 'Save Your Tears', 'One More Time']
[0, 1, 4, 9]
NameError: name 'n' is not defined

Two placements trip up everyone, and they are worth burning in now, because both use the word if. A filtering condition goes after the for: [x for x in xs if cond]. But a transforming ternary goes before the for: [a if cond else b for x in xs]. That ternary is the one-line a if cond else b that reshapes each item. Same keyword, two entirely different jobs. After the for, the if decides which items get in. Before it, sitting inside the result slot, the if/else decides what shape each kept item takes. The tell is the else. A filter never has one, because there is no "else, keep it out" — the item simply isn't produced. A transforming ternary always does.

# pipeline.py -- raw text in, a printable report out, one comprehension per stage

RAW = [
    "Blinding Lights|The Weeknd|200",
    "Titanium|David Guetta ft. Sia|245",
    "  Save Your Tears|The Weeknd|215  ",
    "One More Time|Daft Punk|320",
    "",
    "Strobe|deadmau5|607",
    "bad guy|Billie Eilish|194",
]

# STAGE 1 -- clean: strip the whitespace, drop the blank rows
rows = [line.strip() for line in RAW if line.strip()]
print("1 clean     :", len(rows), "rows kept of", len(RAW))

# STAGE 2 -- parse: one string becomes one dict (a lazy split feeds the unpack)
songs = [
    {"title": t, "artist": a, "seconds": int(secs)}
    for t, a, secs in (row.split("|") for row in rows)
]
print("2 parse     :", songs[0])

# STAGE 3 -- filter: only tracks over 3:30 clear the gate
longs = [s for s in songs if s["seconds"] > 210]
print("3 gate      :", [s["title"] for s in longs])

# STAGE 4 -- format: reshape each survivor into one printable line
def clock(seconds):
    """245 -> '4:05'."""
    return f"{seconds // 60}:{seconds % 60:02d}"

lines = [f"{s['title']:<16}{clock(s['seconds']):>6}  {s['artist']}" for s in longs]
print("4 format    :", len(lines), "lines built")
print()
print("\n".join(lines))
print()
print("runtime kept:", sum(s["seconds"] for s in longs), "of",
      sum(s["seconds"] for s in songs), "seconds")
print("RAW is untouched:", RAW[2])

1 clean     : 6 rows kept of 7
2 parse     : {'title': 'Blinding Lights', 'artist': 'The Weeknd', 'seconds': 200}
3 gate      : ['Titanium', 'Save Your Tears', 'One More Time', 'Strobe']
4 format    : 4 lines built

Titanium          4:05  David Guetta ft. Sia
Save Your Tears   3:35  The Weeknd
One More Time     5:20  Daft Punk
Strobe           10:07  deadmau5

runtime kept: 1387 of 1781 seconds
RAW is untouched:   Save Your Tears|The Weeknd|215  

# playlist: five dicts, each with title / artist / seconds
# "Blinding Lights" 200 · "Titanium" 245 · "Save Your Tears" 215
# "One More Time" 320 · "Strobe" 607

# FILTER — the if goes AFTER the for: keep only the long ones
long_titles = [s["title"] for s in playlist if s["seconds"] > 300]
# ['One More Time', 'Strobe']

# TRANSFORM — the ternary goes BEFORE the for: reshape every song
tags = ["long" if s["seconds"] > 300 else "short" for s in playlist]
# ['short', 'short', 'short', 'long', 'long']

# swap the brackets, swap the container — same three slots
durations = {s["title"]: s["seconds"] for s in playlist}   # dict comp
artists   = {s["artist"] for s in playlist}                # set comp
print(len(artists))   # 4 — The Weeknd appears twice but a set keeps one

The last block is the quietly brilliant part: the brackets on the outside pick the container, and swapping them costs you nothing. Square brackets build a list. Curly braces build a dict when the expression is a key: value pair, or a set when it is a lone value. And a set, being a set, silently drops repeats. That is exactly why pulling artist out of five songs hands you back four unique names, not five. Slide the gate below and watch a filtering comprehension rebuild its list in real time.

you type
playlist = [
    {"title": "Blinding Lights", "artist": "The Weeknd",   "seconds": 200},
    {"title": "Titanium",        "artist": "David Guetta", "seconds": 245},
    {"title": "Save Your Tears", "artist": "The Weeknd",   "seconds": 215},
    {"title": "One More Time",   "artist": "Daft Punk",    "seconds": 320},
    {"title": "Strobe",          "artist": "deadmau5",     "seconds": 607},
]

# DICT COMP -- the colon is the whole difference
durations = {s["title"]: s["seconds"] for s in playlist}
print(durations["Strobe"], "|", len(durations), "entries")

# duplicate keys do not error. The LAST one silently wins.
by_artist = {s["artist"]: s["title"] for s in playlist}
print(by_artist)

# SET COMP -- a lone value, no colon. Repeats collapse.
artists = {s["artist"] for s in playlist}
print(len(artists), sorted(artists))

# the same three slots: gate at the end, expression up front
minutes = {t: secs // 60 for t, secs in durations.items() if secs > 210}
print(minutes)

# THE FLIP -- swap the halves of every pair, in one line
title_of = {secs: t for t, secs in durations.items()}
print(title_of[607], "|", title_of[200])

print(type({}).__name__, type({1}).__name__, type(set()).__name__)

try:
    {[1, 2]: "nope" for _ in range(1)}
except TypeError as e:
    print("TypeError:", e)

you see
607 | 5 entries
{'The Weeknd': 'Save Your Tears', 'David Guetta': 'Titanium', 'Daft Punk': 'One More Time', 'deadmau5': 'Strobe'}
4 ['Daft Punk', 'David Guetta', 'The Weeknd', 'deadmau5']
{'Titanium': 4, 'Save Your Tears': 3, 'One More Time': 5, 'Strobe': 10}
Strobe | Blinding Lights
dict set set
TypeError: unhashable type: 'list'

There is one property of every comprehension you have written so far that is easy to miss precisely because it is invisible: a comprehension is eager. The instant the line finishes running, the entire container exists in memory, every element computed and stored, all at once. For five songs that is nothing. But ask for [n*n for n in range(1_000_000)] and Python obediently parks about 8 megabytes of finished list in RAM before you so much as glance at the first element. The round-bracket version from the curio weighs roughly 200 bytes, because it has built nothing. Hold that gap. It is the hinge to the next section.

lengths = {"Blinding Lights": 200, "Titanium": 245,
           "Save Your Tears": 215, "Levels": 203}

# 1 -- THE ONE-LINER: swap the two halves of every pair
by_length = {secs: title for title, secs in lengths.items()}
print(by_length)
print(by_length[245])

# 2 -- THE COLLISION: add a second 200-second song and run it again
lengths["One Dance"] = 200
naive = {secs: title for title, secs in lengths.items()}
print(len(lengths), "songs in ->", len(naive), "keys out")
print(naive[200])

# 3 -- THE HONEST FLIP, still a comprehension: each key keeps a LIST
grouped = {secs: [t for t, s in lengths.items() if s == secs]
           for secs in set(lengths.values())}
for secs in sorted(grouped):
    print(secs, "->", grouped[secs])

# 4 -- the same grouping in one pass, because the comprehension above is n squared
one_pass = {}
for title, secs in lengths.items():
    one_pass.setdefault(secs, []).append(title)
print(one_pass == grouped)

# 5 -- and the flip only works while the values are hashable
playlists = {"gym": ["Titanium", "Levels"], "focus": ["Strobe"]}
try:
    {v: k for k, v in playlists.items()}
except TypeError as e:
    print("TypeError:", e)

# ------------- what it prints -------------
{200: 'Blinding Lights', 245: 'Titanium', 215: 'Save Your Tears', 203: 'Levels'}
Titanium
5 songs in -> 4 keys out
One Dance
200 -> ['Blinding Lights', 'One Dance']
203 -> ['Levels']
215 -> ['Save Your Tears']
245 -> ['Titanium']
True
TypeError: unhashable type: 'list'

A comprehension of a million items builds all million before you touch the first — and throws the whole thing away if you only needed one. What if the sequence could stay a promise, handing you elements one at a time, only ever computing the next when you ask? →

playlist = [
    {"title": "Blinding Lights", "artist": "The Weeknd",           "seconds": 200},
    {"title": "Titanium",        "artist": "David Guetta ft. Sia", "seconds": 245},
    {"title": "Save Your Tears", "artist": "The Weeknd",           "seconds": 215},
    {"title": "One More Time",   "artist": "Daft Punk",            "seconds": 320},
    {"title": "Strobe",          "artist": "deadmau5",             "seconds": 607},
]

# DRILL 1 -- a pure transform. Folds cleanly.
loop_1 = []
for s in playlist:
    loop_1.append(s["title"].upper())
comp_1 = [s["title"].upper() for s in playlist]
print("1 loop:", loop_1)
print("1 comp:", comp_1, "| same:", loop_1 == comp_1)

# DRILL 2 -- a gate and a transform. Gate goes AFTER the for.
loop_2 = []
for s in playlist:
    if s["seconds"] > 210:
        loop_2.append(f"{s['title']} ({s['seconds'] // 60}:{s['seconds'] % 60:02d})")
comp_2 = [f"{s['title']} ({s['seconds'] // 60}:{s['seconds'] % 60:02d})"
          for s in playlist if s["seconds"] > 210]
print("2 loop:", loop_2)
print("2 comp:", comp_2, "| same:", loop_2 == comp_2)

# DRILL 3 -- this one STAYS a loop. It builds no list; it does things.
running = 0
skipped = []
for s in playlist:
    if s["seconds"] > 600:
        skipped.append(s["title"])
        continue
    running += s["seconds"]
    print(f"   queued {s['title']:<16} running total {running:>4}s")
print("3 total  :", running, "| skipped:", skipped)

# the "clever" comprehension that does the same thing -- and why it is wrong
sink = [print("   NEVER DO THIS", s["title"]) for s in playlist[:2]]
print("3 sink   :", sink)

# ------------- what it prints -------------
1 loop: ['BLINDING LIGHTS', 'TITANIUM', 'SAVE YOUR TEARS', 'ONE MORE TIME', 'STROBE']
1 comp: ['BLINDING LIGHTS', 'TITANIUM', 'SAVE YOUR TEARS', 'ONE MORE TIME', 'STROBE'] | same: True
2 loop: ['Titanium (4:05)', 'Save Your Tears (3:35)', 'One More Time (5:20)', 'Strobe (10:07)']
2 comp: ['Titanium (4:05)', 'Save Your Tears (3:35)', 'One More Time (5:20)', 'Strobe (10:07)'] | same: True
   queued Blinding Lights  running total  200s
   queued Titanium         running total  445s
   queued Save Your Tears  running total  660s
   queued One More Time    running total  980s
3 total  : 980 | skipped: ['Strobe']
   NEVER DO THIS Blinding Lights
   NEVER DO THIS Titanium
3 sink   : [None, None]

04Generators: sequences that don't exist yet

Every collection you've met so far is already there. A list of song lengths is a million boxes filled and sitting in RAM before you read a single one. And a song's length is only its human label. Drop one altitude and Blinding Lights is its actual bytes: a run of roughly 8.8 million signed 16-bit samples, each one a speaker-cone position. A generator flips that on its head. It's a function that says yield where an ordinary function would say return, and that one word changes what "calling" even means. Call it and none of the body runs. Not the first line, not the loop, nothing. What you get back isn't a value. It's a generator object holding one thing: a suspended frame. A running function always has a frame, its private scratch space of local variables plus a pointer to the exact line it is on. Chapter 9 stacked one such frame on every call. A generator simply freezes that frame instead of discarding it when the function pauses: the frozen locals, and a bookmark on the line to resume at, parked at the top and waiting.

The values arrive one at a time, only when something asks. A for loop asks. next() asks. sum() asks. Each request wakes the body and runs it forward until it evaluates the next yield expression. Then it hands over that single value and freezes on the spot, every local variable still standing exactly where it was. Ask again and it thaws from that same line, not from the beginning. This is lazy evaluation: nothing is computed until the instant it's needed. The body advances by exactly one yield per knock. However many statements sit between two yields, one next() runs precisely that far and no further. The sequence doesn't exist in memory. It exists as a promise to produce the next item when you knock.

you type
def countdown(n):
    print(f"   [body woke up, n={n}]")
    while n > 0:
        yield n
        n -= 1
    print("   [body fell off the end]")

gen = countdown(3)
print(type(gen).__name__, "- called, and not one body line has run")

print(next(gen))
print(next(gen))
print(next(gen))

try:
    next(gen)
except StopIteration:
    print("StopIteration - this generator is empty forever")

print(next(gen, "nothing left"))

print(list(countdown(2)))
print(sum(countdown(4)))

again = countdown(2)
print("first pass :", list(again))
print("second pass:", list(again))

def only_two():
    yield "Titanium"
    return "this value never reaches a for loop"
    yield "unreachable"

print(list(only_two()))

you see
generator - called, and not one body line has run
   [body woke up, n=3]
3
2
1
   [body fell off the end]
StopIteration - this generator is empty forever
nothing left
   [body woke up, n=2]
   [body fell off the end]
[2, 1]
   [body woke up, n=4]
   [body fell off the end]
10
   [body woke up, n=2]
   [body fell off the end]
first pass : [2, 1]
second pass: []
['Titanium']

Remember the round brackets from the last section? (expr for x in xs) is a generator expression, a comprehension that builds nothing. Swap a list comprehension's square brackets for round ones and the memory story changes from brutal to free. A list of a million squares materialises a million boxed int objects plus a million-slot pointer array. On this machine that's ≈ 40 MB, every byte of it paid up front, before you so much as glance at the first square. The generator version is a single small object — exactly 200 bytes on this Python — and it stays 200 bytes whether it will eventually yield ten values or ten trillion. It stores the recipe, so its size has nothing to do with how much meal it can cook.

you type
seconds = [200, 245, 215, 320, 607]

gen = (s // 60 for s in seconds)
print(type(gen).__name__, "- round brackets, nothing computed")
print(next(gen), next(gen))
print(list(gen), "- only what was still ahead of the bookmark")
print(list(gen), "- one-shot, and it is spent")

print(sum(s for s in seconds if s > 210))
print(max(s for s in seconds), min(s for s in seconds))
print(any(s > 600 for s in seconds), all(s > 100 for s in seconds))
print(" ".join(str(s // 60) for s in seconds))
print(sum((s for s in seconds), 1000))

titles = ["Blinding Lights", "Titanium", "Strobe"]
print(sorted((t.lower() for t in titles), key=len))

print(type((s for s in seconds)).__name__, type([s for s in seconds]).__name__)
print(type((200, 245, 607)).__name__)

try:
    eval("sum(s for s in seconds, 0)")
except SyntaxError as e:
    print("SyntaxError:", e.msg)

you see
generator - round brackets, nothing computed
3 4
[3, 5, 10] - only what was still ahead of the bookmark
[] - one-shot, and it is spent
1387
607 200
True True
3 4 3 5 10
2587
['strobe', 'titanium', 'blinding lights']
generator list
tuple
SyntaxError: Generator expression must be parenthesized

def song_lengths(songs):
    for song in songs:
        yield song["seconds"]

gen = song_lengths(playlist)   # nothing runs yet — you hold a paused recipe
print(next(gen))   # 200  — runs to the first yield, then freezes
print(next(gen))   # 245  — wakes exactly where it paused

squares_list = [x*x for x in range(1_000_000)]   # ~40 MB, built NOW
squares_gen  = (x*x for x in range(1_000_000))   # 200 bytes, builds nothing
print(sum(squares_gen))   # 333332833333500000 — one square at a time

# memory_honest.py -- the same million squares, weighed two ways

import sys

N = 1_000_000

# THE EAGER WAY -- every square exists before you read the first one
squares_list = [n * n for n in range(N)]
print("list      :", type(squares_list).__name__)
print("  len                :", len(squares_list))
print("  getsizeof           :", sys.getsizeof(squares_list), "bytes")
print("  sum                 :", sum(squares_list))

# THE LAZY WAY -- nothing exists yet; only the recipe does
squares_gen = (n * n for n in range(N))
print("generator :", type(squares_gen).__name__)
print("  getsizeof           :", sys.getsizeof(squares_gen), "bytes")
print("  sum                 :", sum(squares_gen))

# getsizeof weighs ONE object, never the web of objects it points into
one_int = squares_list[-1]
print()
print("  one int object      :", sys.getsizeof(one_int), "bytes")
print("  pointer array alone :", sys.getsizeof(squares_list), "bytes")
print("  honest total (approx):",
      sys.getsizeof(squares_list) + N * sys.getsizeof(one_int), "bytes")

# and the two objects behave differently on a second reading
print()
print("  sum the list again  :", sum(squares_list))
print("  sum the gen again   :", sum(squares_gen))

# the same fold with no name and no list -- the shape you will actually write
print()
print("  straight through    :", sum(n * n for n in range(N)))
print("  peak sample         :", max(abs(s) for s in (0, 12000, -22000, 3000)))

list      : list
  len                : 1000000
  getsizeof           : 8448728 bytes
  sum                 : 333332833333500000
generator : generator
  getsizeof           : 200 bytes
  sum                 : 333332833333500000

  one int object      : 32 bytes
  pointer array alone : 8448728 bytes
  honest total (approx): 40448728 bytes

  sum the list again  : 333332833333500000
  sum the gen again   : 0

  straight through    : 333332833333500000
  peak sample         : 22000

To feel the freeze-and-resume, drive it by hand. Below, each nudge of the slider is one next(gen) on our four-song playlist. Watch the bookmark walk down to the yield, hand you a value, and stop, locals still warm. Then it picks up from the very same line on the next call. Push it one step past the end and you'll meet the signal that ends every loop.

Now slide the count of items to a million and watch who pays. The list's cost climbs in a dead-straight line; the generator's never twitches off its 200-byte floor.

Lazy or eager, a playlist eventually wants to be in ORDER — and Python ships two ways to sort it that look almost identical yet behave in opposite ways. One returns a new list; the other quietly rewrites yours and hands back nothing. →

import sys
from itertools import islice

# 1 -- the bounded generator: n evens, then it stops on its own
def first_n_evens(n):
    """Yield the first n even whole numbers, starting at 0."""
    count = 0
    value = 0
    while count < n:
        yield value
        value += 2
        count += 1

print(list(first_n_evens(5)))
print(sum(first_n_evens(5)))

gen = first_n_evens(3)
print(next(gen), next(gen), next(gen))
try:
    next(gen)
except StopIteration:
    print("StopIteration - it ran out, exactly on schedule")

# 2 -- the endless one: no stopping condition at all
def evens():
    value = 0
    while True:
        yield value
        value += 2

print(list(islice(evens(), 5)))
print(sys.getsizeof(evens()), "bytes -- and it promises every even number there is")

# 3 -- the same job as a generator expression, when the rule is simple enough
print(list(islice((n for n in range(0, 1_000_000) if n % 2 == 0), 5)))

# 4 -- one-shot, so a second walk finds nothing
g = first_n_evens(4)
print("first walk :", list(g))
print("second walk:", list(g))

# ------------- what it prints -------------
[0, 2, 4, 6, 8]
20
0 2 4
StopIteration - it ran out, exactly on schedule
[0, 2, 4, 6, 8]
184 bytes -- and it promises every even number there is
[0, 2, 4, 6, 8]
first walk : [0, 2, 4, 6]
second walk: []

05sorted() vs .sort(): the method/function rule

Two ways to put your playlist in order, spelled almost identically, and they do opposite things to your data. Learn the difference once and you've learned a rule that runs through the whole language, far past sorting. A dot-method reaches into the object and edits that very object, then hands you back None. A wrapping function reads the object, leaves it untouched, and hands you back a brand-new value. Say it out loud with the playlist in hand. playlist.sort() grabs the actual list and reorders the references right where they sit. It never builds a second list object, so the old order is gone for good. (Not literally zero extra memory: the sort borrows a small temporary buffer, up to about half the list, to merge through, but nothing the size of a whole second list.) sorted(playlist) does build that second list. It makes a fresh copy, arranges it, and gives it to you. Your original never so much as twitches.

There's a second split hiding in the punctuation. .sort() is a method, so it lives on lists, and on nothing else. Hand it a tuple or a set and Python shrugs: 'tuple' object has no attribute 'sort'. But sorted() is a free-standing function that will happily chew through any iterable — a tuple, a set, a dict, even a one-shot generator that hasn't produced its values yet. And it always answers with a list. One is a specialist bolted to one type. The other is a generalist that meets your data wherever it lives.

Both accept the same steering wheel, key=: you hand it a function reference, the name, no parentheses. Each item is then judged not by itself but by whatever that function returns for it — a stand-in, a proxy value. Want the playlist ordered by length? Use key=lambda s: s["seconds"], and every song is compared by its number of seconds instead of by the whole dict. That's chapter 9's first-class functions cashing in: a function you can pass around like any other value. Add reverse=True and the same ordering runs longest-first. Miss the "reference, not call" part and you get a face-full of error. key=len() tries to run len right now, with no argument, and dies. key=len passes the function along to be called later, once per item.

you type
playlist = [
    {"title": "Blinding Lights", "artist": "The Weeknd",   "seconds": 200},
    {"title": "Titanium",        "artist": "David Guetta", "seconds": 245},
    {"title": "Save Your Tears", "artist": "The Weeknd",   "seconds": 215},
    {"title": "One More Time",   "artist": "Daft Punk",    "seconds": 320},
    {"title": "Strobe",          "artist": "deadmau5",     "seconds": 607},
]
def names(rows):
    return [s["title"] for s in rows]

# the FUNCTION: builds a new list, leaves yours alone
by_len = sorted(playlist, key=lambda s: s["seconds"])
print(names(by_len))
print(names(playlist))

longest_first = sorted(playlist, key=lambda s: s["seconds"], reverse=True)
print(names(longest_first))

# a TUPLE key sorts on two fields at once, left field first
by_artist = sorted(playlist, key=lambda s: (s["artist"], s["seconds"]))
print(names(by_artist))

# sorted() eats any iterable, even a generator, and always returns a list
print(sorted((s["title"] for s in playlist), key=len))

# one extreme is not a sorting job
print(max(playlist, key=lambda s: s["seconds"])["title"],
      "|", min(playlist, key=lambda s: s["seconds"])["title"])

# key runs once per item, not once per comparison
calls = []
def watched(song):
    calls.append(song["title"])
    return song["seconds"]

sorted(playlist, key=watched)
print(len(playlist), "songs ->", len(calls), "key calls")

# the METHOD: rewrites your list, hands back None
result = playlist.sort(key=lambda s: s["seconds"])
print(result, "|", names(playlist))

try:
    sorted(playlist, key=len())
except TypeError as e:
    print("TypeError:", e)

you see
['Blinding Lights', 'Save Your Tears', 'Titanium', 'One More Time', 'Strobe']
['Blinding Lights', 'Titanium', 'Save Your Tears', 'One More Time', 'Strobe']
['Strobe', 'One More Time', 'Titanium', 'Save Your Tears', 'Blinding Lights']
['One More Time', 'Titanium', 'Blinding Lights', 'Save Your Tears', 'Strobe']
['Strobe', 'Titanium', 'One More Time', 'Blinding Lights', 'Save Your Tears']
Strobe | Blinding Lights
5 songs -> 5 key calls
None | ['Blinding Lights', 'Save Your Tears', 'Titanium', 'One More Time', 'Strobe']
TypeError: len() takes exactly one argument (0 given)

# the function: reads the playlist, builds a NEW list, original untouched
by_length = sorted(playlist, key=lambda s: s["seconds"])
print(by_length[0]["title"])    # Old Town Road — shortest first
print(playlist[0]["title"])     # Blinding Lights — the original never moved

# the method: reorders the actual list, hands back nothing
result = playlist.sort(key=lambda s: s["seconds"], reverse=True)
print(result)                   # None — .sort() returns None, always
print(playlist[0]["title"])     # One More Time — the list ITSELF was reordered

Reading about it is one thing; watching your list's fate flip under your hand is another. Below is one playlist of four songs, drawn as bars — taller is longer. Slide through the four calls you might write. Watch two questions answer themselves at once: did the original playlist change? and what did the call hand back? Each run starts from the same fresh playlist, so you can compare them cleanly.

# key returns a PROXY: sort by whole minutes, not exact seconds
for s in sorted(playlist, key=lambda s: s["seconds"] // 60):
    print(s["title"], s["seconds"] // 60)
# Blinding Lights 3   ← both land in minute-bucket 3,
# Levels 3            ← and keep their ORIGINAL order (stable sort)
# Titanium 4

The mirror image lives one verb over. reversed(order) is a function. It leaves order untouched and hands back a new reversed iterator. order.reverse() is a method. It flips the list in place and returns None. That's exactly the same split as sorted versus .sort, which is the whole point. It was never a one-off rule about ordering. It's the shape of the language.

  File "C:\tmp\gate.py", line 2
    longs = [t.upper() if len(t) > 6 for t in titles]
             ^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: expected 'else' after 'if' expression

Traceback (most recent call last):
  File "C:\tmp\peek.py", line 2, in <module>
    first = lengths[0]
            ~~~~~~~^^^
TypeError: 'generator' object is not subscriptable

Traceback (most recent call last):
  File "C:\tmp\ranked.py", line 3, in <module>
    print(songs[0])
          ~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

# top_tracks.py -- one small report, built entirely out of this chapter

playlist = [
    {"title": "Blinding Lights", "artist": "The Weeknd",           "seconds": 200, "plays": 4120},
    {"title": "Titanium",        "artist": "David Guetta ft. Sia", "seconds": 245, "plays": 2870},
    {"title": "Save Your Tears", "artist": "The Weeknd",           "seconds": 215, "plays": 3990},
    {"title": "One More Time",   "artist": "Daft Punk",            "seconds": 320, "plays": 1510},
    {"title": "Strobe",          "artist": "deadmau5",             "seconds": 607, "plays":  640},
    {"title": "Levels",          "artist": "Avicii",               "seconds": 203, "plays": 2210},
]

def clock(seconds):
    """203 -> '3:23'."""
    return f"{seconds // 60}:{seconds % 60:02d}"

# 1 -- a generator fold: the whole runtime, and no list is ever built
total = sum(s["seconds"] for s in playlist)

# 2 -- a comprehension gate: the tracks people actually play
popular = [s for s in playlist if s["plays"] >= 2000]

# 3 -- sorted with a tuple key: most played first, ties broken by title
ranked = sorted(popular, key=lambda s: (-s["plays"], s["title"]))

# 4 -- a set comp feeding a dict comp: each artist's longest track
longest = {
    artist: max((s for s in playlist if s["artist"] == artist),
                key=lambda s: s["seconds"])["title"]
    for artist in {s["artist"] for s in playlist}
}

print(f"TOP TRACKS   {len(playlist)} songs, {clock(total)} of music")
print("-" * 48)
for rank, s in enumerate(ranked, start=1):
    print(f"{rank}. {s['title']:<16}{s['plays']:>6} plays   {clock(s['seconds'])}")
print("-" * 48)
print("in this table :", sum(s["plays"] for s in ranked), "plays,",
      sum(s["seconds"] for s in ranked), "seconds")
print("share of runtime:", f"{sum(s['seconds'] for s in ranked) / total:.0%}")
print("cut for low plays:", [s["title"] for s in playlist if s["plays"] < 2000])
print()
for artist in sorted(longest):
    print(f"  {artist:<22} longest: {longest[artist]}")
print()
print("playlist order is untouched:", [s["title"] for s in playlist][:3], "...")

TOP TRACKS   6 songs, 29:50 of music
------------------------------------------------
1. Blinding Lights   4120 plays   3:20
2. Save Your Tears   3990 plays   3:35
3. Titanium          2870 plays   4:05
4. Levels            2210 plays   3:23
------------------------------------------------
in this table : 13190 plays, 863 seconds
share of runtime: 48%
cut for low plays: ['One More Time', 'Strobe']

  Avicii                 longest: Levels
  Daft Punk              longest: One More Time
  David Guetta ft. Sia   longest: Titanium
  The Weeknd             longest: Save Your Tears
  deadmau5               longest: Strobe

playlist order is untouched: ['Blinding Lights', 'Titanium', 'Save Your Tears'] ...

You just measured a sorted copy costing eight million bytes while the original sat right beside it. But where does that number come from — what does a list, an int, a whole object actually weigh, and how do you make it lighter? Next chapter opens the hood: weighing anything with getsizeof, slimming objects with __slots__, and caches that trade memory for speed. Memory, mastered →