Comprehensions and generators

List, dict, and set comprehensions

[expr for item in seq] builds a list. Add a filter: [x for x in scores if x > 0.7].

Dict version: {k: v for k, v in pairs}. Set version uses braces with a single expression.

Rule of thumb: one loop and one condition is fine; anything more nested belongs in a normal for-loop. Readability wins — a reviewer should understand it at a glance.

Generators: lazy sequences

Replace the brackets with parentheses and you get a generator expression — values are produced on demand, not stored.

A generator function uses yield instead of return. Each yield hands one value to the caller and pauses; the next iteration resumes right there.

Why it matters: streaming a 50 GB JSONL of training records with for record in read_records(path) uses constant memory. The same pattern powers token streaming from an LLM, where each chunk arrives one at a time.

Generators are single-use. Once consumed, iterate again by calling the function again.

Copy-paste examples

Copy into your own editor, or load one into the compiler below and press Run.

docs = ["  Python  ", "AI", "  Machine Learning ", "ML"]
cleaned = [d.strip().lower() for d in docs if len(d.strip()) > 2]
print(cleaned)

lengths = {d.strip(): len(d.strip()) for d in docs}
print(lengths)

def batched(items, size):
    batch = []
    for item in items:
        batch.append(item)
        if len(batch) == size:
            yield batch
            batch = []
    if batch:
        yield batch

rows = list(range(10))
for i, batch in enumerate(batched(rows, 4), start=1):
    print("batch", i, batch)

def stream_tokens(text):
    for word in text.split():
        yield word

collected = ""
for token in stream_tokens("Retrieval augmented generation grounds answers"):
    collected += token + " "
    print("received:", token)
print("final:", collected.strip())

Takeaways

• Comprehensions replace short build-a-list loops; keep them to one loop and one filter.
• Generators use yield and stream data with constant memory.
• Generators are single-use — call the function again to re-iterate.