Python for data
The Python you need for data work, explained through what each structure costs in memory and time.
Lessons
- 01Lists, dicts and what they costA list is an array of pointers. A dict is a hash table. Measure both, and you will know which one to reach for before you write the loop.○ not started
- 02Generators: stream, don't loadA list holds every row at once. A generator holds one. Learn to stream data through a pipeline so memory stays flat however big the file gets.○ not started
- 03Hashing for idempotencyPipelines retry, and retries load data twice. A stable content hash turns "run it again" from a risk into a no-op.○ not started
- 04DataFrames are columnsA DataFrame is a dict of typed arrays, one per column. Build a tiny one from scratch and the speed of pandas, Polars and DuckDB stops being a mystery.○ not started
- 05Files are bytes, text is an encodingPlanned
- 06CSV parsing without surprisesPlanned
- 07Sorting, heaps and top-kPlanned
- 08Testing data code with small fixturesPlanned
- 09From your loader to pandas and DuckDBPlanned