Pandas in 8 Slides

This cover signals the shift from explanation to execution. The earlier posts built the model; this one is a copyable recipe that takes a real dataset from raw CSV all the way to a finished, exported answer using only Pandas.

The goal is muscle memory for the loop you'll repeat on nearly every dataset: load, inspect, clean, filter, derive, aggregate, join, and export. By the end you should be able to start a fresh analysis and move confidently through these stages without reaching for documentation at every step.

The first step in any analysis is to load the data and look at it before touching anything. read_csv pulls the file into a DataFrame. Then four inspection calls orient you: head() shows the first rows so you see the shape of the data, shape reports the row and column counts, info() lists each column's dtype and non-null count, and describe() summarizes the numeric columns.

This inspection habit is not optional. Cleaning decisions depend on what you find here — which columns have missing values, whether numbers were read as strings, how many rows you're dealing with. Skipping inspection is how people clean the wrong things or miss problems entirely.

Cleaning is where most of the real work happens, and this slide shows the four most common moves. dropna(subset=[...]) removes rows missing a critical field — here, orders with no customer id, which would be useless. fillna(0).astype(int) replaces missing quantities with zero and forces the column to a proper integer type. to_datetime parses a text date column into real timestamps so you can do date math. drop_duplicates removes exact repeated rows.

The order matters and the choices are judgment calls. Whether to drop or fill a missing value, and what to fill it with, depends on the data's meaning. The point is that Pandas gives you a precise, declarative tool for each decision rather than ad-hoc patching.

With the data clean, this slide filters and derives. A boolean mask, df["date"].dt.year == 2026, selects only this year's orders, applied through .loc to assign the filtered frame back. Then a vectorized expression, qty * price, creates a revenue column in one operation across all rows. Finally, dt.to_period("M") derives a month key suitable for grouping.

Note the two distinct patterns here: boolean masking to choose rows, and vectorized column arithmetic to compute new values. These two operations, combined, cover an enormous fraction of everyday data transformation, and both run at NumPy speed rather than in Python loops.

This slide performs the central analytical step: grouping and aggregating. groupby("month") splits the rows into one bucket per month. The .agg call then computes three summaries at once — a count of order ids, a sum of revenue, and a mean of price — using the named-aggregation syntax that gives each output column a clear name. reset_index turns the month grouping back into a regular column.

The named-aggregation form is worth adopting as a default. It computes multiple metrics in a single pass and produces tidy, explicitly named columns, which is far cleaner than computing each aggregate separately and stitching the results together afterward.

This slide explains the concept underneath groupby: the split-apply-combine pattern. Pandas first splits the rows into groups keyed by the grouping column, then applies an aggregation function to each group independently, then combines the per-group results into a single new frame.

Understanding this three-phase model demystifies groupby's behavior and helps you reason about more complex cases. The .agg in the previous slide is the 'apply' phase computing three functions per group; the resulting one-row-per-month frame is the 'combine' phase. Once you see any groupby as split-apply-combine, custom aggregations and transformations stop feeling mysterious.

This pipeline diagram visualizes split-apply-combine as three stages. Split turns the rows into groups by key. Apply runs an aggregation on each group. Combine stacks the per-group answers into one result frame.

Seeing it as a pipeline reinforces that groupby is not a single opaque operation but a sequence you can reason about and intervene in. It's the same mental model that scales up to more advanced patterns like transform (which broadcasts results back to original row shape) and filter (which keeps or drops whole groups).

Real analyses rarely live in one table, so this slide joins a second. customers.csv carries the region for each customer id. merge combines it onto the orders frame, matching on customer_id, with how='left' so every order is preserved even if a matching customer row is missing. With region now attached, a final groupby computes revenue per region.

merge is Pandas' equivalent of a SQL join, and it's how you enrich a primary table with attributes from another. The on parameter names the key column, and the how parameter decides which rows survive — the next slide unpacks that crucial choice.

This slide explains how to choose the join type, which is one of the most consequential decisions in a merge. how='left' keeps every row from the left frame and fills unmatched columns from the right with NaN — the safe default when you're enriching a main table and can't afford to drop any of its rows. 'inner' keeps only rows that match on both sides. 'outer' keeps everything from both, filling gaps with NaN. 'right' is the mirror of left.

The guiding question is: which rows can you not afford to lose? Choosing left when you meant inner can silently introduce NaNs; choosing inner when you meant left can silently drop records. Being deliberate here prevents a whole category of quiet data-loss bugs.

The final step is to persist the result. to_csv writes the monthly summary back to a file, with index=False so the default integer index isn't written as a spurious column. The slide also shows to_excel and to_parquet, demonstrating that the same DataFrame can be exported to whatever format the next consumer needs.

The index=False detail is a small but common gotcha: forget it and your CSV gains an unnamed leading column of row numbers that downstream tools then misinterpret. Choosing the right output format matters too — Parquet preserves dtypes and compresses well for data pipelines, while CSV and Excel are better for human or cross-tool sharing.

These bullets distill the entire walkthrough into the loop you'll actually repeat: read_csv then head/info to inspect, dropna/fillna/astype to clean, loc with boolean masks to filter, groupby().agg() to summarize, and merge to join before to_csv to export.

If you keep this five-step rhythm in mind, you have a reliable scaffold for almost any analysis. The specifics change per dataset, but the sequence — inspect, clean, filter, aggregate, join, export — is remarkably stable across the data work you'll do day to day.

This closes the hands-on post and previews the final angle. You can now drive a dataset end to end in Pandas. The last post catalogs the common mistakes — the traps that don't crash but quietly corrupt your results — so you can recognize and avoid them before they cost you a wrong answer.