AI Ethics & Bias

Welcome to Day 6, post 4 — the code post. We've defined bias, argued its stakes, and mapped where it enters. Now we make it concrete and measurable: 'is my model fair?' is not a vibe or an opinion, it's a number you can compute, and this post shows exactly how.

We'll use loan-approval data and walk through measuring approval rates per group, applying the legal 80% rule, proving the proxy problem from post 3 is real, and applying a simple mitigation. The aim is to leave you able to actually check a model for bias rather than assuming it's fine — because, as the final slide warns, you can't fix what you don't measure.

The framing principle is that fairness cannot be eyeballed from an accuracy score — it's a distinct property you compute by comparing outcomes across groups. A model can post a great accuracy number and still treat two groups very differently, and the only way to know is to measure per-group outcomes explicitly.

This reframes fairness from a matter of intention to a matter of measurement. 'We didn't mean to be biased' is irrelevant if the numbers show disparate outcomes. The whole post operationalises this: instead of debating whether a model is fair in the abstract, we compute concrete quantities — approval rates, ratios, proxy-recovery scores — that turn the question into something you can settle with data. That shift, from assuming to measuring, is the single most important habit this post teaches.

This first snippet computes the most basic fairness signal: the approval rate for each group. With a dataframe of loan decisions, a single groupby on gender and a mean of the approved column gives the share approved in each group — here, 58% for women versus 71% for men.

That gap is the raw material of a fairness analysis. Notice how little code it takes: one line surfaces a disparity that an overall accuracy score would completely hide. This is the per-group slicing that posts 2 and 3 kept insisting on, made concrete. The lesson is that the first step of any fairness check is almost embarrassingly simple — just look at the outcomes group by group — yet it's the step most often skipped, which is exactly why biased models ship.

This snippet turns the raw gap into a standard, recognised metric: the disparate-impact ratio, computed as the disadvantaged group's approval rate divided by the advantaged group's — here 0.58 / 0.71 ≈ 0.82. The comment references the '80% rule' (the four-fifths rule), a long-standing guideline in which a ratio below 0.80 is treated as a red flag for adverse impact.

What makes this valuable is that it's not an arbitrary threshold someone invented — it's a recognised benchmark used in employment and lending contexts, which gives your analysis an objective reference point. An 0.82 ratio sits just above the line, meaning this model is borderline and would warrant scrutiny. Having a named, externally-grounded metric moves the fairness conversation from subjective debate to 'here's the number, and here's the recognised threshold it's measured against'.

This is the post's centrepiece: a proof that the 'just delete the sensitive column' fix fails. We drop gender (and the target) from the features, then train a logistic regression to predict gender from the supposedly 'neutral' remaining columns. It scores 0.89 — meaning the model can recover gender from 'innocent' features nearly 9 times out of 10.

This is the proxy problem from post 3, demonstrated rather than asserted. If a simple model can reconstruct gender from the other features, then your actual loan model can too — and will use that reconstructed signal to discriminate even though you 'removed' gender. The code makes the abstract warning undeniable: deleting the column doesn't remove the information, it just hides it from you. Run this proxy check on your own data and you'll typically be unpleasantly surprised how recoverable protected attributes are.

This slide interprets the two numbers so they don't stay abstract. The 0.82 ratio means women are approved roughly 18% less often than men — close enough to the legal red line to demand attention. The 0.89 proxy score means you cannot fix that by hiding gender, because the 'neutral' features still encode it nearly perfectly.

Put together, they tell a complete and sobering story: the model is meaningfully biased, and the intuitive fix would only blind you to it. 'Deleting the column blinds you, it doesn't debias you' is the line to remember. This is why the measurement mindset matters so much — without these two numbers you'd either not know there's a problem, or you'd 'solve' it in a way that makes it invisible and untrackable. Interpretation turns raw outputs into a decision: this model needs real mitigation, not column-deletion.

This snippet shows one concrete mitigation: group-aware decision thresholds. Instead of applying the same probability cutoff to everyone, we lower the bar slightly for the disadvantaged group and raise it for the advantaged one, which equalises approval rates. It's a few lines, and it directly addresses the disparity the earlier code measured.

The honest framing in the comment — 'one of several strategies' — matters. Group-aware thresholds are simple and effective for equalising approval rates, but they're not the only approach (reweighting the data, adding fairness constraints during training, and post-processing methods all exist), and they involve their own trade-offs and even legal nuances about treating groups differently. The takeaway isn't 'always use thresholds'; it's that mitigation is possible and concrete once you've measured the problem. You have levers — but which one to pull is a deliberate, context-dependent choice.

These practical tips frame the code within real-world rigour. Pick a fairness metric that fits the problem first — because, per post 1, the definitions conflict, so you must choose before you measure. Always slice metrics by group, never settle for the overall number. Keep the sensitive attribute in your data — to audit fairness, not to make decisions on (the 'awareness over blindness' lesson made operational).

And use a dedicated library like Fairlearn or AIF360 rather than hand-rolling everything, because they implement vetted metrics and mitigation methods and save you from subtle errors. The meta-message is that the toy code here teaches the concepts, but production fairness work uses mature tooling and starts from an explicit definition. These tips are the bridge from 'I understand the idea' to 'I can do this responsibly on a real system'.

This bar chart visualises the whole arc of the post: female approval at 58% (biased), male at 71% (the baseline), and female-after-mitigation at 69% (balanced). Seeing the three bars together makes the intervention's effect obvious — the mitigation closed most of the gap.

The numbers are illustrative, but the shape tells the true story: measure the disparity, apply a fix, measure again to confirm it worked. That last step — re-measuring after mitigation — is crucial and often forgotten; you don't get to assume your fix worked, you verify it the same way you found the problem. The chart embodies the entire fairness workflow in one image: detect, mitigate, verify. It's also a reminder that fairness work produces evidence you can show stakeholders, not just a vague claim that you 'handled it'.

The closing mistake ties back to the post's thesis: optimising accuracy alone hides bias. A model can be 95% accurate overall and still fail one group badly, especially when that group is a small fraction of the data — their errors barely dent the aggregate number, so the bias is statistically invisible in the headline metric.

This is precisely why per-group slicing isn't optional. If you only ever look at overall accuracy, the bias stays hidden until a real person gets hurt and the failure surfaces publicly. The whole point of computing approval rates, disparate-impact ratios, and proxy scores is to make visible what aggregate accuracy conceals. 'High accuracy' and 'fair' are different properties, and a responsible practitioner measures both — because the gap between them is exactly where people get harmed.

So fairness is computable: approval rates per group, the disparate-impact ratio against the 80% rule, a proxy check that proves deleting the column doesn't work, and a mitigation you verify by re-measuring. Save the snippets — they're a genuine starting toolkit for auditing any high-stakes model.

Tomorrow closes Day 6 by gathering the five most common misconceptions about AI fairness — including the proxy myth we just disproved — into a single checklist. After that, Day 7 shifts from how AI fails to how it learns, building the intuition behind model training without the heavy math.