What is NLP?

This final post is a pre-mortem, and the cover states the thesis bluntly: NLP projects rarely fail on the model. They fail on data, evaluation, and unexamined assumptions. After four posts showing how accessible NLP is, this one supplies the maturity that keeps that accessibility from becoming overconfidence.

The five mistakes are ordered roughly by where they strike in a project's life — data prep, evaluation, splitting, trusting output, and deployment domain — so the post doubles as a checklist you can walk top to bottom.

The first mistake is treating raw text as clean input. Real-world text is full of HTML tags, emoji, inconsistent casing, runaway whitespace, and encoding artifacts. For classic ML models this noise directly corrupts features; for LLMs, messy context produces messy answers.

The broader principle is that preprocessing is part of the model, not a preamble to it. The cleanliness of your input sets a ceiling on the quality of your output, and skipping this step is the most common silent quality killer in beginner projects.

The cleaning code is intentionally minimal but real — it strips HTML, collapses whitespace, and lowercases — to show that basic hygiene is a few lines, not a research project. The example transforms '<p>Hello WORLD</p>' into 'hello world', demonstrating each operation at once.

The nuance worth flagging is that the 'right' cleaning depends on the task. Lowercasing helps a bag-of-words classifier but can hurt NER, where capitalization signals names. The slide teaches the habit; the reader must tune the specifics to their model and goal.

The second mistake is trusting accuracy as a metric, and the spam example is the canonical illustration. If 95% of email is legitimate, a model that blindly predicts 'not spam' scores 95% accuracy while catching exactly zero spam — useless despite the impressive number.

This is why imbalanced text data, which describes most real datasets, demands precision, recall, and per-class F1 instead of raw accuracy. The lesson is to always ask what the majority-class baseline would score; if your model barely beats 'always guess the common class', accuracy is hiding the truth.

The comparison diagram dramatizes the gap between what accuracy reports and what the model actually does. On the left, accuracy says 95% correct, looks great, ship it. On the right, F1 reveals the model caught zero spam, recall is zero, and the system is worthless.

The contrast is stark on purpose. It's the single clearest demonstration that a metric can be technically true and practically misleading, and it cements the habit of looking past a headline accuracy number to the metrics that reflect the task you actually care about.

The third mistake is data leakage, which inflates your scores and then humiliates you in production. With text, leakage hides in subtle places: near-duplicate reviews split across train and test, overlapping time windows, or the same user's posts on both sides of the split.

The model memorizes the leaked examples and reports a test score it can't reproduce on truly unseen data. The fix is disciplined splitting — deduplicate first, group by entity where relevant, and be suspicious of any result that looks too good. Leakage is the most common cause of 'great in the notebook, terrible in production'.

The dedup-before-split code shows the minimal defense: drop duplicate texts, then split with stratification so class balance is preserved across train and test. Removing duplicates before splitting is what prevents the same example from appearing on both sides.

The stratify argument is a quiet bonus lesson — on imbalanced data, a random split can leave too few minority examples in the test set, making evaluation noisy. Stratified splitting keeps the class ratio stable, which ties back to the accuracy-versus-F1 discussion two slides earlier.

The fourth mistake targets the defining hazard of the LLM era: trusting fluent output. A model's confidence and its prose quality have nothing to do with its correctness. It will fabricate citations, invent API methods, and state false facts in flawless, authoritative language.

This is the failure mode that connects all the way back to the first post's warning that models map patterns, not truth. Fluency is a property of the language, not the facts. For anything that matters, the only safe posture is to ground answers in verifiable sources and check them before acting.

The decision tree operationalizes that warning into a usable rule. If the decision isn't high-stakes, fluent output is fine to use as-is. If it is high-stakes, ask whether the answer is grounded in real sources: if yes, verify and then use it; if no, do not trust it.

This simple branching captures the entire risk-management posture for generative NLP. It's deliberately not 'never trust LLMs' — that's impractical — but rather a graduated rule that matches scrutiny to stakes, which is how the technology can be used responsibly rather than fearfully.

The fifth mistake is domain and language mismatch. A model trained on English news will degrade on Spanish tweets, legal contracts, or clinical notes, because vocabulary, tone, structure, and even the meaning of words shift across domains. The model isn't broken; it's simply out of distribution.

The fix is to check that your model's training domain resembles your data, and to fine-tune on in-domain text when it doesn't. This is the deployment-time mistake — everything can work in testing on familiar data and then fail the moment real, off-distribution inputs arrive.

The pre-flight checklist gathers all five mistakes into a single scannable list: clean text, use F1 not just accuracy, dedup then split, ground and verify LLM output, and match model domain to your data. It's designed to be screenshotted and pinned next to a project.

Each item maps directly to one of the five mistakes, so the slide works as both a summary and a recurring reference. The intent is that a reader runs through these five checks before shipping any NLP system, the way a pilot runs a pre-flight list.

The closing card wraps the five-post arc and points forward to the next day. Having covered what NLP is, why it matters, how it works, how to run it, and how it fails, the reader now has a complete, grounded foundation in the field.

The teaser keeps the momentum by promising to go deeper into the NLP and LLM stack, signaling that this introduction is a launchpad into the more advanced topics the series tackles next.