TensorFlow in 8 Slides

This closing post is the debugging companion to the working example, and its theme is that the worst TF problems do not throw errors. The model runs, the loss prints, and yet nothing useful is learned. These are the failures that waste the most time precisely because they look like success.

Framing it as a checklist gives the reader something to run mentally whenever a model misbehaves, turning vague frustration into a short list of concrete things to verify.

Mistake one, skipping normalization, is first because it is the most common and the most quietly destructive. Raw inputs on very different scales make gradients either explode or barely move, so the loss plateaus and the model never converges. There is no error message — just disappointment.

The fix is one or two lines, which is why it is so worth knowing. Scaling pixels to 0-1, or standardizing tabular features to zero mean and unit variance, resolves a large share of 'my loss won't drop' reports outright.

This fix slide shows both the image case and the tabular case so the lesson generalizes. Dividing pixels by 255 handles images; for tabular data the Normalization layer with adapt computes the mean and variance from training data and applies them inside the model.

Putting normalization in a layer is the more robust pattern because the scaling travels with the saved model. That means inference applies the exact same transform automatically, eliminating a whole class of train-serve skew bugs.

Mistake two is choosing a loss that does not match your label format, and it bites because the symptoms range from a hard shape error to silently wrong training. Integer labels pair with sparse_categorical_crossentropy; one-hot vectors pair with categorical_crossentropy.

The slide also flags the activation pairing: a multiclass head wants softmax with crossentropy, not a sigmoid. Getting the loss-label-activation triad right is foundational, and mismatches here are among the most common beginner errors in any framework, not just TF.

The comparison makes the loss choice mechanical rather than memorized. On the left, integer labels of shape (batch,) call for sparse_categorical_crossentropy with no one-hot encoding needed. On the right, one-hot labels of shape (batch, classes) call for categorical_crossentropy, which means you must one-hot encode first.

Laying out the shapes alongside the loss names is deliberate, because the fastest way to diagnose this mistake is to print your label shape and read off which column you are in.

Mistake three is subtler and specific to TF: re-tracing @tf.function. Because tracing keys on input shape and dtype, and Python scalar arguments trigger a fresh trace per value, a carelessly written decorated function can rebuild its graph on every step. The intended speedup evaporates and the console fills with retrace warnings.

This is the practical consequence of the @tf.function mechanics from the third post. Understanding why tracing happens is what lets you recognize and fix the symptom instead of ignoring the warnings.

The fix demonstrates the cure: pass tensors rather than Python scalars, and pin the expected shape with an input_signature. The signature tells TF exactly what to expect, so it traces once and reuses that graph even as batch sizes vary, using None for the dynamic dimension.

The broader habit is to keep the inputs to decorated functions stable in type and shape. When a @tf.function feels slow or noisy, the retrace warning is the first thing to look for, and an input_signature is usually the answer.

Mistake four, data leakage, is the one that produces the most dangerous outcome: a great-looking score that collapses in production. It happens when you fit a scaler, imputer, or augmentation on the full dataset before splitting, letting test statistics bleed into training.

The discipline is strict ordering: split first, then fit any preprocessing on the training set only, then apply the learned transform to the test set. This is why the Normalization layer's adapt should see training data only — it is the same lesson wearing different clothes.

The flow diagram enforces the correct order visually: raw data stays untouched, you split into train and test, you fit preprocessing such as scaler.adapt on the training portion, then you apply that fitted transform to the test set without ever fitting on it.

Seeing it as a one-directional pipeline makes the rule intuitive. Information is only ever allowed to flow from train to test, never the reverse, and any step that violates that arrow is a leak that inflates your numbers.

Mistake five is judging a model by its training accuracy, which is how overfitting hides in plain sight. A model can memorize the training set and report near-perfect accuracy while failing completely on new data. The only honest signal is a validation or test metric.

The diagnostic is the gap: when training accuracy keeps climbing but validation accuracy stalls or falls, the model is overfitting. The standard responses are dropout, regularization, early stopping, or more data — and watching that gap is what tells you which to reach for.

The pre-flight checklist condenses all five mistakes into a list you can run before trusting any result: inputs scaled, loss matching the label format, @tf.function not retracing, preprocessing fit on training data only, and judgment based on validation rather than training metrics.

These five checks catch the overwhelming majority of silent failures. Running them habitually is the difference between debugging by guesswork and debugging by routine, which is what makes an experienced practitioner fast.

The cover and CTA close out Day 54. Having built a working model and learned the traps that quietly break it, the reader leaves with both a template and a checklist. The next day opens a fresh topic in the 100 Days of AI series, continuing the same concept-to-mistakes arc.