CS / data / stats students
Students preparing for ML internships and new-grad roles who want the practical path, not only theory.
Software engineers
Backend and full-stack developers moving into ML engineering, who already write good code and want the ML on top.
Data analysts
Analysts who know SQL and spreadsheets and want to build and ship models, not just report on data.
QA & test engineers
Test automation engineers heading into ML: evaluation, data checks and model testing use the same mindset.
Data engineers
Pipeline builders who want to own the models their data feeds, and the MLOps around them.
Career switchers
Engineers from other fields (EE, mechanical, finance) with some Python, aiming for an ML engineer job.
Prerequisites
| You need | Level | Where to get it |
|---|---|---|
| pandas DataFrames, groupby, joins | Comfortable | Area 01 |
| Mean, median, std, correlation | Basic | Area 02, Module 4 |
| scikit-learn | None | Introduced here |
Where this area sits in the ML lifecycle — and the roles that use it
| Role | How you use this area on the job | Typical first title |
|---|---|---|
| ML engineer | Build preprocessing pipelines that run identically in training and serving | ML Engineer I / Machine Learning Engineer |
| Data scientist | Find signal and leaks during EDA; engineer features | Data Scientist / Applied Scientist |
| AI / LLM engineer | Clean and split text data and eval sets without contamination | AI Engineer / LLM Engineer |
| MLOps / platform | Version preprocessing with the model; monitor input quality | MLOps Engineer / ML Platform Engineer |
| Data engineer | Deliver complete, typed, deduplicated tables with known timestamps | Data Engineer |
- Run a five-pass EDA and write a data dictionary.
- Spot ids, post-outcome columns and other leaks.
- Choose an imputation strategy per column and add missing indicators.
- Flag outliers with the IQR rule and decide to fix, clip or log-transform.
- Encode categories with one-hot or ordinal encoding and handle unseen values.
- Standardize or min-max scale and know which models need it.
- Engineer ratio, flag and log features from domain sense.
- Build a stratified, leak-free Pipeline with ColumnTransformer and cross-validate it.
| Tool | What it does | Used in this area for | Cost |
|---|---|---|---|
| Python 3.10 – 3.13 | The language of ML: every library on this page is a Python package | Every example and the lab | Free |
| venv + pip | An isolated environment per project and the package installer | Installing the area's packages | Free |
| pandas | Tables (DataFrames): load, clean, group and join data | EDA, cleaning, new features | Free |
| NumPy | Fast n-dimensional arrays and maths | log transforms, arrays | Free |
| scikit-learn | Classic ML: preprocessing, models, metrics, pipelines | Imputers, encoders, scalers, Pipeline, splits | Free |
| Matplotlib | Charts: histograms, scatter plots, learning curves | Histograms and box plots | Free |
| JupyterLab | Notebooks: run code cell by cell and see tables and plots inline | Interactive EDA | Free |
| Google Colab | Hosted notebooks in the browser, with a free GPU on request | No-install option | Free |
Platforms you will meet at work
| Platform | Vendor | What teams use it for |
|---|---|---|
| SageMaker | AWS | Managed notebooks, training jobs, model registry and endpoints |
| Vertex AI | Google Cloud | Training, pipelines, feature store, model serving and Gemini APIs |
| Azure Machine Learning | Microsoft | Workspaces, MLflow tracking, managed endpoints |
| Databricks | Databricks | Spark + notebooks + MLflow on a lakehouse |
| Weights & Biases | W&B | Experiment tracking and model evaluation dashboards |
The same Pipeline object you build here is what gets saved and served in Area 12.
Recommended: WSL 2 (Ubuntu inside Windows) — the same commands as Linux, cloud servers and most ML teams.
- Turn on WSL and install UbuntuOpen PowerShell as Administrator, run the command, then restart.
wsl --install
After the restart an Ubuntu window opens: choose a user name and password. - Install Python's venv support
sudo apt update sudo apt install -y python3 python3-venv python3-pip git make
- Create an environment and install the packages
python3 -m venv ~/ml source ~/ml/bin/activate pip install --upgrade pip pip install pandas numpy scikit-learn matplotlib jupyterlab
Runsource ~/ml/bin/activatein every new terminal. - EditorInstall VS Code, the WSL, Python and Jupyter extensions, then type
code .in Ubuntu. - Check it works
python -c "import sklearn, pandas; print(sklearn.__version__, pandas.__version__)"
Install Python 3.12 from python.org (tick Add python.exe to PATH), then in PowerShell: py -m venv %USERPROFILE%\ml, %USERPROFILE%\ml\Scripts\activate and the same pip install line. Every lab runs the same.
- Install Apple's command-line tools
xcode-select --install
- Install Homebrew and PythonFollow the one-line command on brew.sh, then:
brew install python@3.12 git make
- Create an environment and install the packages
python3 -m venv ~/ml source ~/ml/bin/activate pip install --upgrade pip pip install pandas numpy scikit-learn matplotlib jupyterlab
- EditorInstall VS Code with the Python and Jupyter extensions. Apple-silicon Macs run every lab natively.
- Check it works
python -c "import sklearn, pandas; print(sklearn.__version__, pandas.__version__)"
- Ubuntu / Debian
sudo apt update sudo apt install -y python3 python3-venv python3-pip git make
- Fedora
sudo dnf install -y python3 python3-pip git make
- Arch
sudo pacman -S python python-pip git make
- Create an environment and install the packages
python3 -m venv ~/ml source ~/ml/bin/activate pip install --upgrade pip pip install pandas numpy scikit-learn matplotlib jupyterlab
- Check it works
python -c "import sklearn, pandas; print(sklearn.__version__, pandas.__version__)"
- Open Google ColabGo to colab.research.google.com and sign in with a Google account.
- New notebookFile → New notebook. NumPy, pandas, scikit-learn, Matplotlib and PyTorch are already installed.
- Upload the labClick the folder icon, upload the lab zip, then run
!unzip lab.zip && cd lab && python check.pyin a cell (prefix shell commands with!). - Check the versions
python -c "import sklearn, pandas; print(sklearn.__version__, pandas.__version__)"
- Get the dataDownload the lab kit and run
python make_data.py— it writescustomers.csv. - Create
first.pyimport pandas as pd from sklearn.model_selection import train_test_split df = pd.read_csv("customers.csv") print(df.shape, "| churn rate", round(df["churned"].mean(), 3)) print(df.isna().sum()[lambda s: s > 0].to_dict()) train, test = train_test_split(df, test_size=0.2, stratify=df["churned"], random_state=42) print("train", len(train), "test", len(test))
- Run it
python first.py
Output (captured from a real run):(2000, 9) | churn rate 0.299 {'monthly_fee': 169, 'age': 185, 'region': 95} train 1600 test 400 - What to noticeThree columns have gaps, and churn is about 30%. Stratifying keeps that 30% in both train and test — the starting point of every honest evaluation.
| Session | Do this | Time |
|---|---|---|
| Day 1 | Install · first run · Module 1 (EDA) | 3 h |
| Day 2 | Module 2 (missing values, outliers) + outlier widget | 3.5 h |
| Day 3 | Module 3 (encoding, scaling, features) + encoder widget | 3.5 h |
| Day 4 | Module 4 (splits, leakage, Pipeline) + split widget | 2.5 h |
| Day 5 | Lab: leak-free pipeline · area test | 1.5 h |
Start from zero
New to this? Work through these steps first, in order. Already comfortable? Skim them and jump to the reference below.
Look before you cook
Before training anything, look at the table: how many rows, what each column holds, what is missing.
A chef checks the fridge before planning the menu.
| question | answer for customers.csv |
|---|---|
| rows × columns | 2,000 × 9 |
| missing | fee, age, region |
| target | churned, ~30% yes |
Most model bugs are data bugs you could have seen in the first ten minutes.
Would I know this at prediction time?
A column is only a fair input if its value exists at the moment you make the prediction.
Predicting a football result using the final score is not a prediction.
| column | known before churn? |
|---|---|
| tenure_months | yes |
| support_calls | yes |
| refund_issued | no — set after leaving |
Leaky columns give brilliant test scores and useless production models.
A loan model has a column “days_late_on_this_loan”. Use it to predict default at approval time?
No — it only exists after the loan is running. Leakage.
Going further: the reference
The same ideas at the next level: precise definitions, tables, rules and interview questions.
Key definitions
- EDA
- Exploratory data analysis: looking at the data before modelling it — shape, types, gaps, spread, target.
- Data quality
- Whether values are complete, valid, consistent and unique. Bad quality caps model quality.
- Dtype
- The stored type of a column. Numbers stored as text ("45.0") silently break maths.
- Cardinality
- How many distinct values a column has. An id has one per row and is useless as a feature.
- Class balance
- How often each target value occurs. 30% churn is mildly imbalanced; 1% fraud is severe.
- Skew
- A long tail on one side. Tenure, income and prices are usually right-skewed.
- Duplicates
- Repeated rows. They inflate counts and can leak between train and test.
- Data dictionary
- A table saying what each column means, its unit and when it becomes known.
import pandas as pd df = pd.read_csv("customers.csv") print(df.shape) print(df.dtypes.to_string()) print("\nmissing share:") print(df.isna().mean().round(3)[lambda s: s > 0].to_string())
# output
(2000, 9)
customer_id int64
tenure_months float64
monthly_fee float64
support_calls int64
age float64
plan str
region str
refund_issued int64
churned int64
missing share:
monthly_fee 0.084
age 0.092
region 0.048import pandas as pd df = pd.read_csv("customers.csv") print(df[["tenure_months", "monthly_fee", "support_calls", "age"]].describe().round(1).to_string()) print("\nchurn rate:", round(df["churned"].mean(), 3)) print("distinct customer_id:", df["customer_id"].nunique(), "of", len(df))
# output
tenure_months monthly_fee support_calls age
count 2000.0 1831.0 2000.0 1815.0
mean 17.4 44.9 1.6 41.0
std 12.8 15.1 1.3 11.7
min 0.4 9.0 0.0 18.0
25% 8.1 35.0 1.0 32.7
50% 14.4 45.2 1.0 40.8
75% 23.1 55.3 2.0 48.9
max 89.4 105.9 7.0 77.1
churn rate: 0.299
distinct customer_id: 2000 of 2000Look for signal: what differs between churners and stayers?
import pandas as pd df = pd.read_csv("customers.csv") print(df.groupby("churned")[["tenure_months", "support_calls", "monthly_fee"]].mean().round(2).to_string()) print() print(df.groupby("plan")["churned"].mean().round(3).to_string()) print() print(df.groupby("refund_issued")["churned"].mean().round(3).to_string())
# output
tenure_months support_calls monthly_fee
churned
0 19.15 1.40 44.26
1 13.24 2.22 46.58
plan
basic 0.358
plus 0.237
pro 0.259
refund_issued
0 0.095
1 1.000Every customer with refund_issued = 1 churned — because refunds are issued after cancelling. At prediction time the column is always 0. Ask of every column: would I know this value at the moment I make the prediction?
| Check | pandas | Red flag |
|---|---|---|
| shape & types | df.info() | numbers stored as object |
| missing | df.isna().mean() | > 30% missing, or missing only for one class |
| distribution | df.describe() | min/max impossible (age −3, fee 99,999) |
| duplicates | df.duplicated().sum() | repeated rows or ids |
| cardinality | df.nunique() | one value per row (ids) or one value total |
| target | y.value_counts(normalize=True) | rare positive class |
Skipping EDA and going straight to fit · keeping ids as features · trusting dtypes without checking · not asking when each column becomes known · looking at the test set during EDA (it should stay unseen).
Interview corner
Q. What do you do first with a new dataset? A. Shape and types, missing values, distributions, target balance, then relationships — and a data dictionary with when each field is known.
Q. A feature has 0.99 correlation with the target. Good news? A. Suspicious: check whether it is recorded after the outcome (leakage) or is a copy of the target.
Key takeaways
- EDA is five passes: shape, missing, spread, target, signal.
- Ids and post-outcome columns are not features.
- Keep the test set out of EDA.
Start from zero
New to this? Work through these steps first, in order. Already comfortable? Skim them and jump to the reference below.
Fill the gaps sensibly
Missing values must be filled or the model can't run. Use a typical value: the median for numbers, the most common value for categories.
A missing jigsaw piece: you cut a plain piece of roughly the right colour so the picture holds together.
import pandas as pd fee = pd.Series([40, 45, None, 50, 400]) print(fee.fillna(fee.median()).tolist())
# output
[40.0, 45.0, 47.5, 50.0, 400.0]The median ignores the 400 outlier that would drag a mean fill upward.
A gap can be a clue
Sometimes the fact that a value is missing tells you something. Keep a 0/1 flag for it.
A blank “current employer” field on a form says something on its own.
| fee_missing | churn rate |
|---|---|
| 0 | 29.6% |
| 1 | 33.1% — a small hint |
When gaps are not random (MNAR), the indicator lets the model use them.
Outliers: mistake or treasure?
A value far from the others is either an error to fix or a real extreme to keep but tame.
A 240-year-old customer is a typo; a customer spending 100× the average is your best client.
- Q1 = 15.5, Q3 = 21.5 → IQR = 6
- upper fence = 21.5 + 1.5 × 6 = 30.5
- 61 > 30.5 → flag it, then decide
Extreme values distort means, scales and linear models.
Going further: the reference
The same ideas at the next level: precise definitions, tables, rules and interview questions.
Key definitions
- Missing value
- An empty cell: NaN in pandas/NumPy, None in Python, NULL in SQL.
- Imputation
- Filling missing values with an estimate such as the median or most frequent value.
- Median imputation
- Fill with the column median; robust to outliers and skew.
- Missing indicator
- A 0/1 column saying a value was missing, so the model can learn from the gap itself.
- Outlier
- A value far from the rest. Could be an error (age 240) or real and important (a whale customer).
- IQR rule
- Flag values below Q1 − 1.5·IQR or above Q3 + 1.5·IQR, where IQR = Q3 − Q1.
- Clipping (winsorizing)
- Capping values at chosen percentiles, e.g. the 1st and 99th.
- Log transform
- log1p(x) compresses a long right tail so extreme values pull less.
import pandas as pd df = pd.read_csv("customers.csv") fee = df["monthly_fee"] print("missing:", fee.isna().sum()) print("mean =", round(fee.mean(), 2), " median =", round(fee.median(), 2)) filled = fee.fillna(fee.median()) flag = fee.isna().astype(int) # keep the fact that it was missing print("after fill, missing:", filled.isna().sum(), " indicator sum:", flag.sum()) print("churn when fee missing:", round(df.loc[flag == 1, "churned"].mean(), 3), " when present:", round(df.loc[flag == 0, "churned"].mean(), 3))
# output
missing: 169
mean = 44.94 median = 45.19
after fill, missing: 0 indicator sum: 169
churn when fee missing: 0.331 when present: 0.296| Strategy | When | scikit-learn |
|---|---|---|
| drop rows | few rows missing, at random | df.dropna(subset=[...]) |
| median | numeric, skewed or with outliers | SimpleImputer(strategy="median") |
| most frequent | categorical | SimpleImputer(strategy="most_frequent") |
| constant | “unknown” is meaningful | SimpleImputer(strategy="constant", fill_value="unknown") |
| + indicator | the gap may carry signal | SimpleImputer(add_indicator=True) |
| model-based | strong relationships between columns | KNNImputer, IterativeImputer |
Outliers
Try it: drag an outlier
import numpy as np import pandas as pd df = pd.read_csv("customers.csv") t = df["tenure_months"] q1, q3 = t.quantile([0.25, 0.75]) upper = q3 + 1.5 * (q3 - q1) print(f"Q1={q1:.1f} Q3={q3:.1f} upper fence={upper:.1f} flagged={(t > upper).sum()}") print("skew raw :", round(t.skew(), 2)) print("skew log1p:", round(np.log1p(t).skew(), 2)) clipped = t.clip(upper=t.quantile(0.99)) print("max raw / clipped:", t.max(), "/", round(clipped.max(), 1))
# output
Q1=8.1 Q3=23.1 upper fence=45.6 flagged=80
skew raw : 1.5
skew log1p: -0.33
max raw / clipped: 89.4 / 60.2For each column with gaps or outliers, record the rule (drop / median / clip at p99 / log) and why. That note becomes the spec your pipeline code implements — and what you defend in a review.
Filling with the mean on skewed data · filling before splitting (the fill value then includes test rows) · deleting every outlier, including the real big customers · forgetting the missing indicator when the gap is informative.
Interview corner
Q. Mean or median imputation? A. Median for skewed or outlier-heavy numbers; the mean is pulled by extremes.
Q. How do you handle outliers? A. First decide error or real. Fix or drop errors; for real extremes, clip, log-transform or use robust models (trees).
Key takeaways
- Ask why it is missing; MNAR deserves an indicator.
- Median for numbers, most-frequent for categories.
- Flag outliers with IQR; clip or log, don't blindly delete.
Start from zero
New to this? Work through these steps first, in order. Already comfortable? Skim them and jump to the reference below.
Models only read numbers
Text categories like “basic/plus/pro” must become numbers. One clean way: a 0/1 column per category.
A tick-box form: tick exactly one box for your plan.
| plan | is_basic | is_plus | is_pro |
|---|---|---|---|
| plus | 0 | 1 | 0 |
| pro | 0 | 0 | 1 |
Numbering the plans 0, 1, 2 would invent an order the model takes literally.
Put features on the same scale
A fee in tens and a call count in ones are not comparable. Rescale each column so its typical spread is 1.
Comparing heights in centimetres with weights in tonnes — convert first.
- fee mean 45, std 15
- fee 60 → (60 − 45) / 15 = 1.0
- fee 30 → (30 − 45) / 15 = −1.0
Linear models, k-NN and neural nets learn faster and fairer on scaled inputs.
Tenure 6 months and 2 support calls. Make a feature that captures “calls per month”.
2 / (6 + 1) ≈ 0.29 calls per month (the +1 avoids dividing by zero).
Going further: the reference
The same ideas at the next level: precise definitions, tables, rules and interview questions.
Key definitions
- Feature engineering
- Creating inputs that make the pattern easier for the model to see.
- One-hot encoding
- One 0/1 column per category; no fake order between categories.
- Ordinal encoding
- Map ordered categories to integers (small=0, medium=1, large=2).
- Target encoding
- Replace a category with the mean target for it, computed on training folds only.
- Standardization
- z = (x − mean) / std per column: mean 0, std 1.
- Min-max scaling
- (x − min) / (max − min): squeezes into [0, 1]; sensitive to outliers.
- Ratio / interaction feature
- Combining columns, e.g. calls per month = support_calls / tenure.
- Binning
- Turning a number into ranges (0–6, 6–24, 24+ months).
Try it: encode a column
import pandas as pd from sklearn.preprocessing import OneHotEncoder df = pd.read_csv("customers.csv").dropna(subset=["region"]) enc = OneHotEncoder(handle_unknown="ignore", sparse_output=False) enc.fit(df[["plan", "region"]]) print(enc.get_feature_names_out()) new = pd.DataFrame({"plan": ["pro"], "region": ["central"]}) # "central" never seen print(enc.transform(new))
# output
['plan_basic' 'plan_plus' 'plan_pro' 'region_east' 'region_north'
'region_south' 'region_west']
[[0. 0. 1. 0. 0. 0. 0.]]import pandas as pd from sklearn.preprocessing import MinMaxScaler, StandardScaler df = pd.read_csv("customers.csv") X = df[["tenure_months", "support_calls"]] z = StandardScaler().fit_transform(X) m = MinMaxScaler().fit_transform(X) print("standardized mean:", z.mean(axis=0).round(3), " std:", z.std(axis=0).round(3)) print("min-max range :", m.min(axis=0), "→", m.max(axis=0))
# output
standardized mean: [0. 0.] std: [1. 1.]
min-max range : [0. 0.] → [1. 1.]New features from domain sense
import numpy as np import pandas as pd df = pd.read_csv("customers.csv") df["calls_per_month"] = df["support_calls"] / (df["tenure_months"] + 1) df["new_customer"] = (df["tenure_months"] < 6).astype(int) df["log_tenure"] = np.log1p(df["tenure_months"]) for col in ["support_calls", "calls_per_month", "new_customer"]: print(f"{col:16} corr with churn = {df[col].corr(df['churned']):+.3f}")
# output
support_calls corr with churn = +0.296
calls_per_month corr with churn = +0.294
new_customer corr with churn = +0.075| Model family | Scale? | Encode categories? |
|---|---|---|
| Linear / logistic regression | Yes | One-hot |
| k-NN, k-means, SVM | Yes (critical) | One-hot |
| Neural networks | Yes | One-hot or embeddings |
| Decision trees, random forests, gradient boosting | No | One-hot or ordinal (native categorical in some libraries) |
On tabular data a good ratio or flag often adds more than swapping algorithms. Domain questions — “what does a frustrated customer do?” — lead to features like calls per month.
Ordinal-encoding unordered categories · one-hot on a 10,000-value column (use hashing, target encoding or grouping) · scaling before splitting · computing target encoding on the full data (pure leakage) · forgetting handle_unknown="ignore".
Interview corner
Q. One-hot vs label encoding? A. One-hot for nominal categories (no order); integer codes only for truly ordered categories or tree models.
Q. Do random forests need scaling? A. No — splits compare values within one feature, so monotonic rescaling changes nothing.
Key takeaways
- One-hot for nominal, ordinal for ordered.
- Standardize for distance/gradient models; trees don't care.
- Good domain features often beat a fancier model.
Start from zero
New to this? Work through these steps first, in order. Already comfortable? Skim them and jump to the reference below.
Keep an exam the model never studied
Hide part of the data. Train on the rest. Score on the hidden part to see how the model does on new customers.
A teacher doesn't hand out the exam answers as homework.
| set | rows | used for |
|---|---|---|
| train | 1,600 | learning |
| test | 400 | one final score |
A score on training data only shows memory, not skill.
Don't peek while preparing
Anything that learns from data — a median, a mean for scaling — must learn from the training part only.
Calibrating the exam's difficulty using the students' exam answers.
- split first
- scaler.fit(train)
- scaler.transform(train), scaler.transform(test)
Peeking makes test scores optimistic; production then disappoints.
You remove duplicates, then fill missing ages with the median, then split. Which step is in the wrong place?
The median fill — split first, compute the median on training rows only. (Removing duplicates before splitting is fine and wise.)
Going further: the reference
The same ideas at the next level: precise definitions, tables, rules and interview questions.
Key definitions
- Training set
- Rows the model and every preprocessing step learn from.
- Validation set
- Rows used to compare models and tune hyperparameters.
- Test set
- Rows held back until the end, used once for an honest estimate.
- Stratified split
- Keeps the class mix (e.g. 30% churn) the same in every split.
- Cross-validation
- Rotate the held-out fold k times and average the scores.
- Data leakage
- Information unavailable at prediction time reaching the model during training.
- Target leakage
- A feature that is set because of, or after, the outcome (refund_issued).
- Train-test contamination
- Test rows influencing training, e.g. a scaler fitted on all rows, or duplicates across splits.
Try it: split and stratify
import pandas as pd from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler df = pd.read_csv("customers.csv") X, y = df[["tenure_months", "support_calls"]], df["churned"] X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42) print("churn rate train / test:", round(y_tr.mean(), 3), "/", round(y_te.mean(), 3)) scaler = StandardScaler().fit(X_tr) # learn mean/std from training rows only print("train means used:", scaler.mean_.round(2)) print("test mean after transform:", scaler.transform(X_te).mean(axis=0).round(3), "(near, not exactly, 0)")
# output
churn rate train / test: 0.299 / 0.3
train means used: [17.54 1.67]
test mean after transform: [-0.062 -0.101] (near, not exactly, 0)One object for the whole recipe: Pipeline + ColumnTransformer
import pandas as pd from sklearn.compose import ColumnTransformer from sklearn.impute import SimpleImputer from sklearn.linear_model import LogisticRegression from sklearn.model_selection import cross_val_score from sklearn.pipeline import Pipeline from sklearn.preprocessing import OneHotEncoder, StandardScaler df = pd.read_csv("customers.csv") num = ["tenure_months", "monthly_fee", "support_calls", "age"] cat = ["plan", "region"] pre = ColumnTransformer([ ("num", Pipeline([("imp", SimpleImputer(strategy="median")), ("sc", StandardScaler())]), num), ("cat", Pipeline([("imp", SimpleImputer(strategy="most_frequent")), ("oh", OneHotEncoder(handle_unknown="ignore"))]), cat), ]) model = Pipeline([("pre", pre), ("clf", LogisticRegression(max_iter=1000))]) scores = cross_val_score(model, df[num + cat], df["churned"], cv=5, scoring="roc_auc") print("5-fold AUC:", scores.round(3), " mean =", round(scores.mean(), 3))
# output
5-fold AUC: [0.784 0.753 0.728 0.727 0.715] mean = 0.742import pandas as pd from sklearn.linear_model import LogisticRegression from sklearn.model_selection import cross_val_score df = pd.read_csv("customers.csv").fillna({"monthly_fee": 45, "age": 41}) honest = ["tenure_months", "support_calls", "monthly_fee"] leaky = honest + ["refund_issued"] for name, cols in [("honest", honest), ("with refund_issued", leaky)]: auc = cross_val_score(LogisticRegression(max_iter=1000), df[cols], df["churned"], cv=5, scoring="roc_auc").mean() print(f"{name:20} AUC = {auc:.3f}")
# output
honest AUC = 0.730
with refund_issued AUC = 0.931If rows are events over time (transactions, sensor readings), a random split lets the model peek at the future. Train on the past and test on the most recent period (TimeSeriesSplit).
The lab: make the pipeline honest
Fitting any transformer before the split · random-splitting time series · tuning on the test set · duplicates of the same customer in train and test (use GroupShuffleSplit) · not stratifying a rare class.
Interview corner
Q. What is data leakage? Give two examples. A. Information unavailable at prediction time used in training: a post-outcome column like refund_issued; a scaler fitted on all rows before splitting.
Q. Why wrap preprocessing in a Pipeline? A. Cross-validation then refits preprocessing inside each fold, so no fold's held-out rows leak into fitting; deployment uses the identical recipe.
Key takeaways
- Split first; fit on train; transform everything.
- Stratify classification splits; time-split temporal data.
- Pipeline + ColumnTransformer makes leakage hard.
Which column in customers.csv should never be a feature, and why?
A skewed income column has 5% missing. Mean or median?
Q1 = 10, Q3 = 30. Is 65 an outlier by the IQR rule?
Plan has basic/plus/pro with no real order. Encoding for logistic regression?
Does a random forest need scaled inputs?
You fit StandardScaler, then split. What's wrong?
1% of transactions are fraud. Which split option matters?
Orders over 2 years; predict next month. How to split?
Walk me through your data cleaning process.
How do you handle missing data?
What is target leakage? How do you detect it?
Standardization vs normalization?
High-cardinality categorical feature — what do you do?
Why use cross-validation?
How do you prevent leakage in cross-validation?
Class imbalance in the split — what do you do?
| You see | Why | Fix |
|---|---|---|
| ValueError: Found unknown categories | A category appears at transform time that fit never saw | OneHotEncoder(handle_unknown="ignore") |
| ValueError: Input X contains NaN | Model received missing values | Add a SimpleImputer step before the model |
| could not convert string to float | A categorical column reached a numeric step | Route it through the categorical branch of the ColumnTransformer |
| Test score much higher than expected | Leakage: post-outcome column or fitting before the split | Check when each feature is known; fit inside a Pipeline |
| Cross-validation scores vary a lot | Small data or rare class | Stratified k-fold; more folds or repeated CV; report mean ± std |
| SettingWithCopyWarning | Editing a slice of a DataFrame | Use .loc or .copy() on the slice |
| Resource | Why read it |
|---|---|
| scikit-learn User Guide, “Preprocessing data” and “Common pitfalls and recommended practices” (scikit-learn.org) | Official guide to scalers, encoders, imputers and leakage |
| Zheng & Casari, Feature Engineering for Machine Learning (O'Reilly) | Practical encodings, scaling and text features |
| Kuhn & Johnson, Feature Engineering and Selection (free at feat.engineering) | Missing data, encodings and resampling in depth |
| Kaufman et al., “Leakage in Data Mining: Formulation, Detection, and Avoidance” (ACM TKDD, 2012) | The classic paper on how leakage happens |
Lab 03 · A leak-free preprocessing pipeline
pipeline.py prepares 2,000 customers for a churn model with scikit-learn. It runs cleanly and reports an impressive AUC of 0.935 — which is a lie. One bug leaks the answer, one leaks the test set into fitting, one crashes on a region the model has never seen.
pip install -r requirements.txt, thenpython check.py.- Check 1 and 5: which column is only known after a customer churns? Exclude it in
split_columns. - Check 3: the imputer's medians match all rows. Move the fit so it only sees training rows.
- Check 4: a new region crashes
transform. WhichOneHotEncoderoption fixes it? - Re-run after each fix until all five checks are ok — and notice the AUC drop to an honest number.
Done when you see PASS: all 5 checks passed — leak-free pipeline, honest test AUC = 0.766.