Undergraduate teaching demos
These activities turn common data science mistakes into classroom experiments. Students change the setup, see the result, then explain what the evidence supports.
Lab 1
Model Evaluation and Data Leakage
Students compare a random split with a group held out split and decide which score supports a claim about new sites, cohorts, or sensors.
Try this
Use the controls to make the gap between the random split score and the group held out score larger or smaller. Which score would you trust for a new site, and why?
The group held out score is better for a new site because the test data come from a setting the model did not see during training. The lower score does not mean the model got worse, it means the evaluation is asking a harder and more realistic question.
Lab 2
Machine Learning
Students adjust model complexity and data quality, then explain why a model can look impressive in training but fail to generalize.
Try this
Increase model complexity and watch the training and test scores. At what point does the model start memorizing instead of learning a useful pattern?
A high training score is not enough evidence that the model will work on new data. When the gap grows, the model may be fitting noise or details that do not repeat outside the training set.