Model Comparison Seminar

TU Dortmund, Summer semester 2024

Course introduction

About us

Paul Bürkner

• https://paul-buerkner.github.io/

Šimon Kucharský

• Simon
• https://kucharssim.github.io/

About you

• Please shorty introduce yourself
• How much experience do you already have with model comparison?
• What motivated you to take the course?
• What do you expect to learn?

Why model?

1. Description
2. Explanation
3. Prediction
4. Control

Why model comparison?

• Uncertainty in what is a “good” model
• Narrow sense
  • Which model is “better”
  • Quantify the distance between models
• Broader sense (Bürkner et al., 2023)
  • Fairness
  • Interpretability
  • Parsimony
  • Robustness
  • Estimation speed
  • …

Why so many different methods?

• Goals, e.g.
  • Prediction vs Explanation (Breiman, 2001; Shmueli, 2010)
• Assumptions, e.g.
  • M-Open vs M-Complete vs M-Closed (Bernardo & Smith, 2009)
• Frameworks, e.g.
  • Frequentist vs Bayesian vs ML
• Resources, e.g.
  • Estimation speed, computational demands (Bürkner et al., 2023)

Goals of this course

• Understand the basic principles
• Overview of available techniques
• Practice your technical skills
• Practice your soft skills (academic writing and presenting)

Course info

Course responsibilities

• We provide
  • Broad introduction
  • Basic literature
  • Guidance
• You provide
  • Introduction to specific methods
  • Your own interests

Website

• Basic info
• Course schedule
• Topics
• Guidelines

Moodle

References

Bernardo, J. M., & Smith, A. F. (2009). Bayesian theory (Vol. 405). John Wiley & Sons.

Breiman, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science, 16(3), 199–231.

Bürkner, P.-C., Scholz, M., & Radev, S. T. (2023). Some models are useful, but how do we know which ones? Towards a unified bayesian model taxonomy. Statistic Surveys, 17, 216–310.

Shmueli, G. (2010). To Explain or to Predict? Statistical Science, 25(3), 289–310. https://doi.org/10.1214/10-STS330