Calibrating Large-Scale Travel Demand Models: Workshop Insights
Explore the challenges in calibrating regional travel demand models at a workshop by the University of Toronto. Understand the processes of estimation, calibration, and validation, and address key issues in data collection and performance measures.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Workshop Objectives, Issues & Needs Calibrating Large-Scale Travel Demand Models: Issues & Options Travel Modelling Group Workshop University of Toronto, May 6, 2020 Eric J. Miller, Ph.D. Prof., Dept. of Civil & Mineral Engineering Director, UTTRI University of Toronto
Objectives Explore issues in large-scale regional travel demand model system calibration (& validation). Hear from a variety of public & private sector practitioners. Get ideas & issues out on the table for discussion. Starting point for developing guidance for GTHA agencies & consultants for standardizing calibration methods.
Definitions Estimation: Statistically determine model parameters that generate best-fit to observed (sample) data. Calibration: Post-estimation adjustment (& possibly addition) of parameters to improve fit to base data. Validation: Out-of-sample testing to determine model predictive performance . Data Collection Choose Modelling Method Base Demand (D) Model Estimation & Calibration Base Attributes (A,S,T) Other Historical Data Model Validation Forecasts of Exog. Variables Forecast Demand/ Analyze Policies Specify Policy Options
Estimation Statistically appropriate methods need to be used. Avoid estimation bias. Maximize efficiency of parameter estimates (minimize parameter standard deviations). Data used for model estimation. Largely TTS. Need to keep in mind limitations in the data.
Calibration vs. Validation In practice, we did to confuse the two. Validation can involve: Prediction against data not used in model estimation & calibration: Another year. Hold-out sample. Assessment of elasticities. Sensitivity testing. Arguably, we do too much calibration and too little validation.
Issues in Calibration Data. Performance measures. Calibration procedures. Changing environment for modelling.
Calibration Data Screenline/cordon counts. Accuracy & completeness? Time period collected? Transit boarding & station counts. Agency data: available? Consistent? TTS records. ATC data, other counting technology. Coverage? We tend to ignore that calibration target data are also subject to statistical & measurement error. What is truth?
Performance Measures Goodness-of-fit measures (RMSE, R2, GEH, etc.) Acceptable values? Need to recognize that the target data are not perfect . Other criteria that should be considered? How should we judge our models?
Calibration Procedures Tend to be very ad hoc. Difficult to systematically adjust a complex travel demand model system. Subway station counts at a key transfer station are off . Where does the problem lie: generation, distribution, mode split, assignment? It is very easy to over-calibrate (over-fit) a model.
Challenges Ahead, 1 The current COVID-19 crisis has totally disrupted the transportation system and travel behaviour. We do not know what the new normal will be. How can our current models help us explore these questions? What will new models need to look like? How can we build them?
Challenges Ahead, 2 Arguably, we need much more flexible, behaviourally fundamental models that can better adjust to new & future realities . We were already facing these issues with respect to dealing with new/emerging mobility technologies & services. We now are also facing fundamental changes in how we travel. It is a brave new world !
