Local Government Financing Issues Presentation at LG Budget Workshops

Transportation Systems Management and Operations (TSM&O) Performance Measurement Dimension based on Federal Highway Administration Capability Maturity Model Workshops 1

The Challenges/Opportunities for TSM&O Challenges Congestion and delay are increasing as economic and population growth continue but existing roadways have capacity constraints. A high value is placed on the reliability of the system. Reviewing the existing TSM&O versus state of practice: Unsystematic; and Pockets of excellence. Opportunities Unexploited potential of aggressive, integrated, collaborative TSM&O applied to existing roadways. Agencies can develop the key capacities needed to improve TSM&O effectiveness. 2

What This Series of Webinars Is About Objective: Mainstreaming continuous improvement in TSM&O Process: Major program of SHRP2 research and over 40 FHWA-sponsored State DOT workshops Workshop Process: Agency staff self-evaluation and improvement plans, using capability maturity model (CMM) White Papers are available covering: State of the practice regarding agency capabilities Current agency approaches for improvement www.ops.fhwa.dot.gov/plan4ops/focus_areas/orga nizing_for_op.htm 3

The Dimensions of Capability Effective TSM&O Strategies Systems and Performance Measurement Technology Business and technical processes support strategies Business Processes Capabilities Organization and relationships support processes Collaboration Organization and Staffing Culture 4

The 6 Dimensions of CMM Business Processes, including planning, programming and budgeting (resources) and project development and procurement. Systems and Technology, including use of systems engineering, concepts of operations, systems architecture standards, interoperability, and standardization. Performance Measurement, including measures definition, data acquisition, analytics, communication and utilization. Culture, including technical understanding and business case, leadership, outreach, and program legal authority. Organization and Staffing, including programmatic status, organizational structure and accountability, staff capabilities, training/development, and recruitment and retention. Collaboration, including relationships with public safety agencies, local governments, MPOs, and the private sector. 5

Synergism among Dimensions 6

The Capability Maturity Model Goal for the Future LEVEL 4 Optimized Performance-based improvement Formal program Formal partnerships LEVEL 3 Most Agencies Today Integrated Process documented Performance measured Organization/ partners aligned Program budgeted LEVEL 2 Managed Processes developing Staff training Limited accountability LEVEL 1 Performed Activities & relationships ad hoc Champion-driven 7

Performance Measurement: Criteria for Level Achievement Capability Level 1 Some output-level performance is measured and reported by some jurisdictions Capability Level 2 Output-level performance measures are used directly for after-action debriefings and improvements; data easily available and dashboarded Capability Level 3 Outcome-level measures identified (networks, modes, impacts) and routinely utilized for objective-based program improvements Capability Level 4 Output and outcome performance measures reported internally for utilization and externally for accountability and program justification 8

Workshop Capability Self-Assessments Capability Self-Assessment Dimension Level 1 Performed Level 2 Managed Level 3 Integrated Level 4 Optimizing Business Processes 11 10 2 0 Systems and Technology Performance Measurement 7 12 3 1 9 11 3 0 Culture 8 11 4 0 Organization and Staffing 8 9 6 0 Collaboration 4 12 6 1 9

Advance Warning! Workshop findings will be presented. Followed by a brief case example. Along the way think of your questions or comments: Type these into the webinar comment box. 10

Performance Measurement Dimension CMM Workshop Results The summary in slides that follow will provide: Typical state-of-play; Self-improvement actions suggested by workshop participants (in Implementation Plans); Not a how to but a descriptions of what TSM&O staff identify as key actions for improvement in Performance Measurement Dimension; and May help you think through applicable actions in your agency. State DOTs differ widely in size, number of regions/districts, urban vs rural, etc. No standard solutions (but there are peer agencies). 11

What is Performance Measurement? Why is It Important? Accountability to elected officials and the public: What did we get for our money? Proactive public relations for the agency: Shows we know what s going on without relying solely on anecdotes or reacting to events. Evens the playing field for Operations in promoting a data-driven program. but most importantly 12

What is Performance Measurement? Why is It Important? It is a sound business practice. We measure performance because it helps us get better at what we do: Detects and corrects problems in an era of shrinking budgets and leads to better projects being implemented. Helps to manage, describe, and improve processes: operations programs are better structured. Allows for ongoing evaluations: demonstrating the value of our activities. Becomes the foundation for performance management. 13

Performance Measurement Dimension CMM Workshop Results The key elements of Performance Measurement: Measures definition; Data acquisition; and Measures utilization (taking action based on trends and evaluations.) 14

Element 1 Measures Definition: State of Play Policy visibility of performance. Impending requirements of Moving Ahead for Progress in the 21st Century Act (MAP-21.) Developing performance measures. Lack of performance measure definitions for weather, work zones, and signalized arterials was frequently mentioned as a problem. Transportation Incident Management (TIM) measures often not coordinated between DOTs and safety agencies. 15

Element 1 Measures Definition: State of Play Input, output, and outcome measures: Output measures most commonly measured (operations activities); and, Need for standardization and guidance mentioned. Resources for Performance Measurement: Obtaining funding for Performance Measurement is a challenge for some agencies; and In some cases, upper management is not convinced of the need for it, and the funding must come from existing budgets. 16

Element 1 Measures Definition: Progress MAP-21 performance management requirements will give push to programs. FHWA projects on performance measures for TIM, work zones, weather, as well as conducting evaluations. 17

Element 1 Measures Definition: Participants Proposed Actions Coordinate TSM&O Performance Measurement with TSM&O Strategic Plan. Develop Performance Measurement Plan and/or Data Business Plan that includes: Measures and targets; Data availability and acquisition; Use of measures for project selection/prioritization and resource allocation; Internal and external reporting audiences/needs/methods; and Evaluations of completed projects. 18

Element 1 Measures Definition: Participants Proposed Actions Standardize TSM&O Performance Measurement related to both recurring and nonrecurring congestion, especially TIM. Develop a TSM&O Performance Measurement pilot program or staged approach to implementation. Review MAP-21 requirements being implemented at the agency level: Targets and timeframes; and Ensure common terminology and objectives between TSM&O and other agency activities. 19

Element 2 Data Acquisition: State of Play Existing data availability: Some TSM&O units collect and own TIM data; others are dependent on emergency responder and computer aided dispatch (CAD) systems. Freeway detector data also are widely available but not all agencies use them to develop congestion statistics (outcome measures). Work zone data are difficult to obtain usually overseen by other units within the agency. Signalized arterial data is difficult to obtain. Outsourcing: Private vendor vehicle probe data are becoming more widely available. 20

Element 2 Data Acquisition: Progress Travel time data becoming widely available from private vendors; can serve multiple units within DOTs. Several agencies mentioned MAP-21 as a driving force behind travel-time/speed data acquisition: Many suggested that they were looking into probe data not only to meet MAP-21 requirements but also to fill in gaps where detectors do not exist. Several agencies producing operations-oriented performance reports. 21

Element 2 Data Acquisition: Participants Proposed Actions Identify ongoing collaboration with external partners: Measure definition; Shared data; and Coordinated/integrated planning and programming (e.g., with MPOs.) Identify responsibility for maintaining data system in Business Plan. 22

Element 3 Measures Utilization: State of Play Internal utilization: Incident management and snow and ice control are the two areas where performance data are used for operational management. TIM data widely collected and reported, but infrequently used to modify TIM activities and programs. After-action review of incidents becoming common. Traveler information program performance (e.g., web site hits and VMS messages) monitored and used to make upgrades. The development of outcome measures is impeded by limitations on the availability and integration of multisource data. Overall, agencies struggling with how to integrate performance measurement into decision-making. 23

Element 3 Measures Utilization: State of Play External reporting: Production of periodic performance reports was the most common use of performance measures, although not all agencies produced them. A few states included TSM&O-related activity measures largely output data on external dashboards, usually incident trends. Travel time (congestion) based reports based on measured data were far more rare. Conducting customer surveys related to the delivery of operations services is rare. 24

Element 3 Measures Utilization: State of Play Management accountability: Accountability for TSM&O program performance is in the early stages; and Incident clearance targets are sometimes set; reviews conducted only when the target (often 90 minutes) is exceeded. Comprehensive performance management program: No agency has achieved a fully integrated Performance Measurement system that links inputs, outputs, outcomes, and targets into a formal TSM&O performance management process. Outcomes also are affected by other programs, such as capacity expansion, demand management, alternative mode use, and safety countermeasures. 25

Element 3 Measures Utilization: Progress Again, MAP-21 seen as the driver for agency performance management activities. Agencies are venturing out into components of performance management that are most useful to their own situations, but these examples are scattered around the country. 26

Element 3 Measures Utilization: Participants Proposed Actions Develop a strategy for how performance data will be integrated into agency processes, including project planning and arterial operations. Establish a format and overall approach for how measures will be displayed and reported for both internal and external use. Review experience and activities of peer agencies. Formalize process for post-incident management assessment, including definitions, performance-related criteria follow-up, and data assembled. 27

Element 3 Measures Utilization: Participants Proposed Actions Use of performance measures in business case materials: Guidance on conducting before/after evaluations of operations projects needed; and Demonstrating benefits of operations needs to be a routine activity in order to build support for TSM&O programs: A communication strategy for describing the benefits of TSM&O to upper management and the public is the basis for this. 28

One Executive Summary (covers all dimensions) Six White Papers one on each capability dimension http://www.transportationops.org/ 29

Suggested National Activities to Address Performance Measurement Possible sponsorship FHWA Activities Checklist for developing a TSM&O Performance Management Plan Standardized definitions for a wide range of TSM&O performance measures covering TIM, work zones, weather, and signalized arterials FHWA Best practices for communicating TSM&O performance FHWA OPMM #13 Guide to TSM&O data acquisition, management, and analysis methods NCHRP or FHWA Peer-to-Peer Exchange on TSM&O Performance Measurement FHWA Best practices in using TSM&O performance measures in investment decision-making NCHRP or Pooled Fund FHWA Federal Highway Administration OPMM Operations Performance Measures and Management NCHRP National Cooperative Highway Research Program 30

What You Can Do Read the CMM reports and white papers. Participate in FHWA state DOT Self-Evaluation Workshops and Regional Operations Forums. Utilize NOCoE website to track developments and access resources. Contact and communicate with corresponding staff in peer states. 31

Questions/ Contact FHWA HQ office through your local FHWA Division offices or NOCoE dmotiani@transportationops.org 202.624.5478 32