Enhancing Cost Analyst Training: Best Practices and Challenges

Training Cost Analysts A Cohesive Pedagogical Framework for Success Kammy Mann Herren Associates Washington Capital ICEAA Chapter Luncheon July 27, 2016

The purpose of this effort was to identify: the current state of the Cost Analyst training program, the best practices in the industry, and training gaps. Best Practices (Industry and Research) Training Available Understand Required Skills Gaps in Training Assess Combine information to produce a proposed pedagogical framework & recommendations Recommendation 2 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

How do you Learn Cost Analysis? Field of Education Percent Business/Management 20% Engineering 28% Math/Statistics 11% Economics 11% Accounting/Finance 15% Operations Research 5% Computer Science/Information Systems 1% Physical Sciences 2% Other 7% ICEAA Member Survey, 2013 3 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Current State/Challenges Cost Analysis is not an easy profession! Training employees cost time, money and materials There are currently no common or consistent training requirements for cost analysts Mistrust between the cost shop and the rest of the organization Weather Man analogy 4 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Importance of Training Training employees costs time, money, and materials Research has shown that not training employees leads to discontentment and turn over Time and Money can be lost when untrained workers make mistakes, re- work occurs, and inadequate products are delivered 5 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Challenges Cost Estimators Typically Face GAO Cost Estimating and Assessment Guide, 2009 6 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Academic Literature Review Research was taken from the field of Engineering Education Students learn by doing design/hands-on activities vs. traditional lectures ( Sage on the Stage ) Problem/Project based learning shows higher knowledge retention rates Constructivism knowledge is learned through experience Continuous feedback and assessment, tailor to individual student s learning style Cooperative/cross-functional team learning is integral to prepare students for the work force 7 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Pedagogical Frameworks A Pedagogical Framework is a structured method for teaching that guides the instructor in a systematic way to instruct the students. Step 1: Identify Desired Results Step 2: Determine Acceptable Evidence Step 3: Plan learning experience and instruction Understanding by Design Bloom s Taxonomy 8 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

An understanding of what drives effective training is knowledge of positions, anticipated schedule, and data availability Essential Question: Does current training meet training needs? Survey Feedback What we did: Conducted a Survey of ICEAA members Training Analysis Best Practices Analysis What we did: Reviewed CEBoK, DAU, CCEA and contractor training programs What we did: Studied policies and documents to understand best practices Purpose: Provide quantitative story on training drivers Purpose: Identify available training and certification programs Tie the qualitative story to quantifiable events Purpose: Determine framework and best practices Determine if policy roadmap is consistent with survey feedback Themes: On the job mentoring most profitable Computer Science/math and Statistics desired further training Recurring drivers stem from lack of on the job mentoring Themes: Continuing education lacking Training to become future leaders/mentors lacking Themes: Framework provides applicable building blocks Few opportunities for continuing education 9 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Initial Outputs Phase 1 Data Collection Survey Interviews 15 cost analyst from 05C and Herren Top Issues Pain Points Documentation 73 ICEAA members Top Issues Pain Points Roles/Responsibilities GAO Cost Guide CEBoK Round Two Assessment Feedback 10 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Disciplines and Concepts in Cost Analysis GAO Cost Estimating and Assessment Guide, 2009, p. 51 11 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

An questionnaire was developed as a guide to drive discussion about training for Washington Area Chapter ICEAA members Key Questions Most Profitable Training? Gaps in Training? Applicability of DAU? Areas for Improvement? Job Satisfaction? 12 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Survey Feedback: The goal of the survey process was to be able to tie anecdotal evidence to specific observed training gaps Preferences for 20+ years Output Years of Experience: 20+ Formal classrooms have the least effect on overall learning Mentorship has the greatest effect on overall learning There is a desire for a collaborative/team environment between departments and organizations On the job mentoring On the job mentoring 7% Certifications Certifications 11% 39% In-house training (peer reviews, working groups, brown bags, etc.) In-house training 14% Educational background Educational background 29% DAU Courses DAU courses Area for Improvement Preferences for 0-5 years Greatest Area for Improvement Years of Experience: 0-5 2% On the job mentoring On the job mentoring 3%2%2% Collaboration and understanding between cross-functional departments/agencies Collaboration 7% 7% Educational background Educational background Mentorship and on-the-job training Mentorship 14% 30% Certifications Certifications 61% 72% In-house training (peer reviews, working groups, brown bags, etc.) In-house training 13 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Common Themes - Interviews Pressures that Limit Progress Absence of continuing education Sources limited exclusively to ICEAA Absence of leadership track Need future and leaders mentors Lack of collaboration and understanding Cross-functional departments and agencies Lack of on the job mentoring 14 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Common Themes - Interviews Advice for Training Two-week intensive course Create road-map with accompanied training Increase job-specific training Continued training after basic is completed Offer mentorship programs 15 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Proposed Framework Understanding by Design Find out Student s Learning Style Step 1: Identify Desired Results Acquire (Classroom) Meaning (Constructivism) Transfer (Mentorship) Step 2: Determine Acceptable Evidence Apprentice Reflection & Assessment Methods of Learning Cost Analysis Discovery Incidental Step 3: Plan learning experience and instruction Inductive Deductive Bloom s Taxonomy 16 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Recommendations Establish a systematic methodology for training Leverage current trainings such as DAU, GAO, ICEAA CEBoK. However, don t rely on sage on a stage lecturing! Consider utilizing a hands-on design/project based approach to training (case studies, team projects, etc.) Mentoring is very important! Assessment and continuous improvement is key 17 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Proposed Training Schedule for New Analysts 1. Assign new analyst a mentor to guide them through the learning process Have analysts take a self assessment of learning style and discuss how they can get the most out of the training process 1-2 week intensive classroom setting to acquire basic cost analysis knowledge, applying principles of Bloom s Taxonomy At the end of the classroom session, to create meaning, have the analysts participate in a case study/design project in a team environment Throughout the first 2 years of working in the field have the analyst meet frequently with their mentor and cohort for self reflection and additional hands-on training Have the analysts act as mentors and teachers for the new group of analysts who enter the program in subsequent years 2. 3. 4. 5. 6. 18 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Future Work/Research Test the proposed pedagogical framework and document in a case study Consider other fields of educational research Workforce development research and best practices Let s start a conversation about training our next generation of analysts! 19 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Questions? 20 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Contact Info Kammy Mann: Kammy.mann@jlha.com 21 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Backup 22 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Citations List Adams, R.S., & Felder, R.M. (2008). Reframing Professional Development: A Systems Approach to Preparing Engineering Educators to Education Tomorrow s Engineer. Journal of Engineering Education, 239-240. Berry, J., & Asiello, D. (2011). Department of Defense Sustainability Planning: The View from the Second Year. New Orleans, LA: Noblis. Bloom, B. S.; Engelhart, M. D.; Furst, E. J.; Hill, W. H.; Krathwohl, D. R.(1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company. Bloom, B.S. & Krathwohl, D.R. Taxonomy of Educational Objectives. Handbook 1: Cognitive Domain. Addison-Wesley, New York, 1984. Bordogna, J. "Engineering The Integrative Profession," NSF Directions, May/June, Vol. 5, No. 2, (1992). Brereton, M.F., J. Greeno, J. Lewis, C. Linde and L. Leifer, "An Exploration of Engineering Learning," Proceedings of the 5th International Conference on Design Theory and Methodology. Held: Albuquerque, NM, USA. American Society of Mechanical Engineers, Design Engineering Division (Publication) DE v 53, Published by ASME, New York, NY, USA. p. 195-206, (1993). Cost Estimating Body of Knowledge (CEBoK) (1.2 version). (2013). Vienna, VA: International Cost Estimating and Analysis Association. Estimated Impacts of Sequestration-Level Funding. (2014). Washington, D.C.: Department of Defense. <http://www.defense.gov/pubs/2014_Estimated_Impacts_of_Sequestration-Level_Funding_April.pdf>. Felder, R.M., Woods, D.R., Stice, J.E., & Rugarcia, A. (2000). The Future of Engineering Education, Teaching Methods that Work. Chemical Engineering Education, 34(1), 26-39. Forehand, M. (2010, July 6). Bloom's Taxonomy - Emerging Perspectives on Learning, Teaching and Technology. Retrieved March 23, 2016, from http://www.roe11.k12.il.us/GES%20Stuff/Day%204/Process/Blooms/Mary%20Forehand%20discussion- Bloom%27s%20Taxonomy.pdf Hein. G.E. (1991). Constructivist Learning Theory. Proceedings from: International Committee of Museum Educators Conference, Jerusalem, Israel. ICEAA. (2013). Survey of Members. Vienna, VA: International Cost Estimating and Analysis Association. Jonassen, D.H. & Ronrer-Murphy, L. (1999). Activity Theory as a Framework for Designing Constructivist Learning Environments. ETR&D, 47(1), 61-79. Leifer, L. remarks made at the International Workshop on Project-Based-Learning, Stanford University, Aug. 7-10, (1995). Lipman, V. (2013). Why Employee Development is Important, Neglect and Can Cost you Talent. Forbes. Retrieved from: http://www.forbes.com/sites/victorlipman/2013/01/29/why-development-planning-is-important-neglected-and-can-cost-you-young- talent/#60dd33e62ce0 Mills, J.E. & Treagust, D.F. (2003). Engineering Education Is Problem-Based or Project-Based Learning the Answer? Austrailasian Journal of Engineering Education, 2-16. National Science Foundation (NSF). (1995). Restructuring Engineering Education: A Focus on Change. Report of an NSF Workshop on Engineering Education, June 6-9, 1994, Arlington, VA. Perkins, D.N. "What Constructivism Demands of the Learner," Constructivism and the Technology of Instruction: a conversation, LEA Associates, Hillsdale, NY, p. 161-165, (1992). Richey, Karen. (2015) GAO Cost Estimating & Scheduling, Best Practices and Audit Findings. Washington, D.C. Sanders, M. (2009). STEMmania. The Technology Teacher, 20-28. Sewell, M. & Marczak, M. (2016). Using Cost Analysis in Evaluation , The University of Arizona, retrieved from 23 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Citations List (Part 2) Sheppard, S. & Jenison, R. Freshman Engineering Design Experiences: an Organizational Framework International Journal of Engineering Education. 1996. Soloman, B.A. & Felder, R.M. (2016). Index of Learning Styles Questionnaire. NC State University. https://www.engr.ncsu.edu/learningstyles/ilsweb.html Sonwalkar N. Changing the Interface of Education with Revolutionary Learning Technologies: An Effective Guide for Infusing Technology Enabled Education for Universities and Corporations. Presentation at iUniverse, New York, September 2004. Sonwalkar N. The sharp edge of the cube: pedagogically driven instructional design for online education. Syllabus December 2001, pp. 12 16(www.campus-technology.com/article.asp?id = 5858); Sonwalkar N. Changing the Interface of Education with Revolutionary Learning Technologies. Syllabus November 2001, pp. 10 13, (http://www.campus-technology.com/article.asp?id = 5663). Sonwalker, N. (2013). The First Adaptive MOOC: A Case Study on Pedagogy Framework and Scalable Cloud architecture Part 1. MOOCs Forum, 22-29. Spurlin, J. E., Rajala, S. A., & Lavelle, J. P. (2008). Designing better engineering education through assessment: A practical resource for faculty and department chairs on using assessment and ABET criteria to improve student learning. Sterling, VA: Stylus Pub. Teslow, J.L., L.E. Carlson and R.L. Miller, "Constructivism in Colorado: Applications of Recent Trends in Cognitive Science," ASEE Annual Conference Proceedings, Session 1602, (1994). The Compass. (2016). Understanding by Design: A Framework to Support Effective Teaching and Learning. Newton-Conover s Online Newsletter for Educators. Retrieved from: http://nccscurriculum.org/2014/01/22/understanding-by-design-a-framework-to- support-effective-teaching-and-learning/ Weimer, M. (2002). Learner-centered teaching: Five key changes to practice. San Francisco: Jossey-Bass. Wiggins, G., & McTighe, J. (2005). Understanding by Design (Expanded 2nd ed.). Alexandria, Virginia: Association for Supervision and Curriculum Development. Woods, D.R. Problem-based Learning: How to Gain the Most from PBL. Woods Publishing, Waterdown, 1994. Distributed by McMaster University Bookstore, Hamilton, ON. Wulf, W. (1998). The Urgency of Engineering Education Reform. Proceedings from: Realizing the New Paradigm for Engineering Education, Baltimore, MD. VARK Questionnaire. (2016). Retrieved on 29 March, 2016 from: https://www.engr.ncsu.edu/learningstyles/ilsweb.html U.S Government Accountability Office. (2009) GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. Washington, D.C.: Applied Research and Methods. 24 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION