DoD Standardization Initiative for Systems Engineering Reviews and Audits

Systems Engineering Standard Technical Reviews & Audits Standard Presentation to NDIA SE Division 11 December 2013 Garry Roedler (WG Chair) 1

Agenda DoD Goals and Objectives Background Current Situation and Approach Current Status Looking Forward 2

Goal of DoD Standardization Initiative Provide DoD and DoD contractors with a structured, uniform approach in the areas identified as gaps Improve communication Ensure common expectations Add realism to bids 3

Standardization Objectives Mutual-benefit stipulations: Must meet both party s needs and objectives Potential teaming partners must have existing experience with subject matter of document and existing infrastructure for publishing standards Content of documents must be consistent with government needs Ensure right kind of standard is developed Clear statement of requirements that speaks common language Specific enough to support realistic bidding and resourcing Proper contractual language SHALL and should Ensure right amount of the right standards are included Tailorable for different domains and contracting environments Consistent with DOD approach Conform to established, over-arching industry process standards 4

Current Situation Responsible SDO Expected Publication Notes Standard Focus Approach ISO/IEC/IEEE 15288 (System Life Cycle Processes) will be used as basis for Systems Engineering needs. Full draft available for review in mid-2014 Increments under review now as drafted. Project is expected to produce an addendum to 15288 Standard for Application of Systems Engineering on Defense Programs". Systems Engineering IEEE-CS End of 2014 IEEE 15288.1 Full draft available for review in mid-2014 No acceptable industry standard to use as base. Increments under review now as drafted. New full standard to be developed - Standard for Application of Technical Reviews and Audits on Defense Programs". EIA 649B will be used as basis for Configuration Mgt needs. Technical Reviews and Audits IEEE-CS End of 2014 IEEE 15288.2 Configuration Management Project is expected to produce an addendum to EIA 649B. SAE Intl Mid 2015 Intended to document best manufacturing practices aimed at promoting the timely development, production, modification, fielding, and sustainment of affordable products Manufacturing Management Recent selection - team is forming - approach to be announced. Revision of MIL-HDBK-502A, which provides application guidance to the TechAmerica Standard 0017 Still under review SAE Intl TBD Adopted by DoD in June 2013 TBD Logistics Support Reliability SAE Intl TBD Each standard is intended to be put on future system contracts 5

DOD SE/TRA Standards Process Present effort 6

Membership of DoD Joint SE Working Group Industry Associations Aerospace Corp BAE Systems Ball Aerospace Boeing General Dynamics Harris Corp L-3 Com Leidos Lockheed Martin Northrop Grumman Raytheon SAIC United Technologies Technical Editor, Mark Henley, L-3 Com (TR&A) Defense Air Force (Multi-parts) Army Navy (Multi-parts) OSD DASD (SE) DAU DCMA DSPO AIA IEEE-CS/SA INCOSE ISO/IEC NDIA SAE Intl Leadership Team Chair, Garry Roedler, Lockheed Martin Vice-chair, Dave Davis, USAF SMC Secretary, Brian Shaw, The Aerospace Corp. Technical Editor, Bill Bearden, INCOSE (SE) Membership of WG Has Broad Span Will be adding members to cover academia 7

Review of IEEE Standards Process IEEE Process Stresses Consensus 8

Reviewers Focus Need to ensure: Technical accuracy Usability in defense environment Requirements are relevant and reflect good practice Requirements for deliverables are compatible with current methods, and do not drive a particular method/technique Requirements do not result in unintended consequences 9

Schedule Information Business rhythm/cadence TR&A and SE will have 2-week phase separation for increments Meeting Date +0 +21 +25 +28 +7 +14 Leadership Team Initial Review Editor Consolidates Comments WG Provide Inputs Editor Prepares Draft WG Review of Draft Draft to WG for Review Submit Review Comments to Editor Review Issues from Comments at Mtg 10

IEEE Joint SE WG Schedule/Status Integrated schedule balances efforts of both projects Draft development : on-going and on schedule Working draft: May 2014 Publication: December 2014 1 2 Each project = 7 development increments plus reviews and balloting 15288.1 (SE) 2 increments completed; 3rd increment in review. 15288.2 (TR&A) 2 increments completed; 3rd increment about to be released. Progress: Joint WG for DoD SE Standardization - Schedule D: draft issued I: inputs for draft due W: working group meeting discussion (Review Comments due 2 days prior to meeting) 10/10/2013 10/17/2013 10/24/2013 10/31/2013 11/14/2013 11/21/2013 11/28/2013 12/12/2013 12/19/2013 12/26/2013 10/16/2014 10/23/2014 10/30/2014 11/13/2014 11/20/2014 11/27/2014 12/10/2014 12/20/2014 12/25/2014 8/15/2013 8/22/2013 8/29/2013 9/12/2013 9/19/2013 9/26/2013 10/3/2013 11/7/2013 12/5/2013 1/16/2014 1/23/2014 1/30/2014 2/13/2014 2/20/2014 2/27/2014 3/13/2014 3/20/2014 3/27/2014 4/10/2014 4/17/2014 4/24/2014 5/15/2014 5/22/2014 5/29/2014 6/12/2014 6/19/2014 6/26/2014 7/10/2014 7/17/2014 7/24/2014 7/31/2014 8/14/2014 8/21/2014 8/28/2014 9/11/2014 9/18/2014 9/25/2014 10/2/2014 10/9/2014 11/6/2014 12/4/2014 8/1/2013 8/8/2013 9/5/2013 1/2/2014 1/9/2014 2/6/2014 3/6/2014 4/3/2014 5/1/2014 5/8/2014 6/5/2014 7/3/2014 8/7/2014 9/4/2014 15288.1 SE Standard (addendum 15288.1) SE working group meetings SE Draft outline 8/1 8/15 X 9/12 10/10 11/7 12/5 1/9 2/6 3/6 4/3 5/1 5/29 6/26 7/24 8/21 9/18 10/16 11/13 12/10 SE Inc #1:Business or mission analysis, Stakeholder needs and requirements definition process, System requirements definition process, Architecture definition process, Design definition process, System analysis process SE Inc #2: Implementation process, Integration process, Verification process, Transition process, Validation process, Operation process SE Inc #3: Maintenance process, Disposal process, Acquisition process, Supply process, Project planning process, Project assessment and control process SE Inc #4: Decision management process, Risk management process, Configuration management process, Information management process, Measurement process, Qualifty assurance process I D W I D W 1 I D W Balloting Period I D W SE Inc #5: Lifecycle model management process, Infrastructure management proces, Portfolio management process, Human resource management process, Quality management process, Knowledge management process SE Inc #6:Clauses 1 - 5, annexes SE Inc #7: additional info needed SE Working 15288.1 draft complete SE Mandatory Editorial Coordination (MEC) SE Establish Ballot Group (draft near completion) SE Draft Complete Ready for Ballot SE Establish Small Ballot Comment Resolution Group SE Ballot start SE Ballot close/results SE Recirculation updates/ballot (as needed) SE Submit to RevCom (Hard Date - deadline for meeting) SE RevCom Meeting SE Publish 2 I D W I D W I D W Draft Development D W 5/8 5/15 6/5 6/5 6/19 7/24 8/14 9/4 9/25 10/16 12/10 12/20 TRA 15288.2 Develoment Schedule - Rev 2 TRA working group meetings Draft Outline TRA Inc #1: 3 reviews/audits - ITR, SFR, SRR TRA Inc #2: 3 reviews/audits - SAR, SSR, PDR TRA Inc #3: 3 reviews/audits - CDR, IRR, TRR TRA Inc #4: 3 reviews/audits - FRR, FCA, SVR TRA Inc #1: 3 reviews/audits - PCA, P/MRR, ISR TRA Inc #6: front matter, annexes] TRA Inc #7: any additional info needed TRA Working Draft Complete TRA Mandatory Editorial Coordination (MEC) TRA Close Ballot Group Invitation (draft near completion) 8/1 9/5 X 10/3 10/24 11/21 12/19 1/23 2/20 3/20 4/17 5/15 5/29 6/26 7/24 8/21 9/18 10/16 11/13 15288.2 I D W I D W I D W I D W I D W I D W I D W I D W 5/8 5/15 TRA Draft Complete Ready for Ballot TRA Establish Small Ballot Comment Resolution Group TRA Ballot start TRA Ballot close/results TRA Recirculation updates/ballot (as needed) TRA Submit to RevCom (Hard Date - deadline for meeting) TRA RevCom Meeting TRA Publish 6/5 6/5 6/19 7/24 8/14 9/4 9/25 10/16 12/10 12/20 Note: 2014 RevCom Meetings Aug 19-21; Dec 8-10 Currently on track 11

Looking Forward 2014 Development and publication of aligned base documents ISO/IEC/IEEE 15288 (DIS 1Q14; FDIS 3Q14; Publish 4Q2014) IEEE 15288.1 (draft 2Q14; publish 4Q2014) IEEE 15288.2 (draft 2Q14; publish 4Q2014) Also, INCOSE SE Handbook (2Q2014) Communicate across other working groups to ensure consistency of terms, concepts, and other information Liaisons established with CM and Mfg Mgt Standards WGs Communication across WGs to agree on resolution of identified issues Start preparation for adoption, governance, support, use 2015 Adoption of standards (early 2015) Complete preparation for use of the standards Alignment of SE Resources 12

13

Source Products Source Products for SE Standard ISO/IEC/IEEE 15288 (CD.2), 2013 ISO/IEC/IEEE 15289, Content of life-cycle information products (documentation), 2011 SMC-S-001, Systems Engineering Requirements and Products, July 2013 DoDI 5000.02 NATO AAP-48, NATO System Life Cycle Processes, July 2012 (Addendum Standard to 15288; focused on NATO Armament Systems) Defense Acquisition Guide (DAG), Chapter 4, Systems Engineering, 2013 INCOSE SE Handbook V3.2 /V4.0 Draft SEBoK EIA 632, Engineering of a System ISO/IEC/IEEE 24748-4 NAVSEA Instruction & Policy 5000-009 for Systems Engineering Technical Reviews Source Products for TR&A Standard ISO/IEC/IEEE 15288 (CD.2), 2013 DoDI 5000.02 SMC-S-021, Technical Reviews and Audits for Systems, Equipment and Computer Software, Volume 1, September 2009 NAVSEAINST 5009.9, Naval Systems Engineering Technical Review Handbook, July 2009 NAVAIRINST 4355.19E, Systems Engineering Technical Review Process NAVAIR Systems Engineering Technical Review Process Handbook, initial release Defense Acquisition Guidebook, Chapter 4. Source Products include a combination of Industry and DoD resources 14

Source Products Usage Approach for Use of Source Products Use as body of knowledge for defense programs Identify relevant information that should be considered Identify differences or additional tasks that need to be considered Identify key outputs and their attributes Identify essential guidance needed to provide common understanding Assimilate key information at level that applies to most defense programs State in terms of what , not how 15

Review of PARs and Technical Approach Standard for Application of SE on Defense Programs Summary of PAR for Systems Engineering Identifier of Standard IEEE Std 15288.1 Title: Standard for Application of Systems Engineering on Defense Programs Scope: System life cycle processes, activities, and tasks of ISO/IEC/IEEE 15288 for use on any defense system across the life cycle Purpose: This standard implements ISO/IEC/IEEE 15288 for use by United States Department of Defense (DoD) organizations and other defense agencies in acquiring systems or systems engineering support. Need: Provide the defense specific language and terminology for the standard to ensure the correct application of acquirer-supplier requirements for a defense prgm. Technical Approach: Addendum to ISO/IEC/IEEE 15288 and will: Not repeat processes and information in 15288 Include defense specific language and terminology Include necessary tailoring or changes to existing elements Include any additional explanation or guidance 16

Review of PARs and Technical Approach Standard for Application of Technical Reviews & Audits Summary of PAR for Technical Reviews & Audits Identifier of Standard IEEE Std 15288.2 Title: Standard for Application of Technical Reviews and Audits on Defense Programs Scope: Establishes the requirements for technical reviews and audits to be performed throughout the acquisition lifecycle for the U.S. Department of Defense (DoD) and other defense agencies. Purpose: Amplify ISO/IEC/IEEE 15288 Clause 6.3.2.3.a for selection, negotiation, agreement, and performance of the necessary technical reviews and audits, while allowing tailoring flexibility for the variety of acquisition situations/ environments when the technical reviews or audits are conducted. Need: Provide the defense specific language and terminology for the standard to ensure the correct application of acquirer-supplier requirements for a defense program. Technical Approach: Standard will be in the form of a full standard that has links to ISO/IEC/IEEE 15288 and will: Elaborate on the activities and tasks related to TR&A Include defense specific language and terminology needed for the standard Include the criteria for reviews & audits Include the expected/required outcomes/ products of reviews & audits Include any additional explanation or guidance 17