Print Email
Home » Divisions » Integrated Program Management (IPMD) »

May 2003 meeting

NDIA Program Management Systems Committee Meeting

May 14-16, 2003

Government Roundtable Presentations, May 16, 2003 (in PDF format)

Download/Print/View Minutes in PDF Format

Wednesday, May 14, 2003

Opening and Welcome:
Jim Gasbarro called the meeting to order and initiated discussion regarding the recent Aldridge letter, issued through General Farrell, voicing NDIA concerns regarding EVM implementations and apparent declining support from DCMA, as their resources are stretched to cover additional requirements. DCMA is the executive agent for OSD; more resources are needed to accomplish this. The intent on the memo was to initiate discussion to establish a common approach for all agencies. A second purpose for the memo was to suggest the establishment of an executive steering group to address these issues rather than each agency going in different directions.

No direct feedback has been received in response, as yet. Dr. Nancy Spruill requested, and has been provided, specific program examples from industry to discuss with General Farrell. As the “keeper” of the standard and guidance, NDIA can help with the process of educating contractors and Government agencies, and may need to broaden their current perspective to encompass the needs of all. A common board is required for all agencies to reflect the needs and interests of all. The recommendation is to move further through the specifics of the issues, and must schedule time with Dr. Spruill to accomplish this. Gary Christle provided Dr. Spruill’s office an explanation of the prior Executive steering organization within DoD, which reported annually to the Under Secretary of Defense on the status of current issues. This group spawned the ANSI standard and abolished the original Joint Executive Group.

Working Groups:
The Compliance and Surveillance working groups met for the remainder of the morning session to discuss status and next steps.

The Risk Working Group requested NDIA members to complete a draft “Risk Questionnaire” for feedback, to be used to further refine the questionnaire prior to general distribution.

Government and Industry Informal Discussion:

  • USAF – Lt Col. Buck Ennulat
    Lt Col. Ennulat will be leaving the organization on June 30, 2003. His replacement will be Lt Col. Don Ruffin, whose background includes academic and acquisition experience.

    Currently we’re wrestling with “what is the real OMB requirement,” and we’re awaiting the DoD 5000.2R rewrite. We have a representative on this committee. The services haven’t gotten direction on how to react to OMB requirements.

  • DOE – David Treacy
    DOE has issued our common structural guidance for major systems of $400 M and above. We have several decision points: need, solution, PMB (APB), a requirement from the acquisition streamlining. Guidance applies to all contract types (fixed price, cost type, etc.) Contracts of $20 M and above require a business case and Earned Value reporting. Those contracts of $5 M and above must work within the framework of the guidance. We’ve hired DCMA as our executive agent, to do our contractors’ system acceptance, and provide accurate timely information. The MOA was executed in the beginning of 2003. We also do an external independent review as a function to validate the baseline. It’s an executability review by an independent entity.

  • NAVAIR – Ted Rogers
    Navy audit is completing a Syscom audit driven by ASM, how EVM is being utilized across the navy. The results will be interesting, including the contractors, DCMA and our program managers. NAVAIR has been completed; NAV SEA is next, followed by SUP Ships and then Missile Contractors.

    We’re heavily involved in SAP implementation across the Navy. It could change the way the Navy does Earned Value Management. The Navy is supposed to be a test case for SAP, and the results will affect whether it is then implemented by the other services. Through the MOA process, DCMA is required to focus on the surveillance management process at the sites. They can’t do the data analysis side, too, with limited resources, ensuring that the data coming out of the system is good.

 

Thursday, May 15, 2003

Call to Order & Introductions
Jim Gasbarro opened the meeting with introductions and indicated that currently NDIA has no “Clearing House” items, but would like the organization to address these in the future.

The Fall Conference is scheduled for November 16th through the 18th. The theme is “Project Performance-A Risky Business,” and will be Co-Chaired by David Treacy and Gary Troop (Government & Industry), assisted by Jim Gasbarro.

OSD Update, Debbie Tomsic, Program Analyst & EVM Focal Point
OSD EVM Activities include:

  • EVM data analysis is part of monthly DAES process
  • Collaboration with Program Office staffs
  • Participation on EVM system reviews and IBRs
  • Interface with NDIA, industry and associations
  • IBR Guide – Letter of approval is with Dr. Aldridge
  • Independent Study on the state of EVM in DoD
  • DoD EVM Integrated Product Team
  • EVM web site improvements
  • Streamlining performance reporting
  • OSD is putting more focus on EVM, and plans to stay engaged

NASA Update, Bill Simpson, Marshall Space Flight Center, NASA EVM Primary
Center – Earned Value & Project Management

  • EVM principals and practices – EVM is moving to Project Management
  • Major changes in the draft NASA Policy Directive (NPD)
  • EVM applicable to all NASA programs and projects following the management principles of the ANSI 748 standard
  • $70M and greater require full EVMS Criteria
  • $ 25-70M require modified EVM Criteria
  • EVM is a key program/project management process
  • It was suggested that contractor sharing of EV data, not reporting, per se, is a more effective management tool, and more timely
  • Initiative # 1- Provide uniform agency-wide IBR approach
  • Initiative #2 – EVM Implementation Assessment Model (to assess NASA) Provide a formal assessment process for determining the quality of EVM implementation on NASA programs and projects
  • Initiative #3 – Implement data analysis process and tools for better data analysis
  • Initiative #4 – Develop IAT infrastructure to support EVM activities, centralized database with agency-wide wInsight implementation
  • Initiative #5 – In-house EV System development to provide smaller projects a basic management tool
  • Initiative #6 – Schedule health assessment process implementation to provide a quantitative method to determine schedule credibility and usability for managing a project
  • Initiative #7 – Develop Schedule Best Practices Guide to enhance the value and consistency of schedule discipline and products
  • Senior Executive level support to champion EVM, Theron Bradley, Chief Engineer for NASA, has been identified
  • Enterprise coordination with key management representatives established

DOE Update, David Treacy, DOE EVMS Program Manager

  • Established MOA with DCMA in November 2002, to lead acceptance reviews
  • Program in planning
  • EVM applicability for all DOE acquisitions greater than $20 M
  • Plan to honor prior DCMA acceptance
  • Beginning to develop application guide
  • Industry/Government collaboration effort on application guidance desired

DCMA, Richard Zell
Headquarters activities update:

  • EVMS Validations – large increase in contractors who would like to be validated (and never have been) and an increased workload with the suppliers. This is a challenge to handle with limited resources.
  • Grand fathered Tri-Service Validations – a lot of questions and whether some of these should be re-done or re-visited. This is being reviewed.
  • Listing of validated companies – we do have a list, but not certain of the need or how to maintain this.
  • MOAs & MOUs – DOE, NASA & other agencies are becoming involved and more complex
  • DCMA is working with OSD and other agencies addressing training and consistency for EVM
  • DCMA is focusing on predictive analysis for awareness mode, projecting possibilities (e.g. percentage of possibility)
  • Advance Agreements with DCMA typically state these are in effect until one party elects to rescind the agreement. These are generally actively participating companies utilizing EVM on contracts and maintaining a current EV Management System. These may or may not be adequate for a specific Government agency contract.
  • Comment: DCMA is charged with the responsibility for validation on DoD contractors – these should continue to be valid, except for “cause”
  • Many DCM customers are questioning the validity of validations awarded thirty years prior
  • There’s an action for the Government to define the level of contractor review. There’s a lack of written guidance now. “Review for cause” should be what triggers a review of a previously validated contractor systems
  • Often requirements are written into specific contracts. The letter to Dr. Aldridge is industry conveying the contractor’s view of specific program office requiring unique program applications.
  • Comments: We have an action to bring our issues to NDIA to be dealt with. Someone needs to take an action to correct the problem, then evidence of action taken within DCMA purview
  • Writing of contract content with requirements conflicting with existing EV Management Systems is a problem. The DCMA One Book guidance should be a step in the solution to address the issues
  • NDIA “informal cross talk” for contractors/industry, is a potential opportunity for bringing these current issues to be worked on; develop a list and actions defined to be worked

CCDR – Jack Cloos, Institute of Defense Analyses

  • Mission – Become the comprehensive source of weapons systems cost information for DoD cost analysts
  • Current initiatives: Automation, Support, Analyses, Policy & Training
  • The goal is one CCRD electronic format by October 1, 2003
  • CCDR XML Standard will replace the CCDAR EDI Standard
  • Goal: Complete all CCDR documents this year
  • There’s still a “monitoring & compliance” issue; over a hundred programs are being tracked and over 90% are non-compliant, for a variety of reasons.
  • Training schedule for contractor preparation and Government Analyses
  • CCDR is required on all ACAT 1 programs; Greater than $50 M and high risk contracts between $7M & $50M

NRO –

  • Tonya Chickillo
    • Organization changes: Lt. Col. Messinger is retiring; his replacement is Lt. Col. Jim Gordon, who will join the organization June 1, 2003.
    • Quarterly business reviews began May ’03 with the Deputy Director of Engineering reviewing EV data and contract status charts. What’s EV telling us?
    • Current emphasis is on understanding the data and using it.
  • Barry Levy
    • Policy and Intelligence Community – Three principle agencies; NRO, NIMA and NSA have created an EVM Council/Group and are coming together to share resources for functions such as validations, surveillance and industry issues/needs.
    • EVM Directive 82-3 is being rewritten with guidelines
    • NRO Acquisition Manual has also had some revisions

International Update & Conferences – Wayne Abba

  • CPM is conducting its first offshore conference October 27th – 29th in Tokyo, Japan, with Government sponsorship. This is a major evolution.
  • Conferences: IPMC 2003 Tyson’s Corners, November 17th – 19th.
  • Practice Standard for Earned Value: ECD Spring 2004
  • The college of Scheduling has been established and will conduct its’ first conference in Montreal this year.

Working Group De-Briefs

Process Conformance – Pete Wynn, Lockheed Martin

  • Charter – Map company’s existing processes to the ANSI Standard
  • A White Paper was developed to capture “Best Practices” from the Program Management perspective (which will now be discarded.)
  • We’re now turning it around and looking at it from the ANSI Standard perspective – the intent of the standard – what would be a description of an output or artifact to satisfy the standard, and what is the value added
  • We’re looking for the “program management value” from each of the standards, and are requesting NDIA inputs as a beginning source of data to work with.
  • Estimated completion date of product, July 2003, next meeting

Surveillance – Joe Kusick for Bob Loop, Raytheon

  • A White Paper was developed containing the attributes of a good surveillance process, training, institutionalization of EVMS.
  • Next steps: Develop the White Paper into a guide
  • Where should the guide reside? The most appropriate location for the Surveillance Guide is a reference in the ANSI Standard, to meet the intent of the ANSI Standard
  • Surveillance Team discussion indicated it should be an addendum to the ANSI Standard, such that if the ANSI Standard is invoked the Surveillance Guide would automatically be included
  • Open group discussion indicated concern that if the guide is attached to the ANSI Standard, this implies a prescriptive approach, whereby new and potential EVMS users may misinterpret the intent of the Surveillance Guide, as imposing additional requirements, rather than as an aid to implement effective application assessment, working to improve EV processes
  • Surveillance, unlike an audit function, proactively tries to make recommendations for improvement across the company and suggestions for closure and/or solutions
  • The White Paper suggests that good internal surveillance reduces the need for Joint Surveillance
  • Comments
    • May make EVM appear too difficult to implement
    • What’s the cost impact of making it an addendum to the ANSI Standard? At least here you can control the cost.
    • How do you get on Contract? There’s an unfinished piece here: how the requirement goes on the contract. DFARs clause causes you to use the Standard making the Guide an attachment to the Standard puts it on contract.
  • The team requests NDIA participants to review the White Paper and provide inputs to the team by July 10th.
  • Jim Gasbarro will distribute an electronic copy of the White Paper to the NDIA participants
  • NDIA agreed to revisit where the Guide itself is ultimately located after it is completed
  • Recommendation: Sub-Committee report it out as a completed activity and request NDIA comments by 7/10 and we’ll disposition these
  • All review it for any major changes or omissions that must be incorporated, then NDIA debate on where it should go based on DFAR requirement - then the team can go write the guide.
  • August Meeting will include next discussion on Surveillance Guide and make a decision on where it will go

Risk Management Team – Gay Infanti

  • Mission Statement: engage the customer and contracts in identification, collection and sharing requirements and processes to integrate risk management with EVMS
  • Deliverable: define best practices and lessons learned, skills, guidelines and information
  • A risk survey was developed and distributed for NDIA feedback. The team will then refine the survey, prior to issuance of the test version for general feedback
  • It is perceived that there is value added integrating the RM and EVM processes
  • The end goal is to identify best practices. Phase two activity will be evaluate feedback, consolidate and share information (to improve processes) by November ‘03
  • The survey will be published and could then be tailored and utilized by companies for internal use
  • Risk Performance Index (similar to SPI and CPI) expand as an indicator for program status
  • Centers of Knowledge/Excellence identified:
    • DoD Risk Management Guide
    • Defense Acquisition University
    • PMCoP (addresses Risk and will address EVM)
    • Internet Council of Eng (INCOSE)
    • PMBOK
  • Open issues identified and in work
    • NDIA sponsoring survey to industry
    • Process for sharing best practices
    • NDIA – PMSS collaborate with DAU, PMCoP, PMI
    • Expansion of survey audience

Award Fee – Randy Steeno, Boeing Company
USAF AF Matrix Status - group led by Eleanor Haupt

  • Cost Metrics for Award Fee Evaluation; current metrics don’t motivate proper cost control
  • ASC team brainstormed on approach and conference attendees were requested to provide feed back
  • The team evaluated and develop a menu of metrics, incremental steps in performance for five grade levels
  • The approach was briefed to ASC workshop in September ‘02
  • Current Metrics used (SPI/CPI/VAC) pros and cons: these metrics are being done today and we want to get rid of them. New metrics are needed as a motivator:
    • Tie Award Fee to effective EV Management
    • Reward proactive innovative performance management mix of qualitative and quantitative measures
    • Menu of Recommended metrics:
      • Management – EVM is effectively integrated and used for project management
        • Management of major sub contractors
        • Accuracy and currency of cost and schedule forecasts
        • Adequacy of cost proposals submitted during award fee period
        • Cost control, variance analysis
      • Discipline
        • Accurately timeliness consistency of data and integration of sub contracts
        • Baseline discipline and system compliance
      • A matrix of ascending criteria was developed for assessing satisfactory, to good, to excellent
      • Recommendations to SPOs: tailor metrics and discuss with the contractor, provide feedback to the contractor
      • NDIA next actions – monitor the deployment of USAF Award Fee Metrics, encourage and enhance the menu of metrics

OTB/OTS Handbook Status – Randy Steeno, Boeing Company

  • The draft document developed in 2002, which was controversial, industry reviewed and commented via memo to Tony Finefield and Eleanor Haupt. Suggestions were dispositioned and the document was updated and posted on PMI Web Site, then distributed to NDIA for comments
  • Fall NDIA Conference OTB/OTS Workshop with Tony Finefield
  • 2003 – minor clean up/changes to document – no major changes
  • Suggested that NDIA and CPM collaborate on certain EVM documents
  • A Joint Industry/Government Document Working Group was suggested to review documents. The first such document is the OTB/OTS Handbook. Additional Industry participation is required. Company’s need to provide individuals to support this review effort and provide names to Randy Steeno.
  • Discussion:
    • Government approval is not required to do an OTB
    • OTB - It is a contractor owned process, but funding is tied to dollars. An EAC is tied to the OTB. The Government must review the EAC. It’s a replan that differs from contract, but process is the same
    • DiD now says “prior notification” – “Consultation” is in the guide
    • The OTB Handbook is completed, in the view of the team Eleanor Haupt and Tony Finefield
    • Essentially an OTB is not different from a Program EAC
    • Industry ownership of the OTB process is a Road map; steps to go through to complete an OTB
    • The need for the OTB Guide:
      • Describes areas where a “reset” is done; but the question is whether we need another guide to tell industry how to do this
      • Instructional material would help to do this; explain some of the challenges
      • There is confusion in Government and Industry on how a “Single Point Adjustment” is made
      • The System Description or implementing procedure should describe this process
  • NDIA could place this on Rick Zell’s list as a problem. We don’t have an authoritative forum (avenue) to resolve these issues. We need a mechanism to deal with this. Industry doesn’t have control of the document. There should be less regulation and more reliance on industry
  • Establish joint oversight Executive Organization to deal with issues such as this (e.g. contractors have the authority to do an OTB without Government intervention/involvement)
  • NDIA owes Eleanor and Tony feedback on the document. NDIA believes the content is contained elsewhere and not required (e.g. System Description and procedures)
  • Government Program Managers are putting in contracts the requirements for prior approval of OTBs
  • To help other companies who have never been involved, it would probably be helpful in “how to” do this
  • The only “approval” required now is the plan and approach, and the goal is to omit this, too
  • The OTB process is generally covered in the System Description in one page or less
  • It could be published as a “joint working opinion” for training – mechanics and roles as a educational process
  • The guide contains “Rules of Thumb for the Program Office; when an OTB should be considered, focused on management

Re-issuance of the DoD 5000.1 and 5000.2 series Documents, May 12, 2003

  • Randy Steeno provided then posted on Web Site 5/14/03
  • EVMS is contained in Enclosure 3
  • T – 1 is Regulatory
  • Moved up to T – 2 (for ACAT Contracts) EVMS reference OMB Circular A-11 Part 7
  • ANSI 748 and conduct IBRs
  • RDT&E > $73M
  • O&M > $315M
  • The CPR C/SSR reference is gone. What about reports?
  • Default to CDRL Clause for reports. There are some implicit requirements here. If 748 is on contract – what reporting is required? (-2R remains as guidance - $ thresholds are in conflict – the most current takes precedence)
  • We need to adapt to the philosophy and tailor to the flexibility. There’s an array of opportunities

OMB FAR Clause – Randy Steeno, group led discussion

  • Comments replace “ANSI-748 – A – 1998” with “ANSI-748 latest version”
  • See notes on OMB FAR copy mark-up

 

Friday, May 16, 2003 Panel Presentations/Discussion

For our closing day panel discussion, we were privileged to have distinguished participants from nine government offices provide us with timely summary updates of their Earned Value related initiatives.

These panelists were (in their order of presentation):

 

Lt Col Harold (Buck) Ennulat

USAF, SAF/ AQXR

CDR Lisa Festa

US Coast Guard Business Manager, Systems Integrator Program Office

Robert Buhrkuhl

Deputy Director, Acquisition Management, OUSD/ AT&L, Acquisition Resources & Analysis

Robert Schmitt

DCMA, Executive Director, Contract Management Operations

David Muzio

OMB, Office of Federal Procurement Policy

Michael Donnelly

DOE, Deputy Director, Project Management Office of Engineering & Construction Mgmt

William Simpson

NASA, Chief of the Project Analysis Office, Marshall Space Flight Center

Keith Kratzert

FAA, EVM Focal Point for the Office of Communications, Navigation, & Surveillance Systems

Peggy Poindexter

NIMA, Director of Acquisition Management

 

The panel composition, presentations, and subsequent discussions reflected the increased emphasis the various offices are placing on managing with EVM principles. The focus was on the OMB evaluations of programs in each agency in support of the President’s requirement to better manage resources across the board. The Business Case evaluations for new projects and the ROI evaluations for ongoing projects, the new form 300 requirement, the new FAR clause draft, and the OMB calling out of the ANSI Standard 748 for EVMS, and how the various offices and agencies are dealing with the requirements were discussion points for all the panelists.

The NDIA Program Management Systems Subcommittee and its officers give our sincere thanks to all the participants on this year’s panel. The information from each office was most timely and informative, and we all look forward to next year’s updates as each agency continues in its implementation of the EVM principles in managing project resources.

The meeting was adjourned at 12:30pm.

Integrated Program Management (IPMD)

; ;