Module 1
What are two problems with cost estimators?
(1) Lack of meta-estimating, i.e., cost estimators don't do BOEs, don't make a good case for the resources needed to produce good estimates, which results in make do with what you have; (2) There are some estimators who like to do everything but estimate
What are the seven steps of the cost estimating process?
1.Develop a Work Breakdown Structure (WBS) 2.Develop a program/system baseline 3.Collect data 4.Analyze data 5.Develop cost estimating methodology for each cost element 6.Put together and validate the cost model 7.Generate results and other required output, such as reports
What estimate is useful during the Pre-Systems Acquisition phase?
Analysis of Alternatives (AoA) or trade studies
The answer to a lot of cost estimating problems is in the
CARD
What is an element of an accurate cost estimate?
Cost Estimating Relationships (CERs)
What technique does an Estimate At Completion (EAC) rely on?
Earned Value Management (EVM)
What is a barrier to consistency?
Historical data are not always consistent from one data source to another, or even within the same data source over time. One source of inconsistency in historical data is the use of differing cost element structures or work breakdown structures to collect the data. There may be reasons to restrict access to both the data you need to create your cost estimate and the resulting estimate when you are done. Some data may be considered proprietary. Within the national security arena, data may be classified as Secret or Top Secret, with a whole host of related gradations. These precautions to prevent misuse of data may make estimating a slower process.
What does the Director of National Intelligence Cost Analysis Improvement Group (DNI CAIG) provide?
ICEs to the DNI for selected IC programs
How is cost estimating used, and what benefits does it provide?
It's a management tool used to help decision makers evaluate resource requirements at key milestones and decision points. It is not an end in itself, but is part of a total systems analysis process that includes programmatic, technical, and schedule analysis.
As a starting point, the WBS can be adapted from
MIL-STD-881C
Is there a difference between Work Breakdown Structure (WBS) and Cost Element Structure (CES)?
Not really, these terms can be interchangeable, although usually WBS relates to system/program from an engineering standpoint while CES is specifically related to the cost estimating process (if developing a CES, it should encompass all the elements of the WBS)
What happens if a program runs over budget?
Resources will need to be pulled from other programs to cover costs. Some programs that run far over budget will be canceled, leaving a gap in capabilities.
What does Title 10 USC Section 2434 state about "independent?"
Secretary of Defense may not approve the system development and demonstration, or the production and deployment, of a major defense acquisition program unless an independent estimate of the full life-cycle cost of the program and a manpower estimate for the program have been considered by the Secretary. The Secretary of Defense prescribes regulations governing the content and submission of the ICE. The regulations require that the independent estimate of the full life-cycle cost of a program be prepared by an office or other entity that is not under the supervision, direction, or control of the military department, Defense Agency, or other component of the Department of Defense that is directly responsible for carrying out the development or acquisition of the program. The law states that ICE should be ―prepared or approved by the Director of Cost Assessment and Program Evaluation; and include all costs of development, procurement, military construction, and operations and support, without regard to funding source or management control. Additionally, the manpower estimate must ―include an estimate of the total number of personnel required to operate, maintain, and support the program upon full operational deployment; and train personnel to carry out‖ those activities.
What kind of organizations perform or use cost estimates?
Special purpose organizations exist within the government and industry to perform independent cost estimates (ICEs) and validate or adjust the estimates of other organizations. These are known as cost oversight organizations. There are also other organizations in government that oversee for fiscal responsibility at a broader level, including the Office of Management and Budget (OMB), the Government Accountability Office (GAO), and the United States Congress.
Why do the cost elements included in the estimate tend to expand at each level?
The contractor's estimate only includes contractor cost (including subcontractors), only on the current contract, and only as currently written. The government program office must also take into account government costs, future contracts awarded to this or another prime contractor, and potential modifications to the current contract.
What is cost estimating, and why do we do it?
The process of collecting and analyzing historical data and applying quantitative models, techniques, tools, and databases to predict the future cost of an item, product, program or task; the process of predicting the future based on today's knowledge to help facilitate the successful completion a program, project, or process.
What is the purpose of cost estimating?
To translate system/functional requirements associated with programs, projects, proposals, or processes into budget requirements. It determines and communicates a realistic view of the likely cost outcome of a system or program. This outcome can form the basis of the plan for executing the work to develop, field, and support the system or program.
What is an element of a credible cost estimate?
WBS structure is aligned to organizational structure performing the work
What is the difference between an Economic Analysis, Business Case Analysis, and the Analysis of Alternatives?
Whereas an EA is more of a business case analysis, or BCA, where benefits can be quantified monetarily, an AoA (formerly known as Cost and Operational Effectiveness Assessment, or COEA within DoD) usually involves performance characteristics as well
Why is risk an important factor in the three reasons for cost estimating?
Without risk, budgets are likely to be unrealistically low, resulting in immediate shortfalls; baselines are likely to be unrealistically aggressive, resulting in immediate performance deficiencies; and trade-offs are likely to be unrealistically optimistic, resulting in faulty decisions to pursue ultimately unaffordable options.
What is an element of a comprehensive cost estimate?
Work Breakdown Structure (WBS)
What estimate is useful during During Systems Acquisition?
a Source Selection process (costing competitors' approaches to satisfying requirements)
After the system is developed, what can then be performed before full-scale production?
a cost study, sometimes multiple ICEs will be performed for budgeting purposes or program cost tracking
What is a Life Cycle Cost Estimate, or LCCE?
a cradle-to-grave estimate that includes research and development (R&D), production, operations and support (O&S), and disposal costs (can also be referred to as a Total Ownership Cost (TOC) estimate)
What should be included in the program baseline?
a description of the aircraft, any other similar aircraft, the WBS, and the other details commonly included in a CARD
What is the Cost Analysis Requirements Description, or CARD?
a detailed technical and schedule description of a program/system to be estimated.
What is a parametric estimate?
a mathematical relationship based on historical data that relates cost to one or more technical, performance, cost, or programmatic parameters
What is Cost As an Independent Variable (CAIV)?
a strategy that entails setting aggressive, yet realistic cost objectives when defining operational requirements and acquiring systems and managing achievement of these objectives
What is a cost estimate?
a summary of the cost model, comprising the analysis of individual cost elements using established methodologies to project from data and experience to estimated costs of a program or system, together with the corresponding documentation
What was a significant change authorized by the Clinger-Cohen Act?
a test of the use of the Simplified Acquisition Procedures (SAP) for commercial items between the simplified acquisition threshold of $100,000 and $5 million
What shows the success of estimating to support the budgeting process?
a track record of how long each ICE-based budget proves adequate on a year-by-year and cumulative basis
What are the characteristics of high quality cost estimates?
accuracy, comprehensiveness, replicability/auditability, traceability, credibility, and timeliness
What do cost estimators fail to do?
adequately track their own actuals and write good basis of estimate (BOE) justification for the effort needed to do a good estimate
What did the Simplified Acquisition Procedures (SAP) do?
allow DoD to reduce its administrative costs and the overhead costs for DoD's vendor base for purchases of relatively low risk items, eliminating government-unique requirements previously cited by industry as a barrier to doing business with DoD
What is an Independent Cost Estimate, or ICE?
an LCCE usually developed by an analyst outside the program or project office that tests the reasonableness of the program or project estimate, often employs an estimating methodology different from the one used to develop the program or project estimate, provides an unbiased test of the reasonableness of the Life Cycle Cost Estimate (LCCE)
If a system involves a key piece of state-of-the-art equipment which has never been built before, which of the Big Four is used?
an analogy would be applied in the cost estimates when historical cost data exist for one or more items that are similar to those proposed
What is a budget estimate?
an estimate developed to be used strictly for budgeting purposes in support of an acquisition program - in other words, to get authorization for funding
What is a Rough-Order-of-Magnitude, or ROM estimate?
an estimate developed when very little specific information is known about the project (i.e., detailed schedule, resources, LOE, etc.), design effort for the program or system has not begun, and the estimate is often based on ratios or factored historical information (Quick-Look or Back-of-the-Envelope estimate)
What is an Estimate At Completion, or EAC?
an estimate used for in-process programs and for ICEs when there is sufficient cost history and typically have more program specific data available such as schedule, current and projected resources, and LOE
What is an Independent Cost Assessment, or ICA?
an independent estimate with risk that started with someone else's estimate (usually the contractor's or the program office's) and made adjustments to methodology (i.e. ground rules, assumptions, etc.) based on a more objective view of the program, may also be limited to a validation of the ground rules, methodology, and/or assumptions used in an estimate, include the full spectrum from a full-blown ICE to a cursory sufficiency review.
What are the Big Four?
analogy, parametric, build-up, and extrapolation from actuals
What is DoD Directive 5000.01 used for?
applies to efforts to provide new, improved, or continuing weapon systems and information systems based on validated needs, does not address purchasing, unless a part of the program, non-operational supplies or services
What is the DoD 5000.04-M-1 used for?
as the primary requirements document for the development, implementation, and operation of the contractor cost data reporting (CCDR) and software resources data reporting (SRDR) systems, collectively referred to as the CSDR system, providing guidance and detailed requirements for implementing the mandatory CCDR and SRDR policies and prescribes procedures and instructions that DoD stakeholders in the CCDR and SRDR processes must follow to ensure that the data reported through the CSDR system is accurate and consistent and is quickly made available to DoD cost and software estimators.
What do accurate estimates do?
assess the reasonableness of contractor proposals and program budgets and help the program office effectively defend budgets to oversight organizations such as OSD or ODNI, OMB, and even to Congress; they also assist in quickly and accurately determining the impacts of budget cuts on program baselines and the program/system functionality
What are two big tips for cost estimators?
avoid leaving out or double-counting components, and don't forget about integration costs
The cost analysis process cannot
be applied with cookbook precision (it must be tailored to the situation/program); produce results that are better than input data; predict political and non-cost impacts; substitute for sound judgment, management, or control; or make the final decisions (cost estimates only support decision-making)
Why is the Estimate At Completion (EAC) more important in recent years?
because of the need to certify programs to satisfy the requirements of Nunn-McCurdy and WSARA 2009
How should the Defense Acquisition Guidebook be used?
best practices, lessons learned, and expectations, until replaced.
What does DoD Instruction 5000.02 and DoD 5000.4-M mandate?
both a program office estimate (POE) and a DoD component cost analysis (CCA) be prepared in support of acquisition milestone reviews
What are the three primary reasons for estimating the cost of a program?
budgeting, planning, & trade-offs
What are types of data normalization?
by cost units; by sizing units; by content, such recurring/non-recurring costs, and adjusting end items for homogeneity; by state of development; and by special groupings such as mission application or platform operating environment
How should overall progress toward cost goals be measured?
by periodic life cycle cost estimates (LCCEs)
What can a Work Breakdown Structure (WBS) also do?
can also be used to interrelate the schedule and costs, separates elements of the program/system into component parts, making the relationships of the parts clear and the relationship of the tasks to be completed - to each other and to the end product - clear
What should a data collection plan discuss?
capturing primary or secondary cost, technical, and programmatic data and ensure every cost element is covered, encompassing the full scope of the estimate, primary and backup data sources should be identified and attempts should be made to identify data sources for cross-checks, and a projected data collection timeline
What is analogy?
compares the cost of an item to be estimated to a similar item.
Accurate cost estimates can help management avoid
cost growth and schedule slips
What will a high-level CER already capture?
costs in a detailed build-up at a lower level
What is an element of a traceable cost estimate?
data are traceable back to the source documentation,
What is an Economic Analysis, or EA?
describes a specific mission requirement and lists specific alternative courses of action that will satisfy the requirement, a framework for systematically investigating problems of choice, examines and compares the costs and benefits associated with each alternative course of action.
Cost estimating should capture the cost impacts of both
design decisions and business decisions
What Technology Readiness Levels, or TRLs?
developed in the 1980s by NASA, used to assess the maturity of technologies (there are nine TRLs)
What are the five steps in the data collection process?
developing an understanding of the total picture of the estimating task; establishing a Cost Element Structure and boundaries around the estimate; understanding the estimating technique to be employed and its associated data collection needs; developing a data collection plan; and finally collecting the data.
When should a WBS be developed?
early in the conceptual stages of the program to provide a structure to the design requirements and functional architecture of the system/program before the physical architecture is able to be defined (it is not until later phases that the system is described in terms of its specifications)
How do the requirements for a program change?
either due to external pressures from customers or resource sponsors or due to internal systems engineering decisions
What is the Chief Information Officer's job?
ensures that the mandates of the Clinger-Cohen Act are implemented to include ensuring that IT investments: • Support core mission functions, be undertaken because no alternative private sector or other government source can effectively support the function, and support work processes that have been redesigned or otherwise improved; • Are consistent with the Agency's architecture that integrates work processes and information flows with technology to achieve the Agency's mission and strategic plan; • Reflect a portfolio management approach where decisions on whether to invest in IT are based on potential return, and decisions to terminate or make additional investments are based on performance - much like an investment broker is measured and rewarded based on managing risk and achieving results; and • Reduce risk and enhance manageability by discouraging ―grand‖ information system projects, and encouraging incremental, phased approaches
What is a Work Breakdown Structure, or WBS?
establishes a common frame of reference for relating job tasks to each other and relating project costs at the summary level of detail, provides a consistent and visible framework for specifying the objectives, labor, materials, and contracts of the system/program, the structure is used to define the total program/system by providing detailed definitions of individual elements required (via a WBS dictionary)
What is DoD Instruction 5000.02 used for?
establishes a simplified and flexible management framework for translating capability needs and technology opportunities, based on approved capability needs, into stable, affordable, and well-managed acquisition programs that include weapon systems, services, and automated information systems (AISs) and authorizes Milestone Decision Authorities (MDAs) to tailor the regulatory information requirements and acquisition process procedures in this Instruction to achieve cost, schedule, and performance goals.
What is the DoD Instruction 7041.3 used for?
establishes policy and responsibility for conducting cost-effectiveness economic analyses of investment alternatives and authorizes the Defense Economic Analysis Council and its charter
What does the CJCSI 3170.01H do?
establishes the Joint Capabilities Integration and Development System (JCIDS) as the process used by the Joint Requirements Oversight Council (JROC) to fulfill its advisory responsibilities to the Chairman of the Joint Chiefs of Staff in identifying, assessing, validating, and prioritizing joint military capability requirements
What is the Analysis of Alternatives, or AoA?
evaluates the costs and benefits (i.e., operational effectiveness or utility) of different courses of action to meet recognized needs, aid and document decision-making by showing the advantages and disadvantages of the alternatives being considered.
What is a Rough-Order-of-Magnitude (ROM) estimate used for?
feasibility studies and selection from among alternatives
What is an element of a replicable and auditable cost estimate?
fully traceable to the system specification, thoroughly documented
What did USC Title 50, Section 415a-1, originally part of the Intelligence Reform and Terrorism Prevention Act of 2004 (IRTPA), do?
gave the Director of National Intelligence (DNI) the mandate to oversee the development and implementation of a program management plan for acquisition of major systems
How does the analyst understand data needs?
get an idea of the estimating technique(s) that will be employed in the estimate, identify potential data sources for each component of the estimate, develop and execute a data collection plan, and collect data
The quality of an estimate is reliant on both
good data (input) and good documentation (output)
What is the function of the system description in a CARD?
helps observers understand what the system does, how it works, program risks, and any operating parameters
Who takes a broader view of risk and uncertainty?
higher organizations, at the service or department level, because inflation can have a major impact on the affordability of the portfolio, for which they must budget in then year dollars
Cost estimator's estimate must include
historical experience
What is sensitivity analysis?
identifies key assumptions and variables within the estimate and determines the cost of changing significant input parameters
How should normalization be done?
in an objective and reasonable manner, and not to make the data fit a desirable outcome
What is the master WBS for the life cycle of the program?
includes each phase WBS, as early on as possible
What is the programmatic scope?
indicates the window in the life of the program covered by the estimate, whether it includes the entire life cycle or a specific period, phases (research and development, production, operations and support, disposal) covered by the estimate should also be determined
A cost estimate must be
internally consistent throughout a program's lifespan, with a good track to previous estimates, consistent in content and assumptions with estimates of the program developed by other organizations if these estimates are to be compared
What is included in the DoD 7000.14?
investment-expense criteria
What is engineering build-up?
involves estimating costs at the lowest definable level, frequently making use of industrial engineering techniques such as time standards
What is a WBS dictionary?
lists and defines the work breakdown structure elements, based on the generic definitions in MIL-STD-881, made program specific to define the products being acquired
What should logistics support include?
maintenance, sparing, and upgrade plans, and any ancillary costs
Who determines technical/performance, schedule and risk?
management and/or engineers
Under the description of environmental impacts, what should also be included?
mitigation plans; disposal concept; environmentally acceptable alternatives; environmental conditions expected to be encountered during development, production, transportation, storage, and operation of the subsystem; hazardous, toxic, or radiological materials that may be encountered or generated during the subsystem's development, manufacture, transportation, storage, operation, and disposal, the quantities of each hazardous material used or generated over the subsystem's lifetime should be estimated based on the most current operations and maintenance concepts
A cost estimate serves as an important basis for a
new business proposal or program's budget
What does the Cost Analysis Requirements Description (CARD), or similar documentation, do?
organizes the physical work packages into levels that can be rolled up into control accounts
Once budgets are established, what must occur to lay the foundation for a successful project execution?
planning
What should operational concept include?
program management details, operation specifics, and platforms
What should a budget estimate be able to ensure?
project feasibility and attainable performance levels
Estimating supports the development of
proposal basis of estimate (BOE) documentation, which in turn serves to help establish the integrated baseline
Why document the estimate?
provides any reviewers with: •The purpose of the cost estimate, which states the reason for developing the estimate; whether it is an initial or updated prior estimate; if an update, identifies the prior estimate; the specific tasking and relevant correspondence; and cost estimating team composition. •The scope of the cost estimate in both time and space, describing phases (e.g., development, production, operations and support), specific time periods, and systems and processes encompassed by the estimate, and specific areas not addressed by the estimate and reasons why. •The ground rules and assumptions, which list all technical and programmatic conditions that formed the basis for the estimate to assist management in understanding these conditions. When conditions are dictated to the estimating team, they become the ground rules by which the estimate will be conducted. In the absence of a firm ground rule, the estimator has the privilege of establishing assumptions which fill this void and allow the estimate to proceed. In exercising this privilege, the estimator must ensure that assumptions are not arbitrary but rather are founded on expert judgments rendered by experienced program and technical personnel. One way to elicit these judgments (or better yet, the data behind them!) is to purposely choose a high, conservative, and/or painful number to force a better answer from the cognizant expert, sometimes known as ―taking a hostage. •The program background and system description, which characterizes significant program and system aspects and status in terms of work accomplished to date, current position, work remaining, detailed technical and programmatic descriptions, major components, performance parameters, support concepts, contract types, acquisition strategy, and any other information that will assist the document user in fully understanding the system (see earlier CARD description for more detail). •The program schedule, which should include the master schedule for development, production and deployment, as well as a detailed delivery schedule. An independent schedule assessment is desirable. •The cost estimating methodology should be described in sufficient detail to permit reviewers to follow the logic from assumptions to conclusion and to replicate or update the estimate at a later time. Specifically, Basis of Estimate (BOE) documentation for proposals must support government reviews and audits. The estimate documentation details the primary data, techniques, and rationale for each element of the cost estimate.
What is an element of a timely cost estimate?
provides decision makers the valuable insight they need in time and not overcome by events (OBE)
What should documentation accomplish?
provides sufficient information on how the estimate was developed so that other analysts could reproduce the estimate
What are key characteristics or components of cost models?
rates for labor, materials, tooling, scrap, etc., need to be provided, and audited or negotiated direct and indirect rates are usually applied
What are the Technology Readiness Levels (TRLs)?
relate to the technological maturity
How is the estimate defended?
reports and documentation
What did the Clinger-Cohen Act of 1996 require?
requires that major Federal Agencies establish the position of Chief Information Officer, or CIO, having clear authority, responsibility, and accountability for the Agency's information resources management activities, and providing for greater coordination.
What should you do with the data?
scatter plot, calculate descriptive statistics for the variables of interest (presumably particular cost elements, at a minimum), check data for outliers, compare your descriptive statistics to standard factors, rules of thumb, or other historical data wherever possible
In management of a program, it is important to identify and manage
schedule and technical risks
What three key parameters does a program have?
schedule, performance, and cost
What is the technical scope?
should include all requirements for the program, including a detailed description of work to be performed, work not included in the scope, deliverables, any constraints or special conditions, sequence of events and any interdependencies, and milestones, includes key performance characteristics, hardware/software components, architecture, configurations, reliability and security requirements, etc., documents the impact to the Project Baseline of any applicable contractual changes.
What does the WBS dictionary do?
shows the hierarchical relationship of the elements and describes each work breakdown structure element and the resources and processes required to produce it
What should deployment details include?
sites, schedules, special preparations, site configurations, and transition plans
What should manpower requirements include?
skills, salaries, and location
What kind of knowledge, skills, and competencies are needed for cost estimating and analysis?
sophisticated probability and statistics, application of economic ideas like inflation, and an understanding of economics, econometrics, business, and accounting, the budgeting process, a familiarity with ideas from operations research (OR), the management sciences, and industrial engineering (IE), mechanical engineering, systems engineering, physics, software development, even logistics knowledge, and salesmanship
What should be included in documentation?
specific methodologies, assumptions, resources and data points for the estimate
What can be a valuable estimating tool?
staffing profiles
What did the Clinger-Cohen Act of 1996, which rescinded the Brooks Act, do?
streamline information technology, or IT, acquisitions and minimize layered approvals, eliminating the delegation of procurement authority at the General Services Administration (GSA)
What are Analysis of Alternatives (AoA)?
studies comparing technical, cost, and performance characteristics of multiple approaches and are used to select among alternatives before committing to a particular project
What should analogous systems include?
technical details, reason for replacing, legacy hardware/software, and phase-out plan
What should the system test and evaluation plan include?
test schedule, number of tests, criteria, locations, subsystem tests, and subcontractor test review
What does the Air Force Cost Directorate, SAF/FMCC, prepare after reconciliation between the System Program Office estimate and the CCA prepared by the Air Force Cost Analysis Agency (AFCAA)?
the AF cost position
What does the Army Cost Review Board with the Assistant Secretary of the Army (Financial Management) develop?
the Army cost position briefed to the CAPE
What is the program/system baseline definition essentially synonymous with?
the CARD or CARD-like document
What does the Office of the Deputy Assistant Secretary of the Army for Cost and Economics (ODASA-CE) prepare?
the CCA and briefs the results to the CAPE
What and who does the Naval Center for Cost Analysis (NCCA) prepare and brief to?
the Component Cost Analysis (CCA) to the CAPE
What typically also serves as the baseline for which a program is funded?
the Independent Cost Estimate (ICE)
What does the ODNI CAIG develop?
the Independent Cost Estimate (ICE) based on the Intelligence Community Baseline Document (ICBD), which is developed by the program office
What did the Intelligence Reform and Terrorism Prevention Act of 2004 (IRTPA) establish?
the Office of the Director of National Intelligence (ODNI), provided guidance on the oversight function that the DNI would provide to the Intelligence Community (IC), included in that oversight is the ODNI CAIG, which serves as the independent assessors for the IC
Within the DoD budgeting process, what estimates are required for Program Objective Memorandum (POM) submission?
the Programming, Planning, Budgeting, and Execution System, or PPBES
What is a competitive advantage?
the best value, balancing performance and cost or a successful mission at the best value to the taxpayer
What is a POM?
the budget covering the remainder of the Future Years Defense Program (FYDP), FYXX+2 through XX+6
What is a cost model?
the collection of cost estimating methodologies associated with a cost element structure that best reflects the cost of a future system given three important considerations: (1) the level of detail of technical/programmatic definition of the future system; (2) the level of detail of relevant existing cost estimating methodologies; and (3) the level of detail of historical data available to develop new cost estimating methodologies.
Cost analysis reduces the dependence on
the development contractor estimate of program costs and protects the government from low-ball estimates
What is the Cost and Software Data Reporting (CSDR) Plan?
the framework for reporting costs and software counts to the government
What is the rule of thumb for briefing?
the higher up the chain a briefing is presented there is less estimate detail and more organizational impact (depending on the level of the decision-maker, the detail of the charts depends on the level of the decision-maker being briefed)
What does the program/system baseline definition make apparent?
the intended use of the estimate and the scope of the estimate
What provides specific insights into the relationship between cost, schedule, and performance?
the linkage between the specification requirements, the WBS, the statement of work, and the master and detailed schedules
What is the OSD Cost Assessment and Program Evaluation (CAPE)?
the milestone review authority for major DoD programs and has oversight of the DoD service cost centers (i.e., ODASA-CE, NCCA, AFCAA)
Who develops a WBS dictionary?
the program manager
How does the Work Breakdown Structure (WBS) evolve?
through iterative analysis of the program objectives, functional design criteria, program scope, technical performance requirements, proposed methods of performance (including acquisition strategy, drawings, process flow charts), and other technical documentation
What is the purpose of the CARD?
to collect, integrate, and coordinate all the technical, programmatic, and schedule information necessary to estimate the costs of a program
What is the purpose of a Work Breakdown Structure (WBS)?
to display and define the system/program to be developed or produced in a product-oriented family tree composed of hardware, software, services, data, and facilities that describe the program components at the level necessary to capture all elements of cost, and relates the elements of work to each other and the end product, defines the program in terms of hierarchical, product-oriented elements, and each element provides logical summary points for assessing technical accomplishments and for measuring cost and schedule performance, encompasses the system-level activities such as system engineering, specific builds, integration, fielding, and support of the system throughout its life cycle until it is removed from the inventory
What are the purposes of a budget estimate?
to ensure project feasibility and attainable performance levels, to develop a reliable project cost estimate consistent with realistic schedules, and to use it to establish baseline project definitions, schedules, and costs
Why do government program offices and industry project teams use cost estimates?
to justify obtaining resources from external sources and to help allocate resources internally
Why do budgeting, contracting, and purchasing organizations use cost analysis?
to make sure the cost/price of goods and services being budgeted for, contracted for, or purchased is reasonable and consistent, and cost estimating is often used to support the negotiation process
Why did the DoD develop the MIL-STD-881?
to provide the framework and instructions for developing a product-oriented WBS
What's the use of the CARD?
to understand the full scope of the program
What should the training plan include?
types of training/certification, providers, locations, and frequency
What do trade-off studies do?
use a baseline cost estimate and explore the relative cost of changing requirements; the end goal is to achieve the proper balance of maximizing utility and minimizing cost
What is risk analysis?
used to adjust cost estimates, normally increasing the cost estimate—risk is added to initial point estimate resulting in a range or distribution of costs
What is extrapolation from actuals?
uses of data from prototypes or complete or incomplete units to project the cost of future units
How should program budgets be built?
using an independent cost estimate (ICE), not only to give that program a chance for success, but also because valuable agency resources need to be used as effectively as possible
What is expert opinion?
utilizes the subjective judgment of Subject Matter Experts (SMEs) to corroborate or adjust cost estimates
What do the results of an Analysis of Alternatives (AoA) study help determine?
what design approach to take and help quantify the cost impact
What should be documented?
•All data used, the data source, and normalization procedures (see Module 4 Data Collection and Normalization). •The basis, development, and/or source of all cost estimating relationships (CERs) and factors used, to include a description of how they were applied, their relevance to the program being estimated, and regression statistics (see Module 3 Parametric Estimating). •How the labor hours were developed, and direct and indirect labor rates as industry averages or organization-specific, their content, and how they were developed. •Methodology used to develop cost improvement curves (if applicable) and description of the curve in terms of slope, source, and relevance to the cost elements and program being estimated. Any unique aspects of curve application should also be included. •The material, purchased parts, and subcontracted items that are required and the development of their costs (e.g., vendor quotes, catalog prices, etc.). The cited guides may be useful references for technical writing, which is often not a cost estimator's favorite pastime.
In addition to the data and methodology described on the previous slide, the estimate documentation should include:
•All pre-existing cost models used and their relevance to the estimate, along with complete details regarding any parametric inputs and outputs and any calibration performed to ensure the model is an appropriate estimating tool for the cost element and program; •The specific inflation indices and computations used and the approach used to time-phase the estimate; •The logic and rationale that led to specific conclusions reached by the analyst regarding various aspects of the estimate (so called expert judgments); •The details of all risk analysis conducted and how it formed the basis for reaching conclusions regarding estimate confidence; •In the case of an ICE, a conclusion expressing the cost estimating team's determination regarding the reasonableness of the program office estimate; and finally, •Appendices containing any other information pertinent to the cost estimate, and a reference list. In our world of DBA (death by acronyms!), an Acronym List is always a good idea.
The CARD features programmatic information such as program plan and purpose, and mandates or directives governing the program, and also includes:
•Detailed system technical description, key performance characteristics, replaced system, entity responsible for the system, system user, descriptions of hardware and software components including interactions, technical protocols or standards used, system architecture and equipment configurations, key performance parameters (KPPs), operational profile, reliability analysis, security requirements, and test and evaluation concepts/plans; •Work Breakdown Structure (WBS) being used, determination of sufficiency of the WBS for cost estimating purposes, and whether the WBS accurately reflects the system baseline; •A detailed description of predecessor systems, if any, reasons for replacing legacy system, detailed description of legacy system hardware and software components, technical protocols or standards used, KPPs, maintenance/logistics description, training description, and legacy phase out plan; •Manpower requirements associated with system, how these compare to any legacy system, grade or salary levels, skill level requirements, and location of personnel; •Program manager's assessment of technical/performance, schedule, and cost risk; •Operational Concept, including program management details, how and where the system will be operated, and platforms it will be installed on; •Deployment details including platform/site deployment schedule, special preparations required, user preparations/training, rollout plan/FOC or IOC schedule, standard platform/site configurations, and transition plan between legacy and current system; •Logistics Support details such as maintenance and sparing plan, and handling of upgrades; ... [...continued on next slide...] •Training plan, types of user training or certification, types of training or certification needed for system maintenance or administration personnel, training provider and location, seat requirements, and frequency of training; •Cost and Software Data Reporting (CSDR) plan for completion; •Acquisition Schedule and milestones; •Acquisition Strategy including stated objectives of the program, any secondary program goals, details of vendor support, and contracting details; •System Test and Evaluation plan, number of required tests, test schedule, testing entry/exit criteria, test bed locations and configuration, contingency plans; •Environmental Impacts, if any, mitigation plan, and disposal concept; and finally, •Changes from the prior CARD version defined and summarized including change pages with track changes and margin bars. The reader should be able to easily understand the differences without having to compare page by page.