Module 4
How many steps in (Tecolote) Cost Estimating Process
15 steps
What is the scope and purpose of a Business Case Analysis?
A business case analysis, sometimes referred to as a cost benefit analysis, is a comparative analysis that presents facts and supporting details among competing alternatives. A BCA considers not only all the life cycle costs that an LCCE identifies but also quantifiable and non-quantifiable benefits. It should be unbiased by considering all possible alternatives and should not be developed solely for supporting a predetermined solution. A BCA seeks to find the best value solution by linking each alternative to how it satisfies a strategic objective. Each alternative should identify the • relative life-cycle costs and benefits • methods and rationale for quantifying the life-cycle costs and benefits • effect and value of cost, schedule, and performance tradeoffs • sensitivity to changes in assumptions • risk factors On the basis of this information, the BCA then recommends the best alternative.
1. Why is it important to obtain Management Approval of a cost estimate?
A cost estimate is not considered valid until management has approved it. Since many cost estimates are developed to support a budget request or make a decision between competing alternatives, it is vital that management is briefed on how the estimate was developed, including risks associated with the underlying data and methods. Therefore, the cost estimator should prepare a briefing for management with enough detail to easily defend the estimate by showing how it is accurate, complete, and high in quality. The briefing should present the documented LCCE with an explanation of the program's technical and program baseline.
Factors (CERs)
A factor uses the cost of another element to estimate a new cost using a multiplier. Since a factor is defined to be cost as a function of another cost, it is often expressed as a percentage. For example, travel costs may be estimated as 5 percent of program management costs.
Rates (CERs)
A rate uses a parameter to predict cost, using a multiplicative relationship. Since rate is defined to be cost as a function of a parameter, the units for rate are always dollars per something. The rate most commonly used in cost estimating is the labor rate, expressed in dollars per hour.
Ratios (CERs)
A ratio is a function of another parameter and is often used to estimate effort. For example, the cost to build a component could be based on the industry standard of 20 hours per subcomponent.
What is the purpose of the Analysis of Alternatives?
AOA compares the operational effectiveness, suitability, and LCCE of alternatives that might satisfy established capability needs. AOAs identify several conceptual alternatives; analysis and conclusions are typically used to justify initiating an acquisition program. Both the AOA and CEA address each alternative's advantages, disadvantages, associated risks, and uncertainties and how they might influence the comparison.
Define affordability as it applies to acquisition programs
Affordability is the degree to which an acquisition program's funding requirements fit within the agency's overall portfolio plan. Decision makers should consider affordability at each decision point in a program's life cycle. Affordability analysis validates that the program's acquisition strategy has an adequate budget for its planned resources. The affordability assessment is highly reliant upon a credible, quality cost estimate.
Time-Phase Estimate
After the point estimate (with risk dollars) has been developed, it must be time-phased by allocating costs to specific Government Fiscal Years to determine budget requirements (as expressed in Then-Year dollars). The estimate is spread over the life of the program according to the program schedule, acquisition strategy, budget negotiations, and most importantly, according to the rules for the appropriations from which it is funded.
Why are estimates time phased?
After the point estimate (with risk dollars) has been developed, it must be time-phased by allocating costs to specific Government fiscal years to determine budget requirements (expressed in Then-Year dollars). Time phasing is an integral part of financial management because the yearly funding profile of a program is critical since it supports program execution and budget requests. The estimate is spread over the life of the program according to the program schedule, acquisition strategy and most importantly, according to the rules for the appropriations from which it is funded.
Differentiate between cost estimating and cost analysis - Cost Estimating
Although "cost estimating" and "cost analysis" are often used interchangeably, cost estimating is a specific activity within cost analysis. Cost estimating involves collecting and analyzing historical data and applying quantitative models, techniques, tools, and databases to predict a program's future cost. More simply, cost estimating combines science and art to predict the future cost of something based on known historical data that are adjusted to reflect new materials, technology, software languages, and development teams.
What is the purpose of the Independent Cost Estimate/Assessment?
An ICE, conducted by an organization independent of the acquisition chain of command, is based on the same technical and procurement information used to make the baseline estimate. ICEs are used primarily to validate program LCCEs and are typically reconciled with them. Because the team performing the ICE is independent, it provides an unbiased test of whether the program office cost estimate is reasonable.
Analogy Methodology
An analogy uses the cost of a similar program to estimate the new program and adjusts for differences. Analogies assume that no new program, no matter how technologically state of the art it may be, represents a totally new system. Most new programs evolve from programs already fielded that have had new features added on or that simply represent a new combination of existing components. The analogy method uses this concept for estimating new components, subsystems, or total programs. That is, an analogy uses actual costs from a similar program with adjustments to account for differences between the requirements of the existing and new systems. A cost estimator typically uses this method early in a program's life cycle, when insufficient actual cost data are available but the technical and program definition is good enough to make the necessary adjustments.
Assumptions - technical maturity & technical readiness
Another area in which assumptions tend to be optimistic is technology maturity. Programs that rely on mature technologies are in a better position to succeed than those that do not. Technologies that are not fully developed represent a significant challenge and add a high degree of risk to a program's schedule and cost. Programs typically assume that the technology required will arrive on schedule and be available to support the effort. While this assumption allows the program to continue, the risk, if inaccurate, can greatly affect cost and schedule.
What is a sensitivity analysis?
As a best practice, sensitivity analysis should be performed when building cost estimates because it examines the effects of changing assumptions, ground rules or key technical input parameters. Sensitivity analysis allows program managers to determine how sensitive a program is to changes in key assumptions concerning the technical baseline and implementation approach, and to make decisions based on that information. Uncertainty about the values of some, if not most, of the technical parameters is common early in a program's design and development. Many assumptions made at the start of a program turn out to be inaccurate. Therefore, once the point estimate has been developed, it is important to determine how sensitive the total cost estimate is to changes in the cost drivers. Typically performed on high-cost elements, sensitivity analysis examines how the cost estimate is affected by a change in a cost driver's value. If the effort to reuse the software is 50% of the effort to build it new, how does the overall cost estimate change versus an assumption where it is 25% of the effort to build it new?
What is the purpose of delineating and vetting Cost Estimate assumptions?
Assumptions represent a set of judgments about past, present, or future conditions postulated as true in the absence of positive proof. The analyst should ensure that assumptions are not arbitrary, that they are founded on expert judgments rendered by experienced program and technical personnel. Many assumptions profoundly influence estimates of cost; the subsequent rejection of even a single assumption by management could invalidate many aspects of the estimate. Therefore, it is imperative that cost estimator's brief management and document all assumptions well, so that management fully understands the conditions the estimate was structured on. Failing to do so can lead to overly optimistic assumptions that heavily influence the overall cost estimate, to cost overruns, and to totally inaccurate estimates and budgets.
Why is it important to automate an estimate?
Automation provides for a systematic and standardized approach for developing and running cost estimates. Not only does it provide a repeatable and stable environment for consistently developing cost estimates, it also includes user-friendly tools that speed up the cost model development process.
Common issues in data collection
Common issues in data collection include inconsistent data definitions in historical programs compared to the new program. Understanding what the historical data included is vital to data reliability. For example, was a new manufacturing process implemented that was supposed to generate savings but resulted in more costs because of initial learning curve problems? Knowing the history behind the data will allow for its proper allocation for future estimates.
Step 10 - Cost Estimating Process
Conduct Risk Analyses
Step 8 - Cost Estimating Process
Conduct What-If Analyses
Differentiate between cost estimating and cost analysis - Cost Analysis
Cost analysis is a powerful tool, because it requires a rigorous and systematic analysis that results in a better understanding of the program being acquired. This understanding, in turn, leads to improved program management in applying resources and mitigating program risks. Cost analysis, used to develop cost estimates for such things as hardware systems, automated information systems, civil projects, manpower, and training, can be defined as 1. The effort to develop, analyze, and document cost estimates with analytical approaches and techniques; 2. The process of analyzing, interpreting, and estimating the incremental and total resources required to support past, present, and future systems—an integral step in selecting alternatives; and 3. A tool for evaluating resource requirements at key milestones and decision points in the acquisition process.
Cost data
Cost data generally include labor dollars (to include labor hours, direct costs and overhead rates), material and overhead dollars, facilities, cost of money, and profit.
List 3 main types of data used in cost estimating process
Cost data, Schedule/program data, and technical data
Cost Estimate documentation versus Substantiation
Cost estimates are considered valid if they are well documented to the point at which they can be easily repeated or updated and can be traced to original sources through auditing. Rigorous documentation also increases an estimate's credibility and helps support an organization's decision making. The documentation should explicitly identify the primary methods, calculations, results, rationales or assumptions, and sources of the data used to generate each cost element. Cost estimate documentation should be detailed enough to provide an accurate assessment of the cost estimate's quality. For example, it should identify the data sources, justify all assumptions, and describe each estimating method (including any cost estimating relationships) for every WBS cost element. Further, schedule milestones and deliverables should be traceable and consistent with the cost estimate documentation. Finally, estimating methods used to develop each WBS cost element should be thoroughly documented so that their derivation can be traced to all sources, allowing for the estimate to be easily replicated and updated. Although it is not a precise science, cost estimating has all the logical foundations of a science. Cost estimates use sound logic to substantiate cost is "good". A good cost estimate is briefed, the results should be convincing. Reams of documentation do not do the convincing. Documentation should relate the logical, rationale analysis that supports the estimate. This is substantiation - convincing the reviewer that the estimate has sound basis and providing confidence in the resultant number. It is the sound, logical steps that were taken as the estimate was developed that gives cost estimates credibility. Substantiation is an essential characteristic of a good estimate.
Input to Budget Planning, POM, BES, and SAR Activities
Cost estimates are often used to support the Planning, Programming, and Budgeting System (PPBS) process with POM or BES inputs. Cost estimates provide fiscally phased Investment and Operations and Support funding requirements to Government personnel directly engaged in developing these financial documents. Also, given that cost estimate updates identify funding requirement changes due to economic, schedule, fact-of-life, or technical deviations, estimate results can inform and/or impact an Acquisition Program Baseline (APB).
Discuss the importance of the data collection step of the estimating process
Data is the foundation of every cost estimate. The credibility of data has the single biggest direct effect on the estimate's overall credibility. Depending on the data quality, an estimate will range from an educated guess to a highly defensible cost estimating position. Credible cost estimates are rooted in historical data. Rather than starting from scratch, estimators usually develop estimates for new programs by relying on data from programs that already exist and adjusting for any differences. Thus, collecting valid and useful historical data is a key step in developing a sound cost estimate.
Step 2 - Cost Estimating Process
Define the System
Step 3 - Cost Estimating Process
Define the Work Breakdown Structure (WBS)
Identify 3 major assumption areas where optimism is commonly prevalent
Defining a realistic schedule assessment, technical maturity, and technical readiness, & subjective data (?)
Step 7 - Cost Estimating Process
Develop Point Estimate
Step 6 - Cost Estimating Process
Develop/Select Methodologies
What is the purpose of the Rough Order of Magnitude?
Developed when a quick estimate is needed and few details are available. Usually based on historical ratio information, it is typically developed to support what-if analyses and can be developed for a particular phase or portion of an estimate to the entire cost estimate, depending on available data. Because it is developed from limited data and in a short time, a rough order of magnitude analysis should never be considered a budget-quality cost estimate.
Step 13 - Cost Estimating Process
Document the Analysis
Step 4 - Cost Estimating Process
Establish Ground Rules and Assumptions
What information does a risk analysis provide to decision makers?
For management to make good decisions, the program estimate must reflect the degree of uncertainty, so that a level of confidence can be given about the estimate. Quantitative risk and uncertainty analysis provide a way to assess the variability in the Point Estimate. Using this type of analysis, a cost estimator can model such effects as schedules slipping, missions changing, and proposed solutions not meeting user needs, allowing for a known range of potential costs. Having a range of costs around a point estimate is more useful to decision makers, because it conveys the level of confidence in achieving the most likely cost and also informs them on cost, schedule, and technical risks. Management can use the data in an S-curve to choose a defensible level of contingency reserves. While no specific confidence level is considered a best practice, experts agree that program cost estimates should be budgeted to at least the 50 percent confidence level, but budgeting to a higher level (for example the mean) is now common practice. Moreover, they stress that contingency reserves are necessary to cover increased costs resulting from unexpected design complexity, incomplete requirements, technology uncertainty, and industrial base concerns, to name a few uncertainties that can affect programs. By understanding which input variables have a significant effect on a program's final costs, management can devote resources to acquire better knowledge about them so that risks can be minimized. Finally, knowing early what the potential risks are enables management to prepare contingencies to monitor and mitigate them using an EVM system once the program is under contract. The bottom line is that management needs a risk-adjusted Point Estimate based on an estimate of the range of confidence to make wise decisions.
Establish Ground Rules and Assumptions
Ground rules are generally imposed at the beginning of the process, often by outside sources, while assumptions permit the team to accomplish the analysis in the absence of complete information on the acquisition, operation, maintenance, and support of the system. Descriptions of relevant missions and system characteristics, manning, maintenance, support, and logistics policies are included in the ground rules and assumptions. Each ground rule and/or assumption needs to be examined for realism and potential risk implications.
Provide examples of estimate Ground Rules
Ground rules represent a common set of agreed on estimating standards that provide guidance and minimize conflicts in definitions. When conditions are directed, they become the ground rules by which the team will conduct the estimate. Examples include the base year of the estimate; inflation indices and their sources; inclusion of sunk cost, time-phasing of the estimate, and program schedule.
Step 1 - Cost Estimating Process
Identify User Requirements and Scope of Effort.
Describe what the technical baseline of an estimate is.
In general, program offices are responsible for developing and maintaining the technical baseline throughout the life-cycle. It is normal and expected that early program technical baselines will be imprecise or incomplete and that they will evolve as more information becomes known. However, it is essential that the technical baseline provide the best available information at any point in time. To create an inclusive view of the program, assumptions should be made about the unknowns and should be agreed on by management. These assumptions and their corresponding justifications should be documented in the technical baseline, so their risks are known from the beginning. Many large programs - primarily within the Department of Defense (DoD) - will capture their program's technical baseline as well as key schedule and programmatic information in a Cost Analysis Requirements Description (CARD). * Technical baseline has been developed by qualified personnel such as system engineers. * It has been updated with technical, program, and schedule changes, and it contains sufficient detail of the best available information at any given time. * The information in the technical baseline generally drives the cost estimate and the cost estimating methodology. * Cost estimate is based on information in the technical baseline and has been approved by management. The technical baseline answers the following: * What the program is supposed to do—requirements; * How the program will fulfill its mission—purpose; * What it will look like—technical characteristics; * Where and how the program will be built—development plan; * How the program will be acquired—acquisition strategy; * How the program will operate—operational plan; * Which characteristics affect cost the most—risk.
Discuss the importance of the following statement: "The estimate is spread over the life of the program according to the program schedule, acquisition strategy, and according to the rules for the appropriations from which it is funded."
In the RDT&E appropriation, Program funds are incrementally funded. Each Fiscal Year's Budget Authority (BA) reflects only the costs expected to be incurred in that Fiscal Year. Procurement efforts are full-funded. Each Fiscal Year's Budget Authority includes the total Prime Mission Equipment cost of all of the units authorized for production in that Fiscal Year plus all the associated Systems Engineering and Program Management (SEPM). Training Equipment, Peculiar Support Equipment or Initial Spares do not have to be funded in the same year as the Prime Mission Equipment. Training, Support Equipment and Initial Spares are funded or ordered, lead time away, so the Government will be able to support the Program when the system is delivered. A Program's funding needs to be consistent with the program schedule, including contract award date, prototype material procurement, development period, and test activities. Inaccurate or misaligned time phasing can lead to budget execution issues (Excess or Insufficient Funding), which could impede contractor progress with work stoppages or delay planned activities. The Program time phasing also has to be consistent with the contract and the acquisition strategy. The contract may authorize the Program to select from among different options by Fiscal Year.
Parametrics Methodology
In the parametric method, a statistical relationship is developed between historical costs and program, physical, and performance characteristics. The method is sometimes referred to as a "top-down" approach. Some types of physical characteristics used for parametric estimating are weight, power, and lines of code. The goal of parametric estimating is to create a statistically valid Cost Estimating Relationship (CER) using historical data. The parametric CER can then be used to estimate the cost of the new program by entering its specific characteristics into the parametric model. CERs established early in a program's life cycle should be continually revisited to make sure they are current and the input range still applies to the new program. In addition, parametric CERs should be well documented. More simplified CERs include rates, factors, and ratios.
Step 15 - Cost Estimating Process
Input to Budget Planning, POM, BES, and SAR Activities
Why is cost estimating a critical element in any acquisition process?
It aids decision makers in evaluating resource requirements during milestone reviews & important decision points. They are essential in determining and communicating a realistic view of likely cost and schedule outcomes that may be used to plan the work necessary to design, manufacture, delivery, and sustain a system. Cost estimating provides valuable information to help determine whether a program is feasible, how the system should be designed, and the resources required. It is required in making informed decisions regarding programmatic, technical, and schedule related decisions to include 1) selecting resources; 2) assessing technology changes, analyzing alternatives, and performing design trade-offs; and 3) satisfying Government statutory and oversight requirements.
What is the scope and purpose of a Life Cycle Cost Estimate?
Life cycle can be thought of as a "cradle to grave" approach to managing a program throughout its useful life. A LCCE encompasses all past (sunk), present, and future costs for every aspect of the program, regardless of funding source. The DoD identifies four phases that a LCCE must address: research and development, procurement and investment, operations and support, and disposal. Life cycle costing enhances decision making, especially in early planning stages and concept formulation of acquisition. LCCEs provide a wealth of information on how much programs are expected to cost over time. Design trade-off studies conducted during this period can be evaluated on a total cost basis, as well as on a performance and technical basis. A life-cycle cost estimate can support budgetary decisions, key decision points, milestone reviews, and investment decisions. Having full life cycle costs are important for successfully planning program resource requirements and making informed decisions.
Best practices for validating an estimate
OMB's Circular No. A-94 and best practices established by professional cost analysis organizations, such as ICEAA, identify four characteristics of a high-quality, reliable cost estimate. It is well-documented, comprehensive, accurate, and credible.
Develop Point Estimate
Once data has been collected, the automated model has been developed, tested, and debugged, and cost methodologies have been identified, the cost team is ready to run, or exercise, the model. This simply means setting the model input values to reflect the system being estimated and calculating the estimate.
Perform Crosschecks
Once the baseline system has been defined and the accompanying point estimate has been developed, the cost team must determine if the results are reasonable. Crosschecks compare the results of the primary cost estimating methodologies with results from alternative methods. At a minimum, cross-checks should be performed on cost and risk drivers and at a higher level for the entire system. If the crosschecks support the primary cost estimate, then the estimating team's confidence in the primary estimate is improved. Crosschecks are also used in evaluating contractor proposals and ECPs.
Assumption - defining a realistic schedule assessment
One of the most important ground rules is to define a realistic schedule. It may be difficult to perform an in-depth schedule assessment early to uncover the optimism in initial program schedules, however it is important that this ground rule outline the confidence the team has in the ability to achieve the schedule so that it can be documented and presented to management.
Step 12 - Cost Estimating Process
Perform Cost Trade Studies
Step 9 - Cost Estimating Process
Perform Crosschecks
Step 14 - Cost Estimating Process
Present the Results and Reconcile
Differentiate between risk and uncertainty
Risk and uncertainty refer to the fact that because a cost estimate is a forecast, there is always a chance that the actual cost will differ from the estimate. Moreover, lack of knowledge about the future is only one possible reason for the difference. Another equally important reason is the error resulting from historical data inconsistencies, assumptions, cost estimating equations, and factors typically used to develop an estimate. There is an important distinction between the terms risk, opportunity, and uncertainty: · Risk is the probability of a loss or injury · Opportunity is a favorable event or outcome · Uncertainty is the indefiniteness about the outcome of a situation Cost estimating is analyzed more often for uncertainty than risk, although many textbooks use both terms to describe the effort.
More reasons to document a cost estimate...
Satisfy policy requirements for properly recording the basis of the estimate, Convince management and oversight staff that the estimate is credible, Provide supporting data that can be used to create a historical database, Help answer questions about the approach or data used to create the estimate, Record lessons learned and provide a history for tracking why costs changed, Define the scope of the analysis, allow for replication so that an analyst unfamiliar with the program can understand the logic behind the estimate, and Help conduct future cost estimates and train junior analysts. To convey this information most effectively, high quality estimating documentation is explicit, clear and concise.
Schedule or program data
Schedule or program data provide parameters that directly affect the overall cost. For example, start date, effort duration, option periods, test requirements, contract type, sole source or competitive awards must be considered in developing a cost estimate.
What information does a sensitivity analysis provide to decision makers?
Sensitivity analysis allows program managers to determine how sensitive a program is to changes in key assumptions concerning the technical baseline and implementation approach, and to make decisions based on that information. Many assumptions made at the start of a program turn out to be inaccurate. Therefore, once the point estimate has been developed, it is important to determine how sensitive the total cost estimate is to changes in the cost drivers.
Conduct What-If Analyses
Sensitivity and what-if analysis demonstrates the viability of program options. A major limitation of sensitivity analysis is that it only analyzes the assumptions, alternatives, or variables considered. This limitation emphasizes that critical thinking and dialog with experts are essential to preparing credible and defendable analyses.
What are the limitations of a Point Estimate?
Since cost estimates are uncertain, making good predictions about how much funding a program needs to be successful is difficult. In a program's early phases, knowledge about how well technology will perform, whether the estimates are unbiased, and how external events may affect the program is imperfect. For management to make good decisions, the program estimate must reflect the degree of uncertainty, so that a level of confidence can be given about the estimate. Quantitative risk and uncertainty analysis provide a way to assess the variability in the Point Estimate.
Perform Cost Trade Studies
Some estimates support the comparison and assessment of alternatives for various purposes such as Cost as an Independent Variable (CAIV). In these cases, multiple scenarios are assessed which may satisfy the stated requirements more effectively or efficiently.
Collect Data and Analyze Schedules
Sources include contractor data such as the Contract Performance Report (CPR) and the Contract Funds Status Report (CFSR), program schedules, and technical information; contractor proposals; other contract negotiation data; contractor accounting records; and analogous historical data on other programs. Cost analysts also need to analyze detailed schedules to ensure that development and manufacturing activities are defined, logical, and executable.
Staffloading Methodology
Staff loading is an appropriate approach when the number of people who need to support a particular activity over time is available. For example, Interim Contractor Support (ICS) actuals are 5 Full Time Equivalents (FTEs) on an annual basis. The methodology requires, in addition to the number of personnel, the duration and a labor rate to reflect the required skill mix. Other WBS elements that might be estimated using staff loading are System Engineering, Program Management, Training, or Operations and Support.
Provide examples of potential cost driving technical baseline input parameters.
System purpose, technical system & performance characteristics, WBS, Description of legacy or similar systems, Training plan, Deployment details, System test & evaluation plan, Operational concept, Personnel requirements, Logistics support details, Changes from last technical baseline, (development, test, and production quantities and program schedule)
Technical data
Technical data define the requirements for the equipment being estimated, based on physical and performance attributes, such as length, width, weight, power and size
Define the Work Breakdown Structure (WBS)
The WBS and WBS dictionary provide a consistent and visible framework for the individual cost elements to be estimated. The guidance and mandatory procedures for those programs subject to DoD 5000 series guidance signed 12 May 2003. The MIL-STD-881 appendices identify and define aircraft, electronic/automated software, missile, ordnance, ship, space, and surface vehicle system WBS elements through level 3.
Discuss the challenges of the data collection step of the estimating process
The challenge in doing this is obtaining the most applicable historical data to ensure that the new estimate is as accurate as possible. One way of ensuring that the data is applicable is to perform checks of reasonableness to see if the results are similar. Different data sets converging toward one value provides a high degree of confidence in the data. Data collection is a lengthy process and continues throughout the development of a cost estimate and through the program execution itself. Many types of data need to be collected and analyzed to include technical, schedule, program, and cost information. Data can be collected in a variety of ways, such as databases of past projects, engineering bottoms-up estimating analysis, interviews, surveys, data collection instruments, and focus groups. Once collected, this data needs to be normalized. After the estimate is complete, the data needs to be well documented, protected, and stored for future use in retrievable databases. Cost estimating requires a continual influx of current and relevant cost data to remain credible.
Conduct Risk Analyses
The cost team performs cost, technical, configuration, and schedule risk and uncertainty analysis based on the guidelines found in the Joint Cost and Schedule Risk and Uncertainty Handbook. Cost estimating risk is an assessment of the uncertainty associated with the cost estimating method that is employed on a WBS element-by WBS element basis. Ideally, distribution shape and bounds will be derived from the source data used to generate the cost estimating relationship. Tecolote believes, however, in applying expert opinion to adjust the purely statistical results to fit the real worldview of the situation. Quantitative risk analysis techniques and tools are applied for all aspects of risk analysis according to the purpose of the estimate, the amount of time allowed for the analysis, and the level of specification possible in identifying uncertainty. Formal estimates typically require quantitative risk analysis.
Develop/Select Methodologies
The cost team selects and applies estimating methodologies based on the maturity of the program, the data available, and the purpose for the estimate, the program phase, and multiple other reasons.
Engineering Build-Up Methodology
The engineering build-up cost estimating method builds the overall cost estimate by summing or "rolling-up" detailed estimates done at lower levels of the WBS. Since the lower-level estimating associated with the build-up method uses industrial engineering principles, it is often referred to as engineering build-up and is sometimes referred to as a grass-roots or a bottom's-up estimate. An engineering build-up method is most commonly used during the production phase of a program, because the program's configuration has to be stabilized, and actual cost data are available. The build-up method is used when an analyst has enough detailed information about building an item—such as number of hours and number of parts—and the manufacturing process to be used.
Crosschecks compare the results of the primary cost estimating methodologies with results from alternative methods. What is the purpose and importance of this estimate step?
The main purpose of cross-checking is to determine whether alternative methods produce similar results. If so, then confidence in the estimate increases, leading to greater credibility. Crosschecks compare the results of the primary cost estimating methodologies with results from alternative methods. At a minimum, cross-checks should be performed on cost and risk drivers and at a higher level for the entire system. If the crosschecks support the primary cost estimate, then the estimating team's confidence in the primary estimate is improved. Crosschecks are also used in evaluating contractor proposals and ECPs.
Formulate an Automated Model
The only reliable way to ensure accuracy, meet time constraints, and react to "what-if" drills is to develop an automated tool to perform the required calculations. Some cost analysts use electronic tools like MS Excel. Tecolote-developed ACEIT's Automated Cost Estimator (ACE), a spreadsheet-based application, with databases, that is optimized for cost estimating and risk analysis, is highly recommended to meet these objectives.
What are the benefits of utilizing a MIL-STD-881 WBS to build an estimate?
The primary purpose of the handbook is to provide uniformity in definition and consistency of approach to developing WBSs. The common terminology and definitions of MIL-STD-881 facilitate communication across the acquisition community and with DoD contractors.
Document the Analysis
The process and estimate must be documented to the extent that it can be replicated by another, equally qualified analyst. A technical manager reviews all Tecolote documentation for accuracy and completeness before it is presented to the client.
Identify User Requirements and Scope of Effort
The user requirements and the scope of the effort must be identified before any significant effort begins. Analysts need to engage with the customer to answer the following questions: What will the estimate ultimately are used for (e.g., milestone decisions, budget submissions, and source selections)? What organizations will be receiving the estimate and what are their expectations? When is the estimate due? What are the deliverable requirements (e.g., briefings, reports, and ancillary products)? The team works with the customer to designate and obtain any necessary programmatic, contractual, technical, schedule, and cost data to support the estimate.
Why is it important to understand the purpose, scope, and time constraints of an estimate in very specific terms? Provide examples?
This information impacts how the estimate will be developed and what will be included. Time constraints may dictate the level of analysis possible, and hamper the amount of time available to research data for use to support the analysis. Don't underestimate the importance of the purpose, scope and time constraints of the estimate. Once the cost analysts know the context of the estimate and the customer's needs (purpose), they can determine the estimate scope based on its intended use and the availability of data. For example, an analyst conducting an independent cost estimate is typically provided the time and resources needed to conduct a thorough analysis; the resulting analysis is expected to be more detailed than the response to a what-if exercise.
Present the Results and Reconcile
Throughout the development of the estimate, the team leader considers the final product and how and where it will be reviewed and presented to decision-makers. Different types of estimates require various levels of review and standards of presentation. Establishing and coordinating a "baseline presentation package" with customers during the early stages of the estimate is extremely beneficial. Establishing the baseline presentation package is another way of ensuring all cost elements are being considered in the estimate, and it mitigates last-minute panics in putting a product together.
Step 11 - Cost Estimating Process
Time-Phase Estimate
Define the System
To develop a credible cost estimate, the estimating team begins by establishing a technical and physical description of the system. For example, the following types of questions have to be answered: What equipment (hardware and software) comprises the system? Is there any Government-furnished equipment (GFE) and property in the system? What are the physical and design parameters of the system? The estimating team also needs programmatic information and an understanding of the system operational concept. For a Major Defense Acquisition Program (MDAP) milestone decision estimate required for a Defense Acquisition Board (DAB), all this information—and more—is documented in the CARD. In most of these cases, the cost analysis team plays a significant role in either leading or supporting the development of the CARD.
Step 5 - Cost Estimating Process
a. Collect Data and Analyze Schedules b. Formulate an Automated Model
Parametrics Advantages
• Based on objective analysis of historical data, which increases an estimate's defensibility • Facilitates "what-if" and sensitivity analysis by varying input parameters • Excellent for use early in program life cycle before a detailed design exists (though must know technical parameter inputs)
Staffloading Disadvantages
• Dependent on use of realistic schedule and labor rate representative of required skill mix
Staffloading Advantages
• Easy to understand methodology for labor intensive efforts • In lieu of using parametric or factor to determine staff month effort, is based on actual FTEs based on analogy
Engineering Build-Up Disadvantages
• Expensive, time consuming and slow to implement • Inflexible - cannot address what-if / sensitivity analysis or be easily updated • Prone to errors through omissions
Provide 3 purposes for documenting a cost estimate
• First, thorough documentation is essential for validating and defending a cost estimate. That is, a well-documented estimate can present a convincing argument of an estimate's validity and can help answer decision makers' and oversight groups' probing questions. • Second, documenting the estimate in detail, step by step, provides enough information so that someone unfamiliar with the program could easily recreate or update it. • Third, good documentation helps with analyzing changes in program costs and contributes to the collection of cost and technical data that can be used to support future cost estimates. • Finally, a well-documented cost estimate is essential if an effective independent review is to ensure that it is valid and credible. It also supports reconciling differences with an independent cost estimate, improving understanding of the cost elements and their differences so that decision makers can be better informed.
Engineering Build-Up Advantages
• Gives good insight into major cost contributors
Parametrics Disadvantages
• Obtaining data from several analogous systems can be difficult to obtain • Complicated CERs can be difficult to explain
Sensitivity Analysis examples
• Percent of total software functionality to be provided through "reuse" of pre-existing software or a COTS software product • Effort factor associated with reuse of a hardware or a software product as compared to the effort to build it new (e.g., is reuse factor 25% of building it new, 50%, 75%?) • Variations in learning curves or quantity discount curves applied to hardware production efforts. Additionally, the analyst should perform sensitivity analysis on significant technical ground rules and assumptions. For example, if your program assumes there is competition, you may want to explore the potential impact to cost and schedule if there is only a single response to the Request for Proposal (RFP) in terms of additional documentation and potential increased prices in a sole source environment.
Analogy Disadvantages
• Reliance on a single data point • Difficult to find the detailed cost, technical, and programmatic data required • Tendency to be too subjective in the adjustment factors
Analogy Advantages
• Useful before detailed program requirements are known • Analogies are quickly developed, usually at minimal investment to research • Easy to understand
