You want to be within 10% of the cost? (Internal Variance in Product Costing)

 Product Cost Management, Purchasing / Sourcing  2 Responses »
Mar 182013
 

There are universalities that seem to cross people and cultures, such as, it’s polite to say “please” and “thank you.” These universalities also occur numerically. For example, designs that follow the Golden Ratiopop up all over the world. Many other aspects of one group versus another may vary, but there are these universal touchstones that pervade the world. The same is true with companies. Granted, one might argue that one company simply is imitating another company and that is why they share a simple practice or the important of a certain number. We believe that this is, indeed, true in most cases. Still, there are a couple of numbers in companies that seem to arise independently in all companies. We are going to talk about some of those universal and independent numbers today with respect to Product Cost Management.

The great universal number in PCM is “10%.” I have met hundreds of companies over the years, both in consulting and when I was the Founder, CEO, and then Chief Product Officer of one of the product cost management software companies. Invariably, a meeting with a company will occur, in which one of the customers will utter the “a” word: accuracy. The dialogue proceeds similar to the following:

CUSTOMER: So, how accurate is your software?
PCM TOOL COMPANY: What do you mean by “accurate?”
CUSTOMER: Uhh, um, well, ya know. How ‘good’ is the number from your software?
PCM TOOL COMPANY: Do you mean do we have miscalculations?
CUSTOMER: No, no, I mean how accurate is your software to the ‘real’ cost?
PCM TOOL COMPANY: What do you consider the real cost?
CUSTOMER: Uhh, um, well, I guess the quotes that I get from my purchasing department from our suppliers.
PCM TOOL COMPANY: Oh, I don’t know, because it depends on how close the quote is to the true cost of manufacturing the part plus reasonable margin.  Are you confident your quotes are correct proxies for the true cost of manufacturing.
CUSTOMER: Hhmmmmmmm… yeah, I think think so
PCM TOOL COMPANY: OK, how close do you expect the costs from our PCM software to be to your quote [or internal factory cost or whatever source the customer believes is truth]?
CUSTOMER: Oh, you know, I think as long as you are +/-10% of the quote, that would be alright.

 

Ding! Ding! Ding! – no more calls, we have a winner! The customer has uttered the universal expectation for all costs produced by a product cost management tool, with respect to the “source of true cost”: +/-10%

The universal expectation of customers of Product Cost Management software is that the PCM tool is accurate to within +/-10% of whatever forecast the customer considers the “true cost.”

This expectation is so common that you would think that every customer in the world had gone to the same university and had been taught the same expectation. Of course, that is not the case, but it is a ubiquitous expectation. How did this universal convergence of expectations come to be? We will probably never know; it’s one of the great mysteries of universe, such as, why do drivers is Boston slow down to 4 mph at the lightest sign of snow or rain?

The more important question is: Is the expectation that the cost from a PCM tool should be +/-10% of a quote realistic? To answer this question, we first have to ask:  How truthful is “the truth?” The truth in this case is supposedly the quote from the purchasing department. The reader may already be objecting (or should be), because there is not just one quote, but multiple quotes. How many quotes does a company get? Well, that depends, but we all know how many quotes a typical company gets: THREE!

The universal number of quotes that purchasing gets is 3, and they believe the “true cost” is within +/-10% of whichever of these 3 quotes they select as truth. 

Three shall be the number of the quoting and the number of the quoting shall be three. Four that shalt not quote; neither shalt they quote two, excepting thou proceedest to three.

Variance Within Supplier Quotes

Do all the  quotes have the same price for the quoted part or assembly? No, of course not. If they were the same, purchasing would only get one quote. So what is the range among these quotes? That is a fascinating question, one that I am currently investigating. So far, my research indicated that the typical range among a set of three quotes is 20-40%. That seems about right from my personal experience.

But, is the “true” price (cost + reasonable margin) contained within the range of the 3 quotes? Not necessarily. If we assume that quotes are normally distributed (another assumption that I am researching), the range would be much bigger in reality. For example, if we had three quotes evenly distributed with the middle being $100 and these quotes had a 30% range, the high quote would be $115 (+15%) and the low, $85 (-15%). This gives us a standard deviation that, conveniently, is $15. At two standard deviations (~95% confidence or “engineering” confidence), we predict that the “true cost” of the part is between a predicted  high quote of $130 and a low of $70. This is a range of 60% (+/-30%). You can see this on Figure 1.

OK, but what about if we  just source a single supplier. Well, there will be variance in this supplier, as well. This variance breaks down into two types: physical noise and commercial noise.

  • Physical noise — the difference in cost that could occur due to physical reasons, such as choosing a different machine (i.e. different overheads) a different routing (sequence of the machines), or even simple human variation from part to part or day to day.
  • Commercial noise – differences in pricing driven by the market, emotions, and transient conditions.
Variance in the Cost Forecast from Quotes Hiller Associates

Figure 1 – Variance in the Cost Forecast from Quotes (click to enlarge)

Physical noise can easily account for a range of 20% (+/-10%) in the quote that a supplier might provide to an OEM. However, physical noise can be quantified and discovered. A supplier can share what routing or machine they are using. The problem is that Commercial noise is very difficult to quantify. How do you quantify when the supplier believes you hurt him in the last negotiation and now he is going to repay you for it, or that he needs your company as a new strategic customer and will underbid to get initial business? Worse yet, Commercial Noise is often LARGER than Physical Noise in the quote! How big is Commercial noise? That is difficult to say, because we can’t measure it very well, but from our discussions with purchasing groups, at minimum, Commercial Noise adds at least another +/-10% .

Physical Noise  Commercial Noise
Comes from selection of different machines, routings. Comes from market conditions, emotions, and transient conditions
Quantifiable in general by understanding the selections. Very difficult quantifiable
+/-10% of the “factory average” +/-20%+ on top of Physical Noise

 

Supplier quotes are just one forecast of true cost. There are other forecasts the organization has.

Cost Estimation Experts

What about those people in the organization with the most manufacturing and product cost knowledge? What is the noise in their estimates compared to a source of alleged truth, such as a quote. We are not sure, but we have asked another question about variance to these experts. When asked the question, “How close are you as a cost estimator to the estimates of other cost estimators in your company, people most often reply, “Probably +/-10-20% depending on the complexity of the part cost estimate or assembly.” So, we might say that the cost estimators have at least a 30% range of quotes themselves.

Historical Costs in ERP

What about the historical costs in ERP? How “accurate” are they? There’s actually at least two problems with data in corporate databases. First, sometimes it is just plain wrong from the beginning of its life in the database. However, even if it is correct initially, it gets out of date very quickly. Material cost, labor rates, efficiency, etc. change. Go ask you purchasing buyer how close a re-quote of a current part that has been in the database for three or four years will be to the original quote. To give you an idea of the magnitude of this problem, consider these findings:

The Accuracy (i.e. variance to quote) of a Product Cost Management Software

Variance in Different Forecasts for Product Cost (click to enlarge)

Variance in Different Forecasts for Product Cost (click to enlarge)

So, after all of the discussion of the variance within other cost forecasts, how “accurate” are the forecasts from a product cost management software? Well, if the internal variance among expert cost estimators independently estimating is 30%, the BEST the PCM tool could do would be +/-15%… IF it is controlled by experts. What happens when non-experts use this software? How much does the range increase? Who knows? Obviously, the more automatic and intelligent the PCM Tool, the less the added variance would theoretically be. But, is this added variance +/-5%, +/-10%, +/-20%?  That is hard to say.

The Reality of Accuracy and Variance in Product Cost Forecasts

Regardless of the answer to the above question, the bigger questions are:

  1. What is your EXPECTATION of how “accurate” your PCM Tool’s cost forecast is to the quote forecast?
  2. Is your expectation reasonable and realistic?

We know the answer to question 1:   Be +/-10% of a my selected quote.

To answer the second question, let’s quickly review what we know:

Source of the Cost Forecast Common Variance Inherent in the Forecast
Range among 3 quotes +/-15%
95% confident interval (engineering confidence in quotes) +/- (15%+15%)
Physical noise within one single supplier +/-10%
Physical noise plus Commercial noise within one single supplier +/- (10%+20%)
Internal range among cost experts +/-30%
Best Case PCM Tool used by experts +/-30%
Non-cost expert using PCM tool +/- (30%+ 5%?)
Common [Universal] expectation of PCM Tool Cost Forecast +/-10%

 

Hhhmmmmmmm… Houston, I think we have a problem.

It just doesn’t seem that +/-10% is a reasonable expectation.

Bringing Sanity Back to Product Cost Management Expectations

What can you do in your company to help reset these unrealistic expectations? There are three things.

  1. First make your colleagues (engineering, purchasing, etc.) aware of the reality of the cost forecasting world. Don’t let them develop uninformed and unrealistic expectations.
  2. Don’t focus exclusively on the end cost, but on the physical and immutable concepts that cost is supposed to quantify: mass, time, tooling.
  3. Start to quantify the internal variance in your own firm’s cost forecasts. Your firm’s internal cost ranges in quotes, internal estimates, etc. may be lower or higher than the numbers presented here. However, you won’t know until you start to investigate this.

Is this a painful realization?  Perhaps, but you are already living with the situation today.  It is not a new problem in the organization.  If you don’t acknowledge the potential problem, you run the risk of misleading yourself.  If you acknowledge the potential problem, you may be able to solve it, or at least make it better.

 

Share

DARPA false!? 80% of Cost is Not Controlled by Design?

 Total Cost of Ownership  16 Responses »
Mar 042013
 

 

We are still on our epic quest to find the DARPA study (a.k.a. the legendary seminal study reported to say that ~80% of product cost is determine in the first ~20% of the product lifecycle).  However, during our search we have been aided by Steve Craven from Caterpillar.  No, Steve did not find the DARPA study, but he did send us a study attempting to refute it.

 

Design Determines 70% of Cost? A Review of Implications for Design Evaluation
Barton, J. A., Love, D. M., Taylor, G. D.
Journal of Engineering Design, March 2001, Vol. 12, Issue 1, pp 47-58
 

Here’s a summary of the paper and our comments and thoughts about this provocative article.

Where’s DARPA and Can We Prove this 70-80% number?

First, the authors question the existence of the DARPA study and say that most studies that support DARPA’s findings reference other corporate studies that are alleged to support DARPA’s findings.  Most of these corporate studies are difficult to trace.   They authors then analyze a Rolls-Royce study (Symon and Dangerfield 1980) that investigates “unnecessary costs.”   In the Roll-Royce study, Symon and Dangerfield find that the majority of unnecessary costs are induced in early design.  However, Barton, Love, and Taylor make the point that unnecessary costs are NOT the same as the TOTAL cost of the part itself.   That’s fair.

The authors then go into a more “common sense” line of discussion about how the costs induced at different stages of the product lifecycle are difficult to disaggregate.  The difficulty occurs  because design choices often depend on other upstream product cost choices and the knowledge or expectation of downstream supply chain and manufacturing constraints.  This section of the paper concludes with a reference to a study by Thomas (The FASB and the Allocation Fallacy from Journal of Accountancy) which says that “allocations of this kind are incorrigible, i.e. they can neither be refuted nor verified.”

We at Hiller Associates agree with these assertions in the sense that these statements are tautologically true.  Maybe someone should have given this study to Bob Kaplan of Harvard Business School before he invented Activity Based Costing in the 1980’s in collaboration with John Deere?  After all, wasn’t ABC all about the allocation of costs from indirect overhead?  However, Kaplan’s attempt illustrates the reality of the situation outside of academia.  We in industry can’t just throw up our hands and say that it’s impossible to allocate precisely.  We have to make a reasonable and relevant allocation, regardless.  If it is not ‘reliable’ from a canonical accounting definition point of view, we just have to accept this.

Is DARPA Actually Backwards in Its Cost Allocations?

What if the DARPA study’s 80/20 claim is more that an allocation problem?   What if DARPA is actually promoting the opposite of the truth?   The author references a paper by Ulrich and Pearson that may reverse DARPA.  Ulrich and Pearson investigated drip coffee makers and conclude that the design effect on product cost accounted for 47% of cost, whereas manufacturing accounted for 65% of product cost variation.  They did, of course, make their own assumptions for that type of possible manufacturing environments that could have made the 18 commercially available coffee makers.

Considering the pre-Amazon.com world in 1993 when the Ulrich and Pearson study was done, it brings a smile to my face thinking of MIT engineering grad students at the local Target, Kmart, or Walmart:

CLERK:  Can I help you?
GRAD STUDENTS: Uh, yeah, I hope so.  We need coffee makers.
CLERK:  Oh, well we have a lot of models, what is your need…
GRAD STUDENTS: Awesome, how many do you have?
CLERK:  Uhh… I guess 17-18 models, maybe.
GRAD STUDENTS: Score!  We need 18.
CLERK:  18 of which model?
GRAD STUDENTS: Oh, not 18 of one model.  One of each of the 18 models.
CLERK:  What!  Huh… wha-why?
GRAD STUDENTS:  We’re from MIT.
CLERK:  Ooohhhhh…. right…
GRAD STUDENTS:  Uhh… Say, what’s your name?
CLERK:  Um… Jessica… like my name tag says.  You say you go to MIT?
GRAD STUDENTS:  Um, yeah, well Jessica, we’re having a party at our lab in Kendall Square this Friday.  If you and your friends want to come, that would be cool.    What do you say?
CLERK:  Uh, yeah right… how about I just get you your “18” different coffee makers.  Good luck.

 

… but we digress. Is product cost determined over 50% by manufacturing technique rather than design?   That seems a bit fishy.

Design for Existing Environment

With the literature review out of the way, the authors get to business and propose their hypothesis:

That consideration of decisions further down the chain are beneficial can be illustrated with a new ‘design for’ technique, Design For the Existing Environment (DFEE) that aims to take into account the capacity constraints of the actual company when designing the product… This contrasts with the conventional DfX techniques that take an idealized view of the state of the target manufacturing system.

They then talk about a simulation that they did which they hope takes into account inventory, profit, cash flow, missed shipments to customers, etc.  They run 5 scenarios through their simulation:

  1. A baseline with New Design 1 that lacks sufficient capacity needed by the customer demand
  2. A New Design 2 that uses DFEE to use the existing manufacturing environment and can meet customer demand.
  3. Making New Design 1 by buying more capacity (capital investment)
  4. Pre-Building New Design 1 to meet demand
  5. Late deliver of New Design 1

Not surprisingly, the authors show that scenario 2, using their DFEE technique, beats the other alternatives, considering all the metrics that they calculate.

Thoughts from Hiller Associates

This article is from over ten years ago, but it is thought provoking.  Is 80% of the cost determined in the first 20% of design?  We don’t know.  We certainly believe that over 50% of the cost is determined by design.  In our professional experience, a large part is controlled by design, even allowing for the relationships between design, purchasing, manufacturing, and supply chain.  We’ve personally observed cases in which moving from one design to another allowed for the use of another manufacturing process that reduced total cost by 30%-70%.

Overall, the authors bring up a valid point that goes beyond the traditional ringing of the Total Cost of Ownership (TCO) bell.  They present a simulation in which they claim to calculate Total Cost of Ownership in a rigorous way.  The problem is that the calculation is too rigorous (it took them 4 hours per simulation).  That kind of time and, moreover, the complexity underlying such a model is likely not practical for most commercial uses.   However, a more simplified estimation of Total Cost of Ownership is more appropriate.  In fact, Hiller Associates has helped our client’s teams use flexible tools like Excel, along with a well designed process, to estimate a Total Cost of Ownership.  Is that an end point?  No, but it is a beginning.  Later, as a client’s culture, process, and team improve, more advance Product Cost Management tools can be added into the mix.  And, we do mean TOOLS in the plural, because no one tool will solve a customer’s Product Cost Management and Total Cost of Ownership problems.

Hopefully, we will see some more academic work on the product cost problem.  But, until then, we’re still searching for the original DARPA Study.  Anyone know where it is?

References
  1.  Design Determines 70% of Cost? A Review of Implications for Design Evaluation, Barton, J. A., Love, D. M., Taylor, G. D., Journal of Engineering Design, March 2001, Vol. 12, Issue 1, pp 47-58
  2. Symon, R.F. and Dangerfield, K.J., 1980 Application of design to cost in engineering and manufacturing.  NATO AGARD Lecture Series No. 107, The Application of Design To Cost And Life Cycle Cost to Aircraft Engines (Saint Louis, France, 12-13 may, London, UD 15-16), pp. 7.1-7.17
  3. Thomas, A.L., 1975, The FASB and the Allocation Fallacy, Journal of Accountancy, 140, 65-68.
  4. Ulrich, K.T., and Pearson, S.A., 1993, “Does product design really determine 80% of manufacturing cost? Working Paper WP#3601-93 (Cambridge, MA: Alred P. Sloan School of Management, MIT).
Share
Skip to toolbar