The 6 Reasons Why People and Organizations Withhold Data (Habeas Corpus Data, Part 2)

Jul 232012

Last week we began talking about a common problem in many projects, including Product Cost Management: the withholding of needed data for analysis and modeling. We also talked about the many reasons, some legitimate and some not, why people refuse to share data. Those reasons are summarized in the graph below.

Then I introduced one way of working through the data withholding problem. I call this technique Habeas Corpus Data.

Excuses for no data in Product Cost Hiller Associates

Reasons people withhold data [CLICK TO ENLARGE}

It’s very simple and works like this:

Politely and ardently make your case to the data gatekeeper as to why they should share the data with you.
If you are rebuffed, ask your management or project champion to see if he can break the data roadblock.
If you are still ignored, simply nod politely and stop asking. Instead, go and make an assumption for what the data should be.
Present your findings (based on your assumptions for the withheld data) in the presence of the data prison warden (be it the individual or organization that is withholding).
Sit back and watch the chaotic scrambling of the data warden.

This is the typical flow of the Habeas Corpus Data process, but in reality, the result will be slightly different depending on the real reason that the person or organization is guarding the data. In our post today, we are going to go through each reason and talk about the result that Habeas Corpus Data has in each case.

1. Data truly requires confidentiality / conflict of interest

This is the best and, really, the only legitimate reason that data should ever be withheld. Fortunately, it’s also the rarest. Very little data really needs to be kept confidential, especially to another member of the same product program or a consultant who is under a Non-Disclosure Agreement (NDA) already.

Effect of Habeas Corpus Data: The data holder probably feels guilty already for having to withhold the data. If the data is legitimately confidential, he will likely give you an indication of how good your assumptions are saying something such as: “Well, I can’t tell you the exact values, but you’re not making a bad guess.” Or “Well, I can’t tell you the exact values, but I would reconsider your numbers, because they aren’t remotely close.”

The more important effect of Habeas Corpus Data happens when the person or organization is using “confidentiality” as an excuse for one of the other five reasons. This is the common result. In this case, the data warden will start complaining about your results, to which you will politely reply, “Oh, well I had to make an assumption, but if you have actual data that is better, I am happy to redo the analysis with quality assumptions.” This is basically checkmate for you. The data warden will either immediately produce the data, or mumble something vague about having to check with his manager about seeing if he can share the data. This is your permission to promptly follow up a day later to see if that “permission” has been received.

2. Legitimate concern that colleagues / partners understand the sensitivity

Sometimes data is not officially confidential in the sense that it cannot be technically shared with someone of your clearance, but the data warden is personally concerned that you do not understand the sensitivity of the data.

Effect of Habeas Corpus Data: Habeas Corpus Data will almost always make the data warden reconsider his reservations. He may initiate a discussion with you directly or with your executive champion to re-emphasize the gravity of the data he will now share, but you will get your data.

3. Does not know the data asked for

Sometimes the person you ask does not have the data.

Effect of Habeas Corpus Data: The question is, “should” he have the data? If the answer is no or maybe, consider who else might have the data. However, if the data would normally reside in the bailiwick of the person or organization, the organization may welcome your assumptions as the first steps in researching the needed data and may ask you to work with them in refining your assumptions.

4. Too busy or lazy to assemble data

Effect of Habeas Corpus Data: If the reason that the person or organization does not know the data is because they are busy with other things or the organization is simply negligent, Habeas Corpus Data will force them to prioritize digging out the data that they do have in their organization. If they do not currently have the data they will typically prioritize the due diligence required to get it. Otherwise, they will be forced to accept your assumptions.

5. Job security / authority protection

Sadly, the two most common reasons people withhold data are the two most illegitimate reasons. First, many people see data and/or knowledge [which is valuable] as one of the biggest contributors to personal job security. This may seem strange to some younger readers, who have grown up in the world of the internet and social media. In the internet world, sharing (not hoarding) ideas makes one popular and successful. On the other hand, consider the importance of “content” (the fancy internet word for data) ownership and rights. Many people do not want to share the data they have personally appropriated from the organization, believing they will be less important without the data.

Effect of Habeas Corpus Data: Even if he personally still does not want to share data, if the audience with whom you share results includes the data warden’s colleagues, it is highly likely that one of them will force the Data Warden to relent. They will not want you using incorrect assumptions that produce incorrect results. If these results are used and lead to a bad decision, the executives will start investigating the analysis chain and discover that the Data Warden’s organization had data and were aware of the analysis, but refused to share. Habeas Corpus Data typically makes the fear of being seen as a data hoarder greater than the fear of losing your job from sharing data.

6. Fear of exposure of mistakes

Everyone makes mistakes, but some people are more sensitive than others about having their errors exposed. Without the proper data, it is very difficult to prove that a decision was right or wrong in the past. Obviously, you are rarely explicitly looking for mistakes in Product Cost Management. However, with the right data, you may well uncover honest mistakes from the past, or simply new opportunities to re-source, re-route, or re-design a part or product. Many people are uncomfortable with someone finding out that their original decisions were not optimal and/or they just don’t want the hassle of having to deal with a certain part again.

Effect of Habeas Corpus Data: The effect of Habeas Corpus Data in this case is similar to the effect it has on “5. Job security / authority protection.” Habeas Corpus Data typically makes the fear of being seen as a data hoarder greater than the fear of discovering a past mistake or opportunity discovered.

Is Habeas Corpus Data Too Harsh?

Make no mistake, Habeas Corpos Data is a powerful technique. However, readers should not view it, nor practice it with the attitude of being a “data extortionist.” Much like the martial art of Judo, Habeas Corpus Data is really the “gentle way” compared to your other alternatives, escalating the data withholding in a loud way to the executive ranks of a company. As I discussed above, Habeas should not be used until you have personally made your case to the data gatekeeper and asked your immediate management or project champion to see if he can break the data roadblock. Like Judo, Habeas Corpus Data will often allow you to control your data withholding opponent without harming them. It allows them to gracefully back down from their recalcitrant position in a way that will allow them to save face in front of your executives and theirs.

Habeas Corpus Data (Part 1)

Product Cost Culture, Product Cost Management 2 Responses »

Jul 192012

Data.

At the end of the day, the data and assumptions that we use in Product Cost Management, specifically in cost models, is as (or more) important than the equations and the approach to modeling itself. The classic example in cost modeling is raw material rates ($/kg for steel, plastic, etc.). A close second and third in importance are the labor rate ($/man-hour) for work in a given manufacturing activity and direct overhead rates ($/machine-hour).

Good, trustable data can be very hard to find in a company, even when everyone in the company wants to help the cost modeling effort. However, a bigger problem is not the unavailability of data, but the unwillingness of people to share the needed data with you. If you have ever been involved in a Product Cost Management (PCM) effort, especially one involving cost modeling or implementing a PCM tool, you have certainly encountered this problem. Certain people and whole organizations will be unwilling to share data with other organizations and/or with you personally as the PCM professional who is trying to help the firm.

So, what do you do when you run into this data stonewall? Today, I would like to share a very powerful but simple technique that I discovered years ago. Some may call this technique by another name, but I call it “Habeas Corpus Data.”

Why People Withhold Data

As a young researcher at University of Illinois, I was flabbergasted the first time that I was on an internship and someone became squirrely or delaying regarding data. At a university, of course, the goal is to freely share and promote good data to advance the common good of learning and research. However, in organizations this is not the case. Over the next few years, I realized that withholding data was the norm, not the exception in many organizations. The natural question one asks when told “no” is “why?” I have been given a lot of excuses, ranging from the presumably plausible to the outright ludicrous. However, they mostly break down into the six categories shown in the pie chart below. Furthermore, in my experience they break down in the proportions shown.

Reasons people withhold data [CLICK TO ENLARGE}

As you can see, the first three categories, are legitimate concerns that need to be addressed. However, you will also note that this is a minor part of the pie. The lion’s share of the reasons given are simply excuses that should not be tolerated. The most frustrating thing is that no matter how illegitimate the real reason for withholding data is, the withholder will tell you a covering lie.

So, what do you do when you find the data door locked? Easy! You use Habeas Corpus Data to storm the castle.

Habeas Corpus Data

Many of the readers may remember Habeas Corpus from junior high civics class as the Latin phase meaning “you must present the person in court.” This legal idea originated in 12th century England and means that you cannot hold someone in detention without first showing sufficient cause. Or, as William Blackstone explained in 1305 “The King is at all times entitled to have an account, why the liberty of any of his subjects is restrained, wherever that restraint may be inflicted.” [Maybe that means you are the King and the data are your subjects?… oh, if only Product Cost Management were so glorious!]

Just as the court demands a reason for imprisonment of a citizen, so too you demand a real reason why the data is being held hostage. But, how do you expose the data despot for the unlawful detainment? Here’s the key to the prison cell:

Make and assumption and present the results from your work.

For example, if purchasing will not cough up a supplier’s labor and overhead rates for the cost model on which you are working, you assume a labor and overhead rate based on your past experience and expertise in PCM. It does not even matter if your assumption is close to the real numbers. When you present your results for the costs of the product or part, the warden of the data prison will very likely be present. You should make sure of it, which will be easy, because he is likely be a key stakeholder in the product cost project. As this point, you are effectively calling his hand, and he will have to lay down his cards and show the true reason why he is withholding data.

Why does Habeas Corpus Data have this effect on the warden of the data? First, he will be shocked you were able to proceed with your work without his data. More importantly, he will get very worried how your results (which are based on your assumptions, not his) will affect his job. The exact effect that Habeas Corpus Data will have depends on which of the six reasons for withholding data is the real underlying reason the person should have admitted in the first place.

Stay tuned next week, and I will will let you know the specific effect Habeas Corpus Data has based on the true each reason for data withholding.

The Voice of Should-Cost (Voices Series, Part 3)

PCM Technique / Cost Models, Product Cost Culture, Purchasing / Sourcing, Uncategorized 4 Responses »

Jul 092012

It’s been a couple of weeks, since we discussed the Voices series, so if this post is interesting to you, you may want to go back and read the first two:

In these first two articles we introduced several of the voices that are always present in the Product Cost Management conversation, including:

The Voice of Hopefulness – the Pollyanna voice that assumes product cost will just work itself out in the end. It is a voice of justification to ignore Product Cost Management, because the team is just too busy at XYZ point in the development process to seriously consider product cost. Hope is NOT a strategy.
The Voice of Resignation – the nihilist voice that assumes that you have to accept high prices because the three suppliers that purchasing quoted gave you pricing far higher than what seems reasonable
The Voice of Bullying – the seemingly unreasonable scream of the customer telling you what your product should cost — not based on reality, but based on the customer’s own financial targets.

However, there is another voice in the conversation that can bring some reason to the cacophony. It is a voices of reason — the Voice of Should-cost.

Buck-up Cowboy. The Voice of Should-cost Can Help

Should-cost is just what it sounds like, using one or more techniques to provide an independent estimate of what the cost of a part or product “should” be. The question is, what does “should” really mean? For many, the definition depends on the type of cost being calculated, as well as personal should-cost calculation preferences. I will provide my own definition here, mostly targeted at providing a should-cost for a discretely manufactured part.

Should-Cost – The process of providing an independent estimate of cost for a part, assembly, component, etc. The should-cost is based on a specific design, that is made with a specific manufacturing process, and at a supplier with a specific financial structure. Or, the should-cost is calculated assuming a fictitious supplier in a given region of the world that uses the best manufacturing technology, efficiency operating at maximum sustainable capacity.

I realize that this is a broad definition, but as I said, it depends what you want to estimate. For instance, do you know the supplier’s exact manufacturing routing, overhead and labor rates, machine types, etc.? In this case, do you want to estimate what it “should” cost to manufacture the part under these conditions? OR… do you want to know what the cost “should” be for a new supplier who is well-suited to manufacture your design and has a healthy but not overheated order book? Although you could make many other assumptions, the point is: KNOW YOUR ASSUMPTIONS. You will note that I said nothing about margin. Some people call this a “Should-Price,” while others call it a “Should-Cost” referring to what they will pay vs. what the part costs the supplier to make. The only difference is that you will also make an assumption for a “reasonable” margin for a Should-Price.

The important point is that the team relying on the should-cost information must define the scenario for which they want a should-cost estimate. There is nothing wrong with wanting an answer for all these scenarios. In fact, it’s preferable. Run the calculation / estimate more than once.

Should-cost, Should Be a Choir, not a Solo Act

Manufacturing cost is a very tricky thing to calculate. I often say that the true cost of the economic resources to make a part or product is a number known but to God. Put statistically, you can’t know the true meaning or standard deviation of a population, you can only estimate it from the samples that you take. People take two common approaches to should-cost.

The Pop Star Solo Act

The popular solution that too many people pursue is the solution pictured at the right.

There’s no easy button to should-cost

They want the easy button — the single source of truth. They want the plasticized overproduced solo pop star version of should cost, i.e. the easy button tool. There’s nothing wrong with this and there are some really good should-cost solutions available, but none of them are infallible. In addition, it is not appropriate to put the same should-cost effort into each part or assembly in a problem. One should focus where the money is. However, too many people, especially cost management experts, become sycophants of one particular tool to the exclusion of others.

Single estimates in Product Cost Hiller Associates

The Lonely World of a Solo Should-cost Voice

Looking at the diagram to the left, you can see what the landscape looks like when you make your comparisons to one point in cost space. It is an uncertain, scary world when you only have one point of reference. In this case, all one can do is try to force a supplier to match the should-cost output of your favorite tool.

The Andrews Sisters, Competitive Trio Quoting

The other very popular approach comes from the purchasing department: three competitive quotes. If the auto-tuned single pop star should-cost is too uncertain, purchasing will listen to a trio instead. Why three quotes?

Supplier quotes in Product Cost Hiller Associates

The Trio of Should-cost Quoting

No one seems to know, but in EVERY purchasing department with which I have ever worked, three shall be the number of the quoting, and the number of the quoting shall be three. [If you are an engineer, you know my allusion. If not, watch the video to the left!] The trio of quotes in the diagram to the right do help clarify the picture a little better, but there is still too much uncertainty and what I call “commercial noise” to really believe that the quotes alone bound what the should-cost plus a reasonable margin is in reality.

An Ensemble of Should-Cost Estimates

Returning to our statistics example, one of the first things you learn in statistics is that it takes about 33 samples to characterize a bell curve distribution. At 33 samples, you can start to approximate the true mean and standard deviation of the actual population. I am not saying that one needs 33 estimates of should-cost to triangulate on the true cost, but you should get as many as you can within a reasonable time frame. Have a look at the diagram at the right to see this illustrated. Instead of the single pop star approach or the Andrews Sisters trio of quotes, hopefully what you get is a well-tuned small chorus of voices who start to drown out the Voices of Resignation, Hope, and Bullying. The chorus of should-cost estimates start to bound the “true” should-cost of the part or product and can give the team a lot more confidence.

Triangulating on Product Cost Hiller Associates

Chorus of Should Cost [CLICK TO ENLARGE!]

Sometimes the team does not have time to assemble all the voices of should-cost. Not all parts or products are worth assembling the full choir. More often than not, the organization is either unaware of the should-cost voices at its disposal, or are just too lazy to assemble them.

Don’t let your organization be lazy or sloppy with respect to should-cost, and remember that the best music is made when groups of instruments and voices work together, not when one person sings in isolation.

p.s. Bonus PCM points if you can guess what a cappella group is pictured in the thumb nail to the post