Last week we began talking about a common problem in many projects, including Product Cost Management: the withholding of needed data for analysis and modeling.  We also talked about the many reasons, some legitimate and some not, why people refuse to share data. Those reasons are summarized in the graph below.

Then I introduced one way of working through the data withholding problem.  I call this technique Habeas Corpus Data.

It’s very simple and works like this:

• Politely and ardently make your case to the data gatekeeper as to why they should share the data with you.
• If you are rebuffed, ask your management or project champion to see if he can break the data roadblock.
• If you are still ignored, simply nod politely and stop asking. Instead, go and make an assumption for what the data should be.
• Present your findings (based on your assumptions for the withheld data) in the presence of the data prison warden (be it the individual or organization that is withholding).
• Sit back and watch the chaotic scrambling of the data warden.

This is the typical flow of the Habeas Corpus Data process, but in reality, the result will be slightly different depending on the real reason that the person or organization is guarding the data. In our post today, we are going to go through each reason and talk about the result that Habeas Corpus Data has in each case.

1. Data truly requires confidentiality / conflict of interest

This is the best and, really, the only legitimate reason that data should ever be withheld. Fortunately, it’s also the rarest. Very little data really needs to be kept confidential, especially to another member of the same product program or a consultant who is under a Non-Disclosure Agreement (NDA) already.

Effect of Habeas Corpus Data: The data holder probably feels guilty already for having to withhold the data. If the data is legitimately confidential, he will likely give you an indication of how good your assumptions are saying something such as: “Well, I can’t tell you the exact values, but you’re not making a bad guess.” Or “Well, I can’t tell you the exact values, but I would reconsider your numbers, because they aren’t remotely close.”

The more important effect of Habeas Corpus Data happens when the person or organization is using “confidentiality” as an excuse for one of the other five reasons. This is the common result. In this case, the data warden will start complaining about your results, to which you will politely reply, “Oh, well I had to make an assumption, but if you have actual data that is better, I am happy to redo the analysis with quality assumptions.” This is basically checkmate for you. The data warden will either immediately produce the data, or mumble something vague about having to check with his manager about seeing if he can share the data. This is your permission to promptly follow up a day later to see if that “permission” has been received.

2. Legitimate concern that colleagues / partners understand the sensitivity

Sometimes data is not officially confidential in the sense that it cannot be technically shared with someone of your clearance, but the data warden is personally concerned that you do not understand the sensitivity of the data.

Effect of Habeas Corpus Data:  Habeas Corpus Data will almost always make the data warden reconsider his reservations. He may initiate a discussion with you directly or with your executive champion to re-emphasize the gravity of the data he will now share, but you will get your data.

3. Does not know the data asked for

Sometimes the person you ask does not have the data.

Effect of Habeas Corpus Data:  The question is, “should” he have the data?  If the answer is no or maybe, consider who else might have the data. However, if the data would normally reside in the bailiwick of the person or organization, the organization may welcome your assumptions as the first steps in researching the needed data and may ask you to work with them in refining your assumptions.

4. Too busy or lazy to assemble data

Effect of Habeas Corpus Data:   If the reason that the person or organization does not know the data is because they are busy with other things or the organization is simply negligent, Habeas Corpus Data will force them to prioritize digging out the data that they do have in their organization.  If they do not currently have the data they will typically prioritize the due diligence required to get it. Otherwise, they will be forced to accept your assumptions.

5. Job security / authority protection

Sadly, the two most common reasons people withhold data are the two most illegitimate reasons. First, many people see data and/or knowledge [which is valuable] as one of the biggest contributors to personal job security. This may seem strange to some younger readers, who have grown up in the world of the internet and social media.  In the internet world, sharing (not hoarding) ideas makes one popular and successful. On the other hand, consider the importance of “content” (the fancy internet word for data) ownership and rights. Many people do not want to share the data they have personally appropriated from the organization, believing they will be less important without the data.

Effect of Habeas Corpus Data:   Even if he personally still does not want to share data, if the audience with whom you share results includes the data warden’s colleagues, it is highly likely that one of them will force the Data Warden to relent. They will not want you using incorrect assumptions that produce incorrect results. If these results are used and lead to a bad decision, the executives will start investigating the analysis chain and discover that the Data Warden’s organization had data and were aware of the analysis, but refused to share.  Habeas Corpus Data typically makes the fear of being seen as a data hoarder greater than the fear of losing your job from sharing data.

6. Fear of exposure of mistakes

Everyone makes mistakes, but some people are more sensitive than others about having their errors exposed. Without the proper data, it is very difficult to prove that a decision was right or wrong in the past. Obviously, you are rarely explicitly looking for mistakes in Product Cost Management. However, with the right data, you may well uncover honest mistakes from the past, or simply new opportunities to re-source, re-route, or re-design a part or product. Many people are uncomfortable with someone finding out that their original decisions were not optimal and/or they just don’t want the hassle of having to deal with a certain part again.

Effect of Habeas Corpus Data:  The effect of Habeas Corpus Data in this case is similar to the effect it has on “5. Job security / authority protection.” Habeas Corpus Data typically makes the fear of being seen as a data hoarder greater than the fear of discovering a past mistake or opportunity discovered.

Is Habeas Corpus Data Too Harsh?

Make no mistake, Habeas Corpos Data is a powerful technique. However, readers should not view it, nor practice it with the attitude of being a “data extortionist.” Much like the martial art of Judo, Habeas Corpus Data is really the “gentle way” compared to your other alternatives, escalating the data withholding in a loud way to the executive ranks of a company.  As I discussed above, Habeas should not be used until you have personally made your case to the data gatekeeper and asked your immediate management or project champion to see if he can break the data roadblock.  Like Judo, Habeas Corpus Data will often allow you to control your data withholding opponent without harming them. It allows them to gracefully back down from their recalcitrant position in a way that will allow them to save face in front of your executives and theirs.