The Chief Data Officer: An executive whose time has come

I often ask people whether they know what Netflix, Harrah’s, Amazon and Wal-Mart have in common? The answer is pretty simple. They use data analytics to leave their competitors in the dust. Many other businesses are trying to do the same, spending millions of dollars on data software.

It takes more than a steep investment, however, to squeeze business value out of data. Companies have to establish an entire system to use data to drive competitive advantage.

Data as a competitive advantage needs a department that is responsible for the analytics and getting all the needed data.  The data owners and the data users should reside in the same division to ensure the right data is always available and up to date.  Also, the decisions on resources should be within that department, not within IT.

When IT is in charge of the data, they tend to not understand the business as well as needed to facilitate data.  The operations does not understand databases and technology, however the analysts understand the business and the technology, so they should own the data and the facilitation of the data.  

Can You See the Opportunities Staring You in the Face?

I’ve come to believe that less than 1% of the data is truly useful.

Exactly!  Most businesses are very simple if you look for the key metrics.  So many times people want to show their worth by over thinking the problem.  If I can come up with some new innovative way to look at this problem, I'll be a superstar.  But more times than not it isn't a complex problem.  Humans are fairly simple to predict.  Most humans will fall into patterns and want very straightforward things.  New data doesn't need to be introduced until you have gotten everything out of the current data you have.

Big-data initiatives are proliferating, and the information is getting more complex all the time.

There’s a lot of potential benefit for both retailers and customers.

But only if the data is well managed and well understood. Statistics literacy isn’t very high in most businesses. A few educational institutions have realized this and are making a push to turn out business graduates who know their way around a regression analysis. But for the most part, businesspeople aren’t familiar enough with statistics to use them as the basis for good decisions. If you don’t understand the numbers, you can go a long way down a bad road very quickly. That’s why every team charged with making decisions about customers should include a trusted individual who understands statistics. If that understanding isn’t between your own two ears, make sure you bring a person with that skill set onto your team.

Being able to understand what the data is telling you is more important that having a degree in statistics.  Interpreting data is really where the opportunities present themselves, not in figuring out the most optimal model.  I suggest having someone who is proficient in building statistical models and ask a lot of questions from the output.  Start to understand what the answers  of models are telling you and simplify the results into something that can be used in the future.  A model may tell you that people who buy a particular item are likely to be loyal, but is it the item that drives the loyalty or is this just a coincidence?  The better you understand your data, the better decisions you will make and you don't have to be a data scientist to do that.

FiveThirtyEight's Nate Silver Explains Why We Suck At Predictions (And How To Improve) | Fast Company

When human judgment and big data intersect there are some funny things that happen. On the one hand, we get access to more and more information that ought to help us make better decisions. On the other hand, the more information you have, the more selective you can be in which information you pick out to tell the narrative that might not be the true or accurate, or the one that helps your business, but the one that makes you feel good or that your friends agree with.

This is a great article on using data and predictions.  I just bought this book as a good friend of mine suggested it is a great read.  I always hear "You can make numbers tell whatever story you want."  Ain't that the truth?  So many times colleagues of mine hold on to a certain part of the data that tells the story they want to tell and soon it becomes truth, however this only helps them look good instead of moving the business forward.  

How to Repair Your Data - Thomas C. Redman - Harvard Business Review

No matter what, do not underestimate the data quality problem, nor the effort required to solve it. You must get in front of data quality.

Data warehousing is hard.  To build a model that works for the business users and have data quality that truly delivers "one version of the truth" takes dedication and a group that truly understands the business.

Address preexisting issues.

 There are some problems that have been created already, and you have no choice but to address these before you use the data in any serious way. This is time-consuming, expensive, and demanding work. You must make sure you understand the provenance of all data, what they truly mean, and how good they are. In parallel, you must clean the data.

We are currently in the process of doing this in my organization.  In fact, we are going to rebuild the entire data model.  Sometimes it's easier to start from scratch instead of figuring out what is wrong with the current model.  Of course, our model isn't that wonderful for the business, so this made the rebuild decision quite easy.

Prevent the problems that haven't happened yet.

...build controls (such as calibrating test equipment) into data collection; identify and eliminate the root causes of error;

Data warehousing efforts also fail because end users find the errors most of the time.  When this occurs, getting the organization to trust the data becomes a challenge.  There is always the questioning of if this data is right.  Proactively fix data and let end users trust the data, they will spend more time discussing strategy instead of fighting over data quality.

Too Much Data? →

When data is missing, we overestimate its value. Our mind assumes that since we are expending resource locating information, it must be useful.

We're fascinated with filling information gaps and that obsession can lead us astray. Especially today, when reducing uncertainty has become all too easy.

An interesting study on how our minds work utilizing data.  I see this everyday.  People are more fascinated with the information they don't know, rather than looking at what is known and making a solid decision based on the facts at hand.  So many times a report is issued and there are thirty questions, most of the time the questions are deflecting making a decision.  The lack of some piece of data is reason enough to put off the decision and look for further validation.    ​

I believe there is a combination of data and experience hat goes into making a qualified decision.  ​Validation of whether a decision was right or needs to be tweaked happens after the fact.  There is never going to be enough perfect data to make a right decision 100% of the time, so trust your experience and the data you have.