Models are a Reflection of Reality
The world is a complex place. People say that physicists can describe what happens all the way up to a single hydrogen atom – but get any bigger and you have to start making some serious assumptions. If you've worked with data, you know just how much information there can be for even a small sliver of your business. Building a complete model of how all the data in your organization is connected is simply impossible.
At the same time, all your data is still only part of the picture. What things is your business not considering about the rest of the market when making decisions? Is your company data still analyzed in a highly siloed way? How can you hope to make good decisions when huge amounts of relevant information are left out?
People and Decisions
Historically we have leaned on the fact that humans are remarkably good at pattern recognition. We see a small part of the picture and connect it to what we have already experienced. This characteristic of people helps us recognize faces in pictures, be more alert when we notice a possibly impaired driver on the road, or even decide how much inventory we should order of a new product to put on store shelves.
When people are asked to explain how they make these decisions, they usually point to a few pieces of information combined with a feeling. The data they say they use to make decisions is only ever part of the picture (which our team sees play out in client projects regularly). These issues explain many of the challenges when knowledge workers leave a position. They also can be a major stumbling block in data analytics projects as developers build tools meant to support or replace manual work.
Adding Machines to the Process
That gap between what people know and what they realize they know is where the right tools and approach can help. Trying to simply replace a human making decisions with a computer making decisions only works when the right data and priorities are given to that computer. Give that computer the wrong data or tell it to solve the wrong problem and the decisions it makes will be bad ones (Click here for an interesting post about a machine learning algorithm that tried to classify pictures of dogs as either a wolf or a husky).
However, making it easier for a human decision maker to access and interpret the right data is a much more manageable task. As people get more of the right information, they may even start to understand better how they are making those decisions and begin to improve. These are the harder to quantify but usually the most valuable benefits of a successful data analytics project.
This gets us back to the opening of this post. While models cannot reasonably include every aspect of reality to make the perfect decision, they can include the right elements to make a very good decision. How close that decision is to perfect depends on which things were left out and to some extent, luck. And ultimately, that’s ok. Our side view mirrors give us incorrect information about how close other cars may be, but they also warn us about the bias which lets us make the right decisions anyway.