Is the Data Lake Half Empty or Half Full?

Is the Data Lake Half Empty or Half Full?

Alex Ring and Daryl Atkins
Monday April 02, 2018

The sea of data is more prevalent in business now than it has ever been before. Deloitte completed their first ever Analytics Advantage survey of some of the world’s leading companies. This is what they had to say about the use of analytics:

“The best is yet to come. Ninety-six percent of respondents feel that analytics will become more important to their organizations in the next three years. Two reasons there is plenty of room to grow: a great deal of data is still not used for decision-making; and many organizations have only rudimentary analytical technology.

The phrase that resonates most with me as a Product Owner and Business analyst is “a great deal of data is still not used for decision-making”. Although it is true, this is a hard pill to swallow. The data available to organizations today according to Forbes, “At the moment less than 0.5% of all data is ever analyzed and used.” Organizations that aren’t taking advantage of their own data and leveraging public data are doing themselves a disservice.

Utilizing a current customer’s data, we reviewed a common problem they struggle with in their space, shrink. The customer supplies consigned inventory to more than 40k stores across the US and Canada. One may not think that taking a simple phone charger off the counter or pocketing a 5-Hour Energy drink is a big deal because corporations have millions, right? Wrong. If just 1 5-Hour Energy drink (@ $1.00 cost for example) was stolen from each store serviced every day, that would create over a $14 million-dollar loss. We are only talking about one item creating this type of strain on the business. By using analytics, we were able to take a look at this behavior and determine if the customer's stores in the Denver market were susceptible to higher shrink in higher crime areas, and if so, what patterns were there in what was being taken?

Now that we identified a potential issue and something to correlate it with, we needed data. Lucky for us, many cities within the state of Colorado offer up their data for free. Anything from public transit users, tree placement within parks, commuting, and even their crime statistics. We were able to extract a healthy data set with all crime in the city of Denver and then trim it down removing crimes that weren’t typically related to theft or public disorder in the city. This data was used to create a volume heat map of the city to reflect higher crime areas by latitude and longitude.

Alex fed ex day image

Shrink is distributed evenly across areas of the city with high crime just as it seems to dissipate in the less crime dense area as does the volume of stores. Unfortunately, we are unable to conclude based on the limited data we have that there is any correlation between high crime density and store location. Although we would have liked to prove the hypothesis to be true, we did identify a very interesting set of information that would create savings for the customer. There are six items that are sold in the Denver market that incurred over 100% shrink during the time our data was collected. It would be in their best interest to eliminate these products all together from the market which would generate a savings of over $75k a year. It is safe to assume this analysis could be applied to other markets for significant savings (estimated at one million dollars) for 2018.

As we have seen throughout this project, there is a sea of data that is available for businesses to make decisions. With over 1.2 Million Terabytes of data available for analysis, aggressive growing organizations would be foolish to not capitalize on their ability to make decisions for their business using the data at their fingertips. Our clients will continue to be able to make decisions down to a market level which will help sharpen their competitive edge and allow them to continue their expansion in the C-Store space.

Mashey logo
Mashey is a next generation data and analytics consultancy that designs and implements modern strategies that help transform companies into data-driven and data-informed teams. Through business intelligence, data warehousing, and data science, Mashey provides companies with the advantages of veteran experience and startup agility.