A Case of Bad Operations Misleading Your Data

During a recent visit to the grocery store I stood in the checkout line, placing my items onto the conveyor belts as fast as the cashier could scan my items.  As I watched her scan one of my items and type into the keyboard, I bit down on my lip. I was holding back my unsolicited feedback, as I anticipated that it likely wouldn’t be welcomed. But that didn’t stop my mind from fixing on a broken operational process that I just witnessed and my thoughts of the implications of bad data across the company.

The Back Story

I’m a bit of a data nerd. I’ve been writing SQL for more than 25 years and I’ve spent the last 15 years leading analytics teams at companies including Amazon, eBay, VMware, and more. Data and operations are so ingrained into my brain that I can’t not think about data. In fact, I actually enjoy going to the grocery store because I get to look at prices, promotions, and coupons. But on this visit there was a problem, which started with a couple of cups of yogurt.

I had gone to the store to pick up a few random items and came across my favorite yogurt. It was strawberry flavored. Given my lack of ability to do things with moderation, I bought 6 containers of them. But I ended up putting one back because I wanted to try the cherry flavor. When I got to the checkout line and started putting my items on the conveyor belt, I took a closer look at the containers. At first glance, they all looked the same.

Both flavors had red and white labels. Both were the same size. Both had an image of fruit, and both of the images were red. What was not obvious to me was that one of the containers had an image of a cherry and the others had images of strawberries. The cashier didn’t seem to notice either. Instead of scanning each item one by one, she scanned the first one that she saw and then quickly typed a 6 on the keyboard. This was the problem that I became fixated upon.

What the cashier did was innocent and in a perfect situation, what she did might actually seem like a good practice. Visually grouping items and typing the quantity into the keyboard was faster than scanning each item. But this single operational process just threw off metrics across the entire company. While it’s just a tiny yogurt and one event, the implications can be far reaching and snowball quickly. Here’s what was likely impacted.

Sales Revenue and Profits

The first and likely most obvious issue is that the sales data is incorrect. Instead of correctly capturing the sale of 5 strawberry yogurts and 1 cherry yogurt, the sales data was captured as 6 strawberry yogurts and 0 cherry yogurts. Since the price was the same for both flavors, the sale revenue wasn’t impacted. But what about profits?

If the wholesale price that the grocery store paid wasn’t the same for both flavors, there may be an impact. Also, had one of these items been on sale, the revenue and profits could be negatively impacted. It’s also possible for revenue and profits to be positively impacted by accidentally overcharging the customer.

Inventory

Since inventory data is directly tied to the sales data, the inventory data is now inaccurate as well. The inventory data still says that there’s one additional cherry yogurt on the shelf and one fewer strawberry yogurt. Again, this probably doesn’t seem like much to worry about because we’re talking about a tiny yogurt cup. But what if we were talking about a big screen TV or a gaming console where there is a small quantity on hand. 

This is one way that you, as a customer, encounter situations where you ask a clerk if an item is in stock, and the clerk says, “the computer says there’s one in stock”. They proceed to check the back room for what feels like an hour, only to come up empty handed. It’s a frustrating customer experience. And these small frustrations shouldn’t be pushed under the rug. Over time, these small frustrations erode customer trust and can slowly and subtly have a negative impact on future sales. 

Customer Preferences and Personalization

Almost every company is collecting data about you in an effort to better understand you. In the case of retail companies, they are collecting and analyzing data about you so they can make recommendations with the hope of getting you to spend more money. As powerful as these machine learning recommendation engines are, they can be sensitive to even the smallest detail, such as incorrectly scanning my yogurt.

In my situation, a recommendation system likely saw that I like yogurt. The system also saw that I like the flavor, strawberry. Unfortunately, the system didn’t see that I possibly like the flavor of cherry. I say, “possibly” because this was a one-time occurrence for me purchasing something with a cherry flavor. This operations error and related data error is now a missed opportunity. 

In a perfect situation, the machine learning model would use my data to make new recommendations, such as alerting me to a sale on fresh cherries. But because my data wasn’t properly captured, I’ll never get this alert. And this missed opportunity is potentially missed revenue for the company. But what if the cashier had scanned my items incorrectly, but in a different way?

Had the cashier grabbed the cherry yogurt instead of the strawberry yogurt, she would have inadvertently told the system that I had a strong preference for cherry flavor, which I don’t. Instead of the system recommending strawberry, which is my actual preference, the system would start recommending cherry. Not only would this be a missed opportunity to suggest strawberry items, which are aligned with my preference, but it could cause frustration for me, the customer, because I don’t want to see recommendations for cherry flavored items.

You’ve probably experienced similar situations as a customer. I’m referring to the situations where you clicked on something that you had no intention of buying. Now the recommendation system thinks you have a preference for the item. You’re bombarded with advertisements for something that you don’t even like, and selecting the unsubscribe or uninstall button on an app is only a click or two away.

Dashboards and Reports

Not only is the data wrong in the database and in the recommendation engine, but all of the related dashboards and reports are wrong. Again, this might not seem like a significant issue, but it can be.

This was the case for me when I was the Director of Analytics at GameStop. I had a report that was telling me that I had over 800 units of inventory on clearance in each of our stores. When I looked at my sales reports, something didn’t feel right. My intuition led me to physically go to the store to see what the displays looked like. When I got to the store, I didn’t find 800 units like the report said. I found about 50 units. Obviously something was wrong, and the issue was related to the operational process, not bad data or data pipelines.

Had it not been for the visual inspection, we would have kept lowering prices in an effort to increase sales, but it wouldn’t have mattered. We could have dropped the prices to $0.01 and we wouldn’t have sold any more units because the inventory wasn’t actually on the sales floor where it should have been.

Conclusion

This specific situation seems small and innocent, but when extrapolated across a company and to a large set of customers, there are many potential downstream implications. Unfortunately, no amount of quality assurance checks or automated tests across data pipelines would have prevented bad data from leaking into the sales and inventory datasets, or into the machine learning models.

This is why business acumen and an understanding of processes is critical when working as a data analyst. Without this understanding, you’ll be viewing your data at face value, as it’s displayed in the database or on existing reports. But you shouldn’t. 

If my 25 years’ of experience in tech has taught me anything, it’s that every company has inaccurate reports that are being used every day, without anyone ever noticing. But with a keen eye for details and a solid understanding of operational processes, you can reduce the amount of bad and misleading data in your organization.

Brandon Southern, MBA, is a Sr. Director of Analytics and the founder of Analytics Mentor. With a 25-year career in the tech industry, Brandon has excelled in diverse roles encompassing analytics, software development, project management, and more. He has led analytics teams at Amazon, eBay, VMware, GameStop, and more.

You can learn more about Brandon and Analytics Mentor at http://www.analyticsmentor.io/