Global Food Processor

Using machine learning to solve a $60M inventory loss problem

Woman looking at tablet with phone and cartoon chat icons

A closer look at how the emerging tech and analytics teams applied machine learning to Operational Technology, increasing product identification accuracy to over 90%

workers at a food processing plant, standing at a conveyor belt


One of the world’s leading food processors was losing millions of dollars annually because of an overly manual and error-prone process that resulted in a higher-than-tolerable instance of misidentified and lost inventory on the factory floor. Leaders knew the existing process was woefully inefficient and riddled with repeated patterns of human error. Nearly all agreed that significant change was needed. Namely, a much more efficient system that reduced the load on its factory staff while increasing the accuracy and reliability of the labeling process.






Data Architecture

Data Architecture

Change Management & Upskilling

Change Management & Upskilling



The Challenge

- What we were up against
Tech Republic logo and quotation marks
We’re trying to apply the most cutting-edge technology in order to derive new insights so that we can enable new ways of working.
VP, Emerging Technology & Analytics
frontline employees working at a poultry factory

The existing system was both manual and error-prone

Accurately tracking products as they move through production facilities across the country is critical; the consequences of misidentified or mislabeled products—typically a result of human error—are immediate, resulting in spoiled products and a lot of wasted money. Still, the identification and labeling processes used at most of the plant locations across the country were too dependent on human accuracy.

Typically, tracking inventory through this particular production facility is a largely manual process that relied heavily on human judgement before critical data was then entered into a system. What’s more, that process wasn’t always accurate. In fact, more often than not, it was wrong, which led to a number of unwanted outcomes, including both under and over-processing products. Each of these outcomes cost the company millions of dollars.

Any new solution would first need to pass an initial test

Together, we quickly zeroed in on a number of potential solutions that might help to mitigate the product identification and processing errors. Many of which utilized machine learning (ML) and computer vision (CV) to increase the accuracy and efficiency of the identification process. Still, each of these solutions were untested and unproven. So, we set out to quickly prototype and test in a real-world environment in order to prove or disprove the efficacy of such a solution before investing more fully. The goal was simple: to test and deploy the ML and CV models in a small, controlled fashion before a larger planned rollout across other processing facilities.

Spotlight —

For all involved, the challenge was difficult: How do we design an entirely new, restaurant training tool when: (1) the majority of the restaurant team members aren’t allowed to use their own mobile devices on the job, (2) there are only a few, shared training devices for each restaurant, and (3) Leaders across the country reported a mostly positive experience with printable materials?

We noticed a few other critical issues, too


There wasn’t any existing training data, so we had to create it from scratch

There’s rarely existing training data for most bespoke enterprise machine learning initiatives; this was the case in this instance, too. Most importantly, for this effort, there wasn’t an existing way to get an overhead view of the packed product inside the plant. Instead, we had to devise a way to capture that data ourselves. So, in short, we found a way to associate the timestamps from the weight scale with the timestamps from overhead streaming video to generate a base training data set. Lastly, we took the necessary step to validate every single image (since they were known to be imperfect) and balanced the training set so that we represented the various images equitably to ensure quality results.


We didn’t quite know how human staff would respond to machine augmentation.

We simply didn’t know how staff (those working inside the production facilities) would actually “take” to a new, tech-driven solution. What’s more, there’s not yet any well-defined rules or best practices to follow here. So, in order to give ourselves the best chance at success, we worked closely with staff to create a more efficient experience without the complexity one might imagine from a machine learning solution. Doing so both created a better solution from the start, but instantly formed the next round of training in order to “dial in” the model even further.


There were a lot of potential “gotchas” and unknowns when it came to deployment

It’s common practice to deploy transactional user software in the cloud, but it wasn’t that simple in this case. There wasn’t a straightforward method to deploy an intricate computer vision system—complete with live video streams, machine learning, and user interaction dialog—within a production facility. All involved ultimately evaluated a number of options, which, when coupled with some of the strategic aims of the effort, helped inform the construction of a truly modern combination of services that were used to bring the solution to life.


The Partnership

- How we tackled the challenge, together
Tech Republic logo and quotation marks
The driving goal of this effort—and our number one priority—was to find a way to correct the existing inventory loss issues. We knew there was some promise in using tech like computer vision and machine learning, coupled with the most modern infrastructure, to devise a solution that worked well within a really complex environment. Ultimately, the fusion of design, machine learning engineering, DevOps and production operations gave us a fighting chance.
Colin Shaw
Director of Machine Learning

Phase 01: Research

female manager with glasses talking to another employee, wearing hear nets

Timeline: 4-5 Weeks

Key Activities:

First and foremost, we had to understand the existing processes in workflows in each production facility. So, we set out for several “ride along” visits, carefully watching how different facilities each handled the same tasks. Additionally, we spent time with those who were actually doing the work, observing and asking questions along the way. These field visits and interviews, which were but a small part of our initial research, gave us enough of a starting point to quickly and confidently move forward.

Phase 02: Data Modeling & Prototyping

man, data worker, with curly hair and glasses working on a computer

Timeline: 3 Weeks

Key Activities:

Next, we made quick strides to develop a machine learning model so that we would be able to develop a working prototype that consumed real-world training data. First, we began collecting critical training data from some of the cameras that had been stationed at various locations inside of the product facilities. Second, we worked quickly to create multiple training sets while simultaneously validating their accuracy with several trained sets of eyes. Finally, through a bit of repeated trial and error, we were able to get to a reliable data model for prototyping purposes. After just three weeks, we had a functional model working at 90% accuracy.

Phase 03: Delivery & Upskilling

two men wearing hair nets holding tablets and paperwork in a factory

Timeline: 2 Months

Key Activities:

Lastly, we created a front-end application that was then connected to the training model so that all involved could showcase technical feasibility and accuracy to senior leadership. In effect, the front-end UI allowed us to show how the product identification process could be vastly improved using a combination of machine learning and computer vision, allowing plant workers to easily verify accuracy while focusing on higher-level tasks. Finally, as part of the handoff of the model and prototype, we spent several weeks working hand-in-hand with a variety of internal teams in order to train each on the various tools and skills needed in order to maintain and scale the solution to other production facilities.

The Solution

- What we created together
Tech Republic logo and quotation marks
We’re trying to apply technology everywhere we can as a company to drive down costs and drive up efficiency and business value.
VP, Emerging Technology & Analytics

A functional prototype that’s helping solve a $64m problem

Together, we quickly developed a machine learning model and accompanying front-end UI that allows production-plant staff to more efficiently and accurately label massive quantities of products so that they can be processed correctly. The functioning model produced a 90% accuracy rate after just three weeks of development, testing, and validation, which represents an improvement of more than 20% when compared with the existing, manual process.

In short, the model is able to very quickly identify a specific product type, the associated stock-keeping unit (SKU), and a variety of other associated data inputs, all of which are absolutely critical for accurate inventory processing. An automated scale then records the weight of the product before an operator simply verifies the correct weight and SKU number. The system is also able to more accurately detect flaws or impurities in either product or processing equipment, which has led to elevated food safety measures.

screenshots of meat and chicken product tracking app and people working at a chicken packing plant

screen shots of a product/chicken tracker

New Capabilities

  • Introduced machine learning into an overly manual and outdated process
  • Utilized existing infrastructure and hardware to drastically improve efficiency
  • Created near real-time access to product inventory data, more accurate reporting
  • Increased product identification accuracy to 90% (a massive improvement)
  • Allowed plant employees to focus their efforts on on higher-level tasks
  • Internal teams acquired new skills to allow for ongoing maintenance and upkeep
  • Increased confidence among senior leadership teams,  specifically operations leaders
  • Modeled how to work quickly and efficiently, shipping a prototype at scale
Spotlight —

The process and results gave senior leadership more confidence; specifically, they now had another “proof point” that objectively showed how more novel emerging technologies could be used to potentially solve a variety of critical operational challenges at scale. What’s more, the success of the prototype helped to unlock additional investment for similar initiatives.


- What we accomplished together
Tech Republic logo and quotation marks

Increased inventory accuracy to 90%, a 20% increase over the previous process

Solved a critical component of what amounted to a $64m inventory problem

Initial prototype and data model deployed to production facilities nationwide

Tell us about your next big idea.

Get more information on our machine learning case study.

We've worked with:
J.B. Hunt logoArcBest logoWalmart Logo
Tyson logo
UniGroup logoSam's Club logo
H-E-B logogulfstream logoHyatt logoChick-fil-a logoUnder Armour logoChevron logo