Super Hosokawa lowers food waste and boosts sales by sharing demand forecasts across the entire supply chain with Azure Databricks

Businesses cause over half of food waste, bullwhip effect is a major factor

In financial year 2021, Japan wasted 5.23 million tons of food. Business-related food waste accounted for over half of this figure. Although food waste from the business sector had been decreasing since 2015, it began increasing again slightly in 2021. For Japan to halve food waste by 2030 and fulfill an SDG on responsible food production and consumption, further measures are crucial.

In a project commissioned by the Japanese government to achieve this goal, seven companies participated in a trial in Oita and Fukuoka prefectures to reduce food loss by linking data between supply chains. These participants were Imamura Shoji, which created demand forecasting models and built a system; Super Hosokawa, which supplied a retail presence for testing; Kyushu CGC, a supermarket cooperative; three food manufacturers (Asahi Shokuhin, Kyuichian, Fujimitsu); and Japan Research Institute, intermediary for the government ministry in charge. The experiment verified the benefits of sharing and linking demand forecasts based on ID-POS data between manufacturers, wholesalers, and retailers for food logistics.

According to Takuto Hayashi, who played a pivotal role in the trial as Senior Vice President of Imamura Shoji, “Food logistics is split into manufacturing, wholesale, and retail, with each sector having its own demand forecasting data, which causes the bullwhip effect.” The bullwhip effect is a phenomenon in which downstream demand fluctuations in the supply chain amplify as they move upstream.

“For example, if a consumer needs one case of something, retailers stock two cases to avoid opportunity losses,” says Hayashi. “To promptly respond to an order for two cases, wholesalers stock three cases, and manufacturers make four cases to accommodate the order for three cases plus some leeway. If we could accurately perceive true demand using AI or other technologies and share it through the supply chain, we could suppress the bullwhip effect and reduce food waste for businesses.”

“However,” counters Super Hosokawa CEO Tadashi Hosokawa, “if we reduce waste loss to zero, sales volumes will steadily drop, which will impair the growth potential of businesses.” The philosophy should be to minimize waste while allowing a certain extent of speculative losses to tap latent demand, he says.

“I learned this trial would enhance the accuracy of demand forecasting using AI,” Hosokawa continues. “So I requested the ability to properly control losses by cutting waste loss while also uncovering latent demand.”

A 60-pattern model to order food prone to waste using demand forecasts two days ahead

A project oriented toward a trial began in August 2023 with the objectives of all relevant parties aligned. The first step was to build a demand forecasting system that could fully leverage Super Hosokawa’s big ID-POS data.

To do so, Hayashi chose Azure Blob Storage, Azure Databricks, and Snowflake running on Azure. To create a demand forecasting model, Super Hosokawa ID-POS data was sent to Azure Blob Storage via Web-EDI, aggregated on Snowflake, and analyzed with Azure Databricks. Hayashi explains the choice of this configuration below.

“I originally worked in the sales department of Mitsubishi Foods, where I performed duties heavily dependent on my own experience and intuition. When I joined Imamura Shoji in 2021, I realized that data was the key to the future of logistics. I took part in a three-day Imamura Shoji training program where I learned basic data analysis methods with Azure Databricks and mastered them over the next three months. Azure Databricks is easy to learn without much data analysis experience and integrates easily with many other services. As an Azure managed service, system development and operation are straightforward and its cost of a thousand to tens of thousands of yen per month is immensely appealing.”

Hayashi created customer personas using Azure OpenAI Service and a 60-pattern demand forecasting model in just four months. Meanwhile, in September 2023, the team used data to verify and enhance retail locations (shelf arrangement) to prepare for the trial.

The focus was on the tofu, fish paste products, and fried food sections of Super Hosokawa's three stores (Manda, Okishiro, Buzen) in Oita and Fukuoka prefectures. According to Hosokawa, the team selected these products because, “They are delivered to stores every day and more prone to waste due to their shorter expiration dates.” Some of these products were also the subjects of demand forecasts in the trial.

Hayashi says, “We started by verifying and improving shelf arrangement because correct shelf placement is often different in practice than in theory. For example, retailers often think a high-priced product won’t sell next to a low-priced one, but a lot of data-based analysis shows the opposite.”

“We analyzed Super Hosokawa’s big data and made improvements, such as placing 188-yen tofu next to an 88-yen private brand,” Hayashi continues. “Store managers initially said high-priced products wouldn’t sell with this shelf arrangement, but President Hosokawa assured them, and at one outlet in November 2023, monthly sales increased by 34,000 yen before the trial had even started.”

Developers also created a mechanism to return two-day-ahead demand forecasts, made with a data analysis platform built first, to Super Hosokawa. For the trial, Super Hosokawa had to place orders to manufacturers based on these forecasts. Following these preparations, the trial began in January 2024 and produced numerous insights within just one month.

Proving lower waste rates and greater sales, finding more partners to solve the dark secret of logistics

The first notable aspect of the trial was its accuracy in forecasting demand. Accuracy was measured using three patterns: orders placed by humans, demand forecast model only (no orders), and orders placed by humans using the forecast model. Orders placed by humans returned an average of 249.8 errors per day across the three stores. This figure dropped to 194.1 errors using the demand forecast model and 181.2 with humans using the demand forecast model.

Human accuracy in the Manda store, in which a manager with 20 years of experience placed orders, was slightly higher than the forecasting model. The Buzen store, in which a part-time worker with three years’ experience placed orders, roughly matched the model. In the Okishiro store, with orders placed by a part-time worker with six months’ experience, the model was more accurate.

“Accuracy equivalent to a part-timer with three years of experience is higher than I expected,” says Hosokawa. “Since AI forecasting accuracy can continuously improve through learning, there’s plenty of potential for AI to place orders on behalf of humans. Our system will reduce store ordering workloads, and we can expect full automation in the near future.”

So, what benefits can we anticipate from reduced waste rates? First, the overall waste rate of tofu and fried products in stores was 0.52 percent. In contrast, the waste rate of these products fell to 0.20 percent when ordered by combining demand forecasts with human ordering. Ordering fish paste products (overall rate waste of 0.52 percent) by combining humans and demand forecasts cut the rate to 0.13 percent. This represents major savings in each category. 

We can also expect massive drops in manufacturing waste rates. For example, calculations show that ordering two days ahead will slash the waste rate of two Kyuichian company products included in the trial from 10 percent to zero. Moreover, because planned production will become more manageable, emergency response costs associated with production shortages and shipping delays will fall emphatically.

There are even more advantages. Surprisingly, sales also benefited.

“Over the trial period, supermarket sales were around 20 percent below the previous year, partly because food vouchers issued during the COVID-19 pandemic disappeared,” explains Hosokawa. “However, items that were part of the trial maintained their sales volumes, which was effectively a 20 percent sales increase. I believe this was the result of creating demand forecasts while considering latent demand.”

Hosokawa wants to drive the adoption of similar initiatives by offering proposals to other supermarkets in the Kyushu CGC cooperative and transforming logistics by advancing demand forecasting. He began making proposals to other companies in August 2024, he says. Meanwhile, plans for a new system are underway. These include operating various generative AI models with Azure OpenAI service and other solutions; using Azure AI Studio to manage generative AI models; and integration with external automated order and receipt systems.

“Until now, businesses have kept POS and ID-POS data private, strongly believing that retail was the king of logistics,” says Hosokawa. “But this is no longer the case. Retail waste is the tip of the iceberg, with manufacturers also creating a considerable amount. To eliminate this dark secret of domestic logistics, the entire supply chain must seamlessly connect. We’ll rely on Microsoft for further technical support to achieve this.”