Food waste is an important problem in today's society, with several negative impacts such as waste of
valuable food, waste of natural resources used to produce it and the cost of organic waste management.
Once prepared, food is often not fully consumed leading to a large amount of wasted resources very evident
especially in work and school canteen settings.
The goal of this project was to develop a computer vision software capable of scanning a canteen consumer's
food tray at the end of a meal to estimate the amount of leftovers for each type of food.
Such software is able to analyze pairs of images: one of the tray before the meal and one at the end of the meal.
From the first image, the system will recognize the various types of food ordered, keeping track of the initial
quantity of each food; at the end of the meal, the system must analyze a new image of the tray, recognizing
which types of food are still present and in what quantity
Example of the pair of images to be analyzed. The first, the customer's tray before the meal, the second one
the tray after the meal with some food leftovers.
The software has been developed taking into account a provided benchmark dataset.
It consists of 8 different trays of food, each containing a first course, second course,
side dish, and possibly salad and bread. For each tray, a “before” image is provided with the state of the food
immediately after placing the order, and several “after” images with the presence of leftovers in the tray
Leftover computation
To compute the estimation of the leftover quantity the system compares the “before
meal” and “after meal” images to find which food among the initial ones was eaten and which was not. The
leftovers quantity is then estimated as the difference in the number of pixels of the food item in the pair of
images.
The software should be Robust to various conditions:
It should recognize all the food on the tray, even food items that are not on a plate (e.g., fruit, bread);
it should ignore any non-food object on the tray (e.g., smartphone, badge);
it should segment different food placed on the same plate.
Output Example
In the first image Food recognition of a tray (each color
corresponds to a specific plate). In the second one Food segmentation of the same plate (to
compute food quantity).
Performance analysis and evaluation
For measuring the system performance, we implemented the following metrics:
In the following images the portions in blue represents the food segmented by the algorithm, the parts in green are defined by the ground trouth in the dataset
and the cyan portions are the intersection of the other two.
