%201.svg)
Jessie Inchauspé
Biologist
In order to understand the user challenges, I conducted one-on-one in-depth interviews with 9 regular shoppers (with low or medium nutritional knowledge), a focus group with 2 food science graduates, as well as a total of 2 hours on-site observations at 4 different supermarkets in Melbourne.
🎯 How do general consumers make healthy food choice? Any strategies?
🎯 What frustrations did they encounter when trying to eat healthier?
🎯 How about their perspectives on online and offline shopping experience?
🎯 How do the professional define healthy diet?
🎯 How do they make food choices?
🎯 What are the major challenges faced by the general public from their perspectives?
🎯 Their suggestions on the design of potential technology to improve the situation.
🎯 How do shoppers behave in supermarkets or groceries?
🎯 How do people interact with the commodities?
🎯 How is the environment of supermarkets like?
🎯 What do people look at when selecting food products?
.jpg)
.jpg)
While nutrition fact labels are an important reference for making healthy food choices, a lack of prior knowledge poses significant challenges for general consumers in effectively interpreting these panels, making healthy purchase decisions difficult. The research identified two main challenges faced by participants in interpreting nutrition information panels: 1. Uncertainty about impact of an unfamiliar nutrient on body & 2. Difficulty interpreting intake amount standards
Comparing products is a common behavior when shopping in supermarkets. From the interviews, 8 out of 9 regular shoppers stated that they compare products for various reasons, including health, taste preferences, allergies, and dietary habits. However, they mentioned that this process "takes time and effort" and that they still "make mistakes."
In addition, some consumers prioritize personal taste preferences over health considerations when making decisions. However, they also expressed a willingness to “select relatively healthier options from the ‘unhealthy’ foods.”
From on-site observations, shoppers' hands are often occupied with baskets, carts, or products, and they frequently need to handle items directly. This diminishes their motivation and willingness to use a mobile application that requires additional effort to operate while shopping. ("I will not scan every item on the shelves...it’s too time-consuming.")
To address the identified user challenges, the concept of NutriAR is proposed. NutriAR is an AR solution compatible with both HoloLens and mobile phones, designed to assist people in making healthier food choices. Its core features include an AR nutrition label and a comparison mode.
Access Latest Prototype
Calculate health ratings and display results using icons, colors, and stars to facilitate easy and efficient information retrieval
AR components, compatible with HoloLens, display relevant details based on proximity to objects
Nutritional information is visualized with diagrams and explained in plain language, directly indicating what is good or bad
Struggle with interpreting nutrition information? NutriAR Integrates AI recognition technology and a comprehensive database, offering three different levels of detail to suit your needs.
When you're relatively far from the shelves, the system displays simple recommend or not recommend icons to help you quickly filter options.
As you get closer to a product, you'll unlock a more detailed view of its nutrient information. This includes an overall health rating and the levels of each nutrient, explained in plain language and diagrams that are easy for people to understand.
If you want to learn more about a specific nutrient, simply click on it. A window will pop up to explain how it impacts your body, with images supporting your decision-making process.
Simply mark one food as product A and another as product B, then click "Go to Compare." A table comparing the nutrients of the two products will be displayed to help you find the food that meets your needs. You can also switch between table and diagram views according to your preference.
Data is powerful and impactful. When you're about to finish shopping, simply scan your cart, and a report will be generated regarding the healthiness of the food you selected. This encourages and guides you towards making better selections.
NutriAR enhances your shopping with Hololens for hands-free, immersive AR and mobile compatibility for widespread accessibility.
AR integrates virtual objects into the real world. NutriAR, compatible with HoloLens, adapts product detail based on proximity, freeing your hands. The mobile version saves effort by avoiding physical interaction with items like large ice cream boxes in fridges.
Health is subjective, and individual diversity often goes unrecognized in current offerings. NutriAR addresses this gap by prioritizing your personal experience. By allowing users to input their personal data, NutriAR provides tailored suggestions and experiences based on your specific conditions and preferences.
Sketches were created with pencil and paper to explore how different features work and to be connected.
Based on the sketches, mid-fidelity prototypes and some variations on certain features were created on Figma.
View Data
The preference among users was clear for version A navigation (4 out of 4), as it provides "contents of each feature with texts," though "the naming can be improved" for better clarity.
Additionally, the main page was removed as it was deemed "unnecessary," enhancing ease of access for returning users to core features in supermarkets.
During the evaluation, 2 participants expressed a desire to emphasize certain nutrients they care about most, as it would help them "quickly and effortlessly find wanted information."
In the high-fidelity prototype, users can set their preferred nutrients of interest. The AR labels then emphasize relevant information using colors, making it easier for users to identify important details at a glance.
.svg){kind=spacer}
All participants identified an insufficiency on the clarity of current comparison view, expressing that “it is not direct or clear enough for me to see the differences.”
In the hi-fi prototype, information of two products are displayed within one form or diagram instead of showing them separately.
During the evaluation, questions about users' preferences regarding color modes and placement of attached AR components were raised. Participants highlighted the importance of context, such as the color of the background, in their responses. This underscores the need for the system to have context awareness capabilities.
In the high-fidelity prototype, both dark and light modes for the AR labels were created to accommodate different environmental conditions.
.png)
Participants expressed positive attitudes towards the features offered by NutriAR. Specifically, regarding the AR nutritional labels, participants commented, "It's information I am interested in."
When participants tested NutriAR in a supermarket, they experienced fatigue when trying to access information about products on higher shelves. The current design requires users to continually hold their phones up to scan products. Directly displaying information on the screen may be a more practical solution for mobile cases, enhancing usability and reducing physical strain.
Regarding the shopping cart report, participants showed a positive reaction to the visualized data, stating, "It’s interesting to see how many healthy or unhealthy foods I took." However, 2 participants expressed a desire for detailed information on where to find recommended foods. "You know, it’s often hard to locate a food in a supermarket - maybe some AR arrows to lead the way? That would be very helpful..."