Perception Test

Before starting the qualitative research, a Perception Test (Annex 3) was engineered. The goal of the test was to verify the user’s awareness about the calorie content and weight of food items and to find if this has an impact on the lack of progress that users mentioned in the quantitative research.

Three different tests were performed. A food item was placed in front of the participants, and they had to guess its weight and calorie content.

The tests were made with a block of cheese (Fig. 1), a plate of mashed potato (Fig. 1) and an apple (Fig. 1). Before the tests, it was established that any answer between 10% more or 10% less from the actual answer would be accepted as correct. In total, 71 people participated in this exercise.

Figure 1. Items used during the Perception Test.

Results

The results (Fig. 2) showed that 22.53% of the calorie guesses stayed within the established accuracy margin while 15.49% of the weight guesses remained inside that margin.

Figure 2. Participants’ accuracy.

 Block of CheeseMashed PotatoApple
Touching AllowedYesNoYes
Multiple ChoiceNoNoYes
Number of Participants213020
Weight180 grams285 grams133 grams
Calories702 kcal233 kcal70 kcal
Weight (± 10%)4.75%15%35%
Calories (± 10%)9.52%23.33%35%

Figure 3. Perception Test data.

In conclusion, participants were not able to have an accurate or an approximate accepted guess when comes to weight and calorie contents of food products, but when having a visual aid together with a multiple choice, their accuracy increases substantially.

Contextual Inquiry – Deep diving into information

Contextual Inquiry was the user research method chosen because it gives realistic data as it uses a natural setting to gather behavioural insights around the use of a product, in this case, MyFitnessPal.

A master-apprentice technique will be employed (Beyer and Holtzblatt, 1998). By design, the context inquiry makes it possible to reveal tacit knowledge. Other relevant aspects are that the information that emerges tends to be accurate and it produces highly detailed data. The goal was to observe commonly used features: adding food to their diary, monitoring progress, managing calorie intake and updating goals. The contextual inquiry was conducted in a place where users typically interact with the product. The results will be used to define requirements. 

Six MyFitnessPal’s users, matching the proto-persona defined in the questionnaires, were sourced.

Along with the contextual inquiry, a semi-structured interview (Annex 4) was performed to gather information necessary to triangulate the information from the questionnaires. Before the contextual inquiry, users were informed about the project, what was the goal of that exercise and were asked to sign a consent form (Annex 5). 


Figure 1. One of the contextual inquiries performed with a MyFitnessPal’s user.

The videos, audios and notes from the contextual inquiries can be observed in Annex 6.

The main points were added in virtual Post-Its on an online tool called Boardthing (Fig. 2)

Figure 2. Overview of Boardthing with data from contextual analysis.

Afterwards, the Post-Its were grouped by themes (Annex 7), and the topics that raised were discussed:

  • Trust issues with data in MyFitnessPal
  • Observations of use
  • Desired Features
  • Used Features
  • Location of use & Time of use
  • The information being tracked
  • Motivations behind usage
  • The use of scale to measure food weight
  • User abilities
  • Useless Features
  • Pain Points

No major issue surfaced during the Contextual Inquiries – users were satisfied with the current solution. Despite the attitude of the users when reviewing the contextual inquiry, all users struggled to locate and demonstrate features in the app. Another point was that users mentioned desired features that they would like to have within MyFitnessPal but these features already existed in the app (Fig. 3). Over a period of time, long time users were demonstrating the same behaviour as novice users.

Figure 3. Desired features mentioned by users already available in the app.

Another example is that users responded to the questionnaire that they have more long-term goals than short-term goals but the contextual inquiry revealed that users have more short-term goals – often initiated when a life or social event appear on the horizon.

References

Beyer, H. & Holtzblatt, K. (1998). Contextual Design: Defining Customer-Centered Systems. San Francisco, CA: Morgan Kaufmann Publishers.

Annex

Annex List

Annex 1 – Questionnaire
Annex 2 – Questionnaire Responses
Annex 3 – Perception Test
Annex 4 – Contextual Inquiry and Interview Questions
Annex 5 – Consent Forms
Annex 6 – Contextual Inquiry Videos, Audios and Notes
Annex 7 – Contextual Inquiry Board
Annex 8 – Demographics Questionnaire
Annex 9 – Demographics Questionnaire Results
Annex 10 – Demographics Questionnaire Analysis
Annex 11 – Usability Test – Test Plan
Annex 12 – Usability Test – Videos
Annex 13 – Usability Test – Notes
Annex 14 – Usability Test – Information Sheet & SUS Questionnaire for Participants

Annex 1 – Initial Questionnaire

https://docs.google.com/document/d/1sJJHFhI9ODsvfH4w3P7nsJjWXcTR24leeknmOU_QugE/edit?usp=sharing

Annex 2 – Questionnaire Responses

https://drive.google.com/file/d/19duq5bNvtl2Y1VdkHj2XLKi9d6R7WYlB/view?usp=sharing

Annex 3 – Perception Test

https://docs.google.com/document/d/1iIR95IvumkyupjwnwmIna7oPLXTFwtvPMOERQGhoWtk/edit?usp=sharing

Annex 4 – Contextual Inquiry and Interview Questions

https://docs.google.com/document/d/1CvzVt25qH3SXnkBvCcx4Ins6nsyYtTY5xuzvcyI2yKo/edit?usp=sharing

Annex 5 – Consent Form

https://docs.google.com/document/d/16JLvZIJtgOdH0eJbnkQy1k8BkBgn8hTBILAkf20sP3I/edit?usp=sharing

Annex 6 – Contextual Inquiry Videos, Audios and Notes

  • User A – https://youtu.be/WsM9nlHU9D0
  • User A – https://soundcloud.com/leandroiwai/user-1-contextual-inquiry
  • User B – https://youtu.be/oLHPJutHcKM
  • User C – https://youtu.be/6lpuTsajAZQ 
  • User C – https://youtu.be/G1amrDnfuT4
  • User C – https://youtu.be/5kzKQtK0Slk
  • User D – https://www.youtube.com/watch?v=s9AsybrvBpw
  • User C & D Notes
    https://docs.google.com/document/d/1UezWOGTQkdCKxyIoqKqmbkc31P9NLGI-okaJvVew6kI/edit?usp=sharing
  • Users A & B Notes
    https://docs.google.com/document/d/1T5T0iF0LnuOBaXtnpWMoCl-pkRCCHw6iiWO31ouhF64/edit?usp=sharing
  • Users E & F Notes
    https://docs.google.com/document/d/1BEZn2qjNbB1bCQEY0bUuNbpRqGEnHJQ7j7d-9JMbiBc/edit?usp=sharing

Annex 7 – Contextual Inquiry Board

https://boardthing.com/board/5a8dc9fef1a13978424e4a24

Annex 8 – Demographics Questionnaire

https://drive.google.com/open?id=1tTjDPGuCY5zUJxwckdYBJ_d5CFfxrCtz

Annex 9 – Demographics Questionnaire Results

https://drive.google.com/open?id=1SsWzKPXjY-YJZ8yAdF76gP1QlDF_qr-y

Annex 10 – Demographics Questionnaire Analysis

https://drive.google.com/open?id=1PQw3db17JpWRfzphHuvitvnwcW8toGh9-l9FNz9K6H4

Annex 11 – Usability Test – Test Plan

https://docs.google.com/document/d/1akJ-0LKHmGqDi4Hcq0T-i6AYiebgtReQ-yMbFMe_dsY/edit?usp=sharing

Annex 12 – Usability Tests Videos

  • Test 1 – User A – https://youtu.be/bCBUgsL581k
  • Test 1 – User B – https://youtu.be/neNAR-IPkyg
  • Test 1 – User C – https://youtu.be/7E5WwYTwer0
  • Test 1 – User D – https://www.youtube.com/watch?v=D-BJCTWWsqE
  • Test 1 – User E – https://www.youtube.com/watch?v=tsQVyT2aI_8
  • Test 2 – User A – https://youtu.be/-Gy-D2SRbO8
  • Test 2 – User B – https://youtu.be/_c113cEAj_4
  • Test 2 – User C – https://youtu.be/CTIPmkwTJUU
  • Test 2 – User D – https://youtu.be/x2oxZsbYdMo
  • Test 2 – User E – https://www.youtube.com/watch?v=tsQVyT2aI_8

Annex 13 – Usability Tests Notes

  • Test 1 and 2 – Errors table, timings and SUS Scores – https://docs.google.com/spreadsheets/d/1DZjIggxmuuXDXz4bsS_5n1Lzw9F3L7_k88KB4A5Jq4s/edit?usp=sharing
  • Errors Notes Test 1 & 2 – https://docs.google.com/spreadsheets/d/15wcnBVN7FV3othoVrcSRmFY40zkX-UIgt0YltCi15Cs/edit?usp=sharing
  • SUS Scores Test 1 & 2 – https://docs.google.com/spreadsheets/d/1zcWpjvTQestUa-gGeH6KFeR-gkn2nJ1lvCILZ6qi6TQ/edit?usp=sharing
  • Test 1 – Test Feedback Summary – https://docs.google.com/document/d/1v7kQT1q2mPylTZ8zZY5cQvX31w8ZDJdQom6NIxchPOY/edit?usp=sharing
  • Test 2 – Test Feedback Summary – https://docs.google.com/document/d/1TPEUyCmdpqkJOVOvCBX7HQp0itR9dTgr-mTkZUPlWYo/edit?usp=sharing

Annex 14 – Usability Tests – Information Sheet & SUS Questionnaires for Participants

Version 1: https://docs.google.com/document/d/1f5h66mhJ9DCJe94y_UjK8unRQFRfm9OGbRcyjPZMRUc/edit?usp=sharing

Version 2:
https://docs.google.com/document/d/14nfQqqJCZ1ILMcEpM7fSkV2NDwQB4Ozyf4NU9kuer4s/edit?usp=sharing