Reflection

This project was a huge learning curve for all researchers in the team. At the initial stages, it was defined that this would be a consultancy work, the idea was to treat as one of the health & fitness apps approached us to write a report. The plan worked.

Weeks before starting this project, a report about user research methods guided the team to take right decisions when determining what were the user research methods necessary to achieve the goals of this project. Even with the correct research methods, the lack of a holistic view of the whole project meant that the questionnaire failed to answer questions about the target audience. The solution was to send another questionnaire focused solely on MyFitnessPal users. The mistake helped the team to consolidate the knowledge it had about the users, and the Persona was constructed with a solid foundation.

Another relevant aspect of the user research for this project was the triangulation of information. This project, because of its nature, proved that user behaviours and attitudes can be different. Users mentioned that they have long-term goals, but in reality, they have short-term goals triggered by life events. Users also indicated the ease of use of MyFitnessPal, but the contextual inquiry detected that users have trouble performing simple tasks or finding functionalities within the app.

Working remotely proved challenging in some aspects, which were not thought they could be problematic at first. Online tools like Boardthing allowed the team to collate and collaborate. The Skype calls were crucial to enable the team to talk over elements on Boardthing. This process proved that the user research can be done working remotely. On the other hand, prototyping proved to be a different situation, the lack of collaboration tools within Sketch meant that there was a disconnection between the talks and the screens produced. Another prototyping tool might have been better for a team where its members are far apart. Besides that, the team members were happy to have learned Sketch, I included. As Sketch is a leading prototyping tool, the decision to go with it was probably overlooked.

The screens were designed following a thought process that was not correct. Users wanted to add ingredient by ingredient, the same way they operate on the current MyFitnessPal. Over time this mental model could change, but that was out of the scope for this project. Because of the limitations of the prototyping, it was difficult to predict every interaction the user might have with a chatbot. The next step for this project would have been to give the user interaction with a fake chatbot via Whatsapp, one of the researchers would control the other end. This step would allow better insight into the user’s mental model.

Working with users interacting with chatbots was a vast unknown for the team. Stripping down the UI did not mean less work, it meant more research in areas that were entirely new for everybody in the group.

Contributions

Looking back at the project, everybody in the team contributed in all aspects of it; everybody participated, opinionated, discussed and interviewed users. These discussions allowed us to explore different areas and to view from different angles.

A participation scale was designed (Fig. 1) and the scores were assigned to each area of the project (Fig. 2).

RatingAvaliationNotes
1No ParticipationMy participation was non-essential for this part of the project.
2Poor ParticipationMy contribution was below of the efforts of my teammates.
3Medium ParticipationMy contribution was at the same level as my teammates.
4Good ParticipationMy contribution was important for this part of the project.
5Excellent ParticipationMy contribution was essential for this part of the project.

Figure 1. The participation scale.

AreaParticipation Rating
Quantitative Research5
Perception Test5
Qualitative Research4
Heuristic Evaluation5
Information Architecture3
Problem Statement4
Demographics5
Personas3
Task Analysis4
Scenarios3
Storyboards3
Competitive Analysis5
Prototyping3
User Testing4
Redesign5

Figure 2. The participation scale was given to each area of the project.

Annex

Annex List

Annex 1 – Questionnaire
Annex 2 – Questionnaire Responses
Annex 3 – Perception Test
Annex 4 – Contextual Inquiry and Interview Questions
Annex 5 – Consent Forms
Annex 6 – Contextual Inquiry Videos, Audios and Notes
Annex 7 – Contextual Inquiry Board
Annex 8 – Demographics Questionnaire
Annex 9 – Demographics Questionnaire Results
Annex 10 – Demographics Questionnaire Analysis
Annex 11 – Usability Test – Test Plan
Annex 12 – Usability Test – Videos
Annex 13 – Usability Test – Notes
Annex 14 – Usability Test – Information Sheet & SUS Questionnaire for Participants

Annex 1 – Initial Questionnaire

https://docs.google.com/document/d/1sJJHFhI9ODsvfH4w3P7nsJjWXcTR24leeknmOU_QugE/edit?usp=sharing

Annex 2 – Questionnaire Responses

https://drive.google.com/file/d/19duq5bNvtl2Y1VdkHj2XLKi9d6R7WYlB/view?usp=sharing

Annex 3 – Perception Test

https://docs.google.com/document/d/1iIR95IvumkyupjwnwmIna7oPLXTFwtvPMOERQGhoWtk/edit?usp=sharing

Annex 4 – Contextual Inquiry and Interview Questions

https://docs.google.com/document/d/1CvzVt25qH3SXnkBvCcx4Ins6nsyYtTY5xuzvcyI2yKo/edit?usp=sharing

Annex 5 – Consent Form

https://docs.google.com/document/d/16JLvZIJtgOdH0eJbnkQy1k8BkBgn8hTBILAkf20sP3I/edit?usp=sharing

Annex 6 – Contextual Inquiry Videos, Audios and Notes

  • User A – https://youtu.be/WsM9nlHU9D0
  • User A – https://soundcloud.com/leandroiwai/user-1-contextual-inquiry
  • User B – https://youtu.be/oLHPJutHcKM
  • User C – https://youtu.be/6lpuTsajAZQ 
  • User C – https://youtu.be/G1amrDnfuT4
  • User C – https://youtu.be/5kzKQtK0Slk
  • User D – https://www.youtube.com/watch?v=s9AsybrvBpw
  • User C & D Notes
    https://docs.google.com/document/d/1UezWOGTQkdCKxyIoqKqmbkc31P9NLGI-okaJvVew6kI/edit?usp=sharing
  • Users A & B Notes
    https://docs.google.com/document/d/1T5T0iF0LnuOBaXtnpWMoCl-pkRCCHw6iiWO31ouhF64/edit?usp=sharing
  • Users E & F Notes
    https://docs.google.com/document/d/1BEZn2qjNbB1bCQEY0bUuNbpRqGEnHJQ7j7d-9JMbiBc/edit?usp=sharing

Annex 7 – Contextual Inquiry Board

https://boardthing.com/board/5a8dc9fef1a13978424e4a24

Annex 8 – Demographics Questionnaire

https://drive.google.com/open?id=1tTjDPGuCY5zUJxwckdYBJ_d5CFfxrCtz

Annex 9 – Demographics Questionnaire Results

https://drive.google.com/open?id=1SsWzKPXjY-YJZ8yAdF76gP1QlDF_qr-y

Annex 10 – Demographics Questionnaire Analysis

https://drive.google.com/open?id=1PQw3db17JpWRfzphHuvitvnwcW8toGh9-l9FNz9K6H4

Annex 11 – Usability Test – Test Plan

https://docs.google.com/document/d/1akJ-0LKHmGqDi4Hcq0T-i6AYiebgtReQ-yMbFMe_dsY/edit?usp=sharing

Annex 12 – Usability Tests Videos

  • Test 1 – User A – https://youtu.be/bCBUgsL581k
  • Test 1 – User B – https://youtu.be/neNAR-IPkyg
  • Test 1 – User C – https://youtu.be/7E5WwYTwer0
  • Test 1 – User D – https://www.youtube.com/watch?v=D-BJCTWWsqE
  • Test 1 – User E – https://www.youtube.com/watch?v=tsQVyT2aI_8
  • Test 2 – User A – https://youtu.be/-Gy-D2SRbO8
  • Test 2 – User B – https://youtu.be/_c113cEAj_4
  • Test 2 – User C – https://youtu.be/CTIPmkwTJUU
  • Test 2 – User D – https://youtu.be/x2oxZsbYdMo
  • Test 2 – User E – https://www.youtube.com/watch?v=tsQVyT2aI_8

Annex 13 – Usability Tests Notes

  • Test 1 and 2 – Errors table, timings and SUS Scores – https://docs.google.com/spreadsheets/d/1DZjIggxmuuXDXz4bsS_5n1Lzw9F3L7_k88KB4A5Jq4s/edit?usp=sharing
  • Errors Notes Test 1 & 2 – https://docs.google.com/spreadsheets/d/15wcnBVN7FV3othoVrcSRmFY40zkX-UIgt0YltCi15Cs/edit?usp=sharing
  • SUS Scores Test 1 & 2 – https://docs.google.com/spreadsheets/d/1zcWpjvTQestUa-gGeH6KFeR-gkn2nJ1lvCILZ6qi6TQ/edit?usp=sharing
  • Test 1 – Test Feedback Summary – https://docs.google.com/document/d/1v7kQT1q2mPylTZ8zZY5cQvX31w8ZDJdQom6NIxchPOY/edit?usp=sharing
  • Test 2 – Test Feedback Summary – https://docs.google.com/document/d/1TPEUyCmdpqkJOVOvCBX7HQp0itR9dTgr-mTkZUPlWYo/edit?usp=sharing

Annex 14 – Usability Tests – Information Sheet & SUS Questionnaires for Participants

Version 1: https://docs.google.com/document/d/1f5h66mhJ9DCJe94y_UjK8unRQFRfm9OGbRcyjPZMRUc/edit?usp=sharing

Version 2:
https://docs.google.com/document/d/14nfQqqJCZ1ILMcEpM7fSkV2NDwQB4Ozyf4NU9kuer4s/edit?usp=sharing