What we learned from running a diary study in Lookback

LloydsDirect is a pharmacy which allows people to manage their repeat prescriptions online and get them delivered to home for free. We have all been through the frustration of taking 30 minutes out of our busy day to go to the pharmacy, only to find that we have to wait another 20 minutes, or they actually don’t have the medicines we need.

There are many other problems which can occur when collecting prescriptions in the ‘traditional’ way, and it is important for us to identify these. This helps us know what things to address in our marketing messaging, and also we can learn what things people like about the traditional experience - so we can make sure we include this in our offering.

A video diary study felt like the perfect way to capture the traditional pharmacy experience. We wanted to see the end to end journey of ordering and collecting prescriptions, as close to the events happening in time as possible. This means we capture the small contextual details which are often forgotten when recalling events in an interview setting, and also environmental cues which can shed more light on the participant’s everyday life.

Set on the method, we then began exploring what diary study tools exist. Although there are some great tools which allow you to run fairly complex studies, what we wanted to do was simple. Set a series of tasks which allows participants to read a set of questions on screen, as it films them talking about their experiences. It didn’t seem necessary to pick a more complex all encompassing tool which comes at a premium price, so we looked into the viability of using the tool Lookback, which we have used to run remote usability sessions in the past.

Lookback is a user testing tool, in which you can interact with users through screen sharing, either in an interview or unmoderated format. Participants can access it on the website or on the app. There aren’t many extra fancy features - which are exactly what we don’t need for a simple study like this, and means it is a more affordable option.

How we set lookback up to do the diary study

The tool isn’t designed specifically for diary studies, so elements of the set up were a bit more ‘hacky’ than when using a specialised diary tool. This meant that we had to do a lot of the heavy lifting in the initial email, to help participants understand what to do, without any intuitive diary study based instructions on the tool itself.

We sent participants step by step instructions on how to get into Lookback on their phones, as the set up isn’t necessarily that clear and involves quite a few steps. We also had to set up each diary task as a different ‘session’ in Lookback. This meant participants received a list of links, which they needed to refer back to and click every time they moved on to the next task.

As part of the introduction email, we included short videos of us completing the first task of the diary study. This meant that participants got to know a little bit about who we are, and also what a diary entry would look like. It can feel quite strange completing a diary study for the first time, so we hoped that this made participants feel more comfortable filming themselves, and that they could feel open in talking to us about their prescriptions, as they weren’t talking to a faceless entity.

What worked well

Once participants were set up and completing a task, Lookback served its purpose well. Participants were able to read our pre-set questions on their screen as they spoke, meaning that they weren’t likely to miss any points, or what the aim of that particular task was.

Lookback recommends that you only upload videos onto the app when using Wifi - which is obviously a problem for participants trying to film when visiting their local pharmacy. However, we tested this out using our own data and it worked well, and we didn’t encounter any problems with this during the actual study.

Another part of Lookback which worked well for the study, was that it filmed both the participants' screen and their face. This allowed us to capture some other interesting things, such as one participant who showed us how they order their prescriptions over email.

What didn't work well

The problem with participant’s being able to see the pre-set instructions and not their own camera, meant they didn’t always realise that it was filming them. Although we emphasised in our initial email that they won’t be able to see themselves when the camera is on, participants still struggled to know if they had set it up correctly. This resulted in long videos being uploaded which participants didn’t even realise had been filmed.

We quickly realised and re-emphasised this point in the actual instructions that participants could see on the screen - but by this point, some participants had emailed to say they couldn’t get set up on the tool and that they’d like to upload videos another way. This meant about half of the participants successfully used Lookback to complete the study, and half didn’t.

For those participants, we realised they felt much more comfortable filming themselves with their camera and sending to us afterwards, or uploading directly to youtube. The only problem with this is that they weren’t looking at the pre-set questions whilst they were recording their entries, so sometimes points were forgotten or missed. It also meant all of our entries were in multiple places so we needed to keep track of them all separately.

What we learned for next time

What we learned is that if participants are struggling in the early stages of setting up the study, their confidence will be knocked and the likelihood of them persevering with the tool is low.

Although using Lookback worked for a sample of participants, it still involved a lot of correspondence back and forth between us and the participants to help them get set up. Evaluating how the project went, we felt like actually this hacky approach resulted in more work for both us and the participants. It’s easy to assume that our participants will be okay downloading and using a brand new app, and remember many different instructions from the initial emails - but actually this meant there were many different areas where participants could get stuck.

We’re running another diary study soon, this time with our own patients to understand what that end to end journey looks like at the moment. We’re thinking of other low barrier methods that participants may be more familiar with to collect the entries, for example Whatsapp or Youtube. We will let you know how we get on!

If anyone has any questions, or diary study tool recommendations, don’t hesitate to get in touch — abigail.ingram@lloydsdirect.co.uk