User research can be applied to a range of service delivery problems. This guide is specifically for helping government staff process benefit applications more quickly and efficiently. At times of crisis, the volume of applications can rise dramatically — for example, we saw a spike of applications pouring into state unemployment agencies as a result of the Covid-19 pandemic. If applications are incorrect or incomplete it takes even longer to process them.
Our goal is to decrease the number of incomplete or incorrect applications by helping applicants better understand what they’re eligible for and how to apply.
Because the applicants are your research participants, you can skip the step of recruiting participants, which will save you days in the research process.
User research is:
A set of observation techniques and feedback gathering—including interviewing and other methodologies—that help you understand the behaviors, needs, and motivations of your users.
This toolkit can help you:
- Gather information about why applicants are having trouble accurately completing an application form.
- Discover commonly confusing or misunderstood parts of the form.
- Develop updated designs or content for the form.
- Improve the rate of accurately completed forms for faster processing.
Plain language templates for user research is a useful companion to this guide. It’s referred to and linked to a few times.
Improve your forms in five days
Day 1: Prepare for your interviews
Day 2: Conduct your interviews
Day 3: Analyze your results
Day 4: Develop revisions
Day 5: Share results and agree on revisions
Prepare for your interviews
Here are the steps you need to take as you get ready for your interviews:
Review incomplete applications to identify the questions or requests that are most often the sources of errors. If you don't have access to this information, go where the claims processors are and ask them to share the most common types of errors they see as they are processing applications.
Select 20 to 30 applications with those errors. You will only need to complete 10 interviews but it’s a good idea to have extras lined up.
Determine whether or not you’re able to offer participants an incentive. If you’re able to, we recommend compensating participants at least $50 for a 30 minute call to help ensure they’ll be able to give you their time.
Email participants to schedule phone interviews (see example outreach email below). Track responses in a spreadsheet. You may want to use this spreadsheet template from the Plain language templates for user research toolkit. Try to schedule 15 to 30 minutes in between each interview.
Follow-up by email with confirmed participants to get informed consent. Find email and form templates for getting informed consent and protecting participants’ privacy during research in Nava's Plain language templates for user research toolkit.
Create an interview guide with a script you can repeat (see example script and questions below). Your goals are to set expectations, get informed consent, and solicit as much natural feedback as possible. Avoid leading or loaded questions or using jargon that applicants may not understand.
Consider any other stakeholders on your team who may need to participate in this process. For example, if there are key stakeholders who would benefit from hearing firsthand how people are succeeding and failing with your forms, invite them to quietly listen to your interviews. You will probably also need the early support of a key decision-maker from your legal and compliance team, as well as the person responsible for updating the form. Engage them as soon as possible.
Find a research partner to help do some of the interviews and take notes that capture as much of the participants’ own words as possible. Share your interview guide with your research partner.
Use this example outreach email
Subject line: Help improve our [form name] application and earn $50
Email: Dear [name],
I’m a researcher working with a team at [name of agency]. We received your [form name] application and it has missing or incomplete information.
I’m following up to help you complete the form. I’d also like to ask you some questions so we can learn how we can make the form more clear for you and future applicants.
Once you finish a 30-minute interview with me, I will submit your completed application and send you a $50 thank you for your time.
Simply reply to this message with “yes,” your phone number, and the best time we can reach you tomorrow, between 8 am and 6 pm Eastern Time.
📄 Use this script template
Use this script to get informed consent from your participant. Confirm their participation, remind them of their rights to skip questions, stop the interview, or opt out completely.
Use non-leading questions for natural feedback
Solicit natural feedback by giving your participant space to answer freely. For example, say:
As much as you can, be yourself and act as you naturally would. I did not design this form, I’m just collecting feedback. So please be honest with your positive and negative thoughts. Think aloud, if you would.
Ask non-leading questions like:
What is this asking you to do?
How would you complete this question?
What would you call this instead? (For example, a form may request “proof of income” and applicants say, “oh, you mean a pay stub?”)
Why do you think this information is being requested?
Would you seek help to find answers to your questions? If so, how?
At any point did you feel lost? What would have helped you get back on track?
What would you do next?
Are you done with the application? What makes you think so?
When do you expect to hear back about your application?
What would you expect to happen next?
Tweak any leading or loaded questions
Loaded question: Is this question clear?
Better way to ask: What is this question asking you to do?
In the example above, don’t assume whether or not something is clear or unclear. Leave as much room as possible in your questions for the participant to share descriptions, reactions, or needs.
Conduct your interviews
Make sure you or your partner are ready to take verbatim notes during the call. You may want to use this template from the Plain Language Templates for User Research toolkit. If you intend to record the call, make sure you obtain consent from your participant first.
Follow the schedule you’ve outlined and interview each participant.
Follow your script and remember the more detailed your notes are, the better your results will be. Capture as many examples as possible of your participants’ natural language. Don't skimp on note taking, you’ll need this data.
Conclude your day when you’ve completed interviews with 10 participants. Cancel appointments with any participants you do not need to speak with.
Analyze your results
Read through your notes from each participant 2 to 3 times. Highlight parts of the form that caused errors in one color. Highlight the ways in which participants would rephrase the labels, instructions, or other content in another color.
Create a comprehensive list of what caused the errors. Note how many times applicants were confused by each. Look for any patterns or common feedback and group them together into clusters to help you create an overview of what you learned.
Develop revisions
Referring to your results and lists, develop a few revisions for each part of the form that caused errors. Use this template to start your revisions alongside some of your quoted language (for labels, instructions, or other content) from participants.
Be mindful of character counts to help ensure that your revisions don’t cause too many changes to the layout of the form. Your form designer will thank you for this.
