Case Study

Summary

The COVID-19 pandemic and resulting economic impacts led to an unprecedented surge of claims for unemployment benefits. As state agencies abruptly transitioned to remote work, the IT systems behind unemployment insurance (UI) programs slowed or failed altogether, creating backlogs that agencies struggled to process. New federal benefits and programs helped alleviate financial hardship for many, but most state UI systems were not designed to rapidly or easily adapt to these new programs and benefits. With a significant increase in available benefits created by federal legislation, many states experienced organized criminal efforts to defraud UI programs. In New Jersey alone in 2020, the state received the equivalent of seven years of claims in three months. In states where IT systems faltered or failed, the need for modernizing aging, brittle, and unstable information technology had never been more apparent. 

- 84% usability score for claimant portal, indicating high trust

- 8 out of 10 rating on average for the content on the confirmation page

The New Jersey Department of Labor (NJDOL) recognized the urgent need to make its UI technology more accessible, flexible, and resilient. The complexity and age of New Jersey's IT systems require a long-term vision and plan to truly modernize. A new Office of Unemployment Insurance Modernization was established within NJDOL to direct the IT modernization strategy and coordinate across programs and agencies. With additional funding and support from the United States Department of Labor (USDOL), New Jersey selected Nava to support them on their modernization journey.

Leveraging user research to prototype a new claimant portal and centralized data hub for UI agents, Nava hopes to aid NJDOL in reducing administrative burden, promoting equity, and decreasing the risk of fraud.

Approach

Promoting transparency with a new claimant portal

Transparency is key in building trust in public institutions, which is why we’re helping NJDOL build a claimant portal that will give UI claimants access to important information and updates on their claim. USDOL and Truss Works (Truss) started this project by building out the first iteration of the portal’s intake application, which is where claimants can file an unemployment claim online.

Nava began integrating the new claim application interface with the state’s legacy backend systems. In addition to creating a robust, API-first integration layer, the team is building out additional features that will help guide claimants through the application process, verify their eligibility for UI benefits, and reduce administrative burden for both claimants and UI staff. This includes pages around the intake application that direct people to the best way to file a claim for their situation (directional page), provide simple, plain language instructions on how to file a claim (homepage), and help them understand next steps after they submit (confirmation page).

In order to ensure that the portal works for those who need it, Nava conducted extensive usability testing, both virtually and in-person. Core to Nava's human- centered design approach is testing with real users early on and as often as possible, and to ensure that usability testing accounts for and appropriately represents the services' intended population. NJDOL leadership joined Nava in observing some of the sessions, allowing them to see firsthand human-centered design principles in practice.

This kind of human-centered approach is continuous and incremental—we constantly check design assumptions by testing with New Jerseyans and observing how they interact with systems, ensuring that the products Nava and NJDOL are building truly meet their needs.

“We’re engaging with actual residents for feedback as they test our solutions,” said Robert Asaro-Angelo, Commissioner of New Jersey Department of Labor and Workforce Development, at a recent budget hearing. “Our vendors and agents are sitting side-by-side with each other and with individual users to determine how to make this process better.”

Outcomes

In-person testing

The team traveled to New Jersey to test the new claimant intake experience with New Jerseyans. The goal of this research engagement was to find out if people know what to do before, during, and after using the application, if they’re able to move through the application unassisted, if they feel confident about their answers to the intake questions, and if they understand what each page of the application is asking them to do, or trying to communicate.

Human-centered design principles emphasize designing usability tests such that they reflect a wide range of people, particularly women, those who are nonwhite, those who are not U.S. citizens, and those who are low-wage workers. To reach the widest possible audience, the team worked with NJDOL, public institutions, and community organizations to advertise usability testing sessions. In total, over 70 New Jerseyans signed up to fill nine user testing slots at two locations over a two day period. The team then selected and conducted user testing with these nine people on both desktop and mobile at local Camden County libraries ahead of the public launch of this work.

Once participants were identified, Nava designers conducted and moderated 60-minute in-person tests of the new intake application experience. Nava staff observed participants as they viewed the homepage, completed the screener, viewed the directional page, moved through the intake application, and viewed the confirmation page. They were then asked about their overall experience of the application, taking note of what was working and what was confusing. Testers were compensated for their time.

The portal was rated a 5.88 out of 7, or 84% usability score, indicating that overall, testers felt confident navigating the new claimant portal experience. Additionally, several NJDOL staff, including the Commissioner and Deputy Commissioner, were able to observe the testing and have a rare first-hand opportunity to better understand people's experience applying for benefits.

As one testing participant said, “Everything was running really smooth. It was very simple. Like I said, I love simple.” This sentiment combined with other feedback led directly to developing plans for future improvements. From there, NJDOL felt the feedback was positive enough that they could move forward with plans to launch the new experience to New Jerseyans.

Confirmation page

Before conducting in-person testing, the Nava team gut-checked early versions of new content through remote usability testing. One example is the confirmation page, where claimants are directed to once they hit the submit button on their UI application. The confirmation page provides claimants with instructions on what to do next in the application process. NJDOL wanted to ensure that claimants understood the content presented on the confirmation page, so we tested the feature with four people. The team conducted 30-minute moderated, remote highlighter tests, or tests where users highlight content that’s clear in green and content that’s confusing in red.

A screenshot of highlighter tests superimposed on one another to form a heat map of what’s working and what’s not. 

It was important to test with a variety of users—three participants were experienced in helping people file UI claims and one had no experience. This helped us identify improvements that will make the confirmation page understandable for everyone, from those who are highly familiar with the UI process to those who are brand new to it.

The tests revealed several crucial insights that led to recommendations on how to amend the confirmation page. For example, participants had questions about the online identity verification solution the service uses. As a result, the team added a sentence clarifying what the identity verification service is and provided a link to a page that answered more detailed questions on it, such as how to handle out-of-state or expired IDs.

The team also asked people to give the content a score on a scale of 1-10, one being the worst and 10 being the best. The content on the confirmation page received an average score of 8, signifying that we are doing a lot of things right.

The team was able to conduct the highlighter tests, speak with participants, and share findings and recommendations with NJDOL all within two weeks. This agile approach decreased the risk of launching a confusing experience for UI claimants that could lead to a surge in calls to the NJDOL call center. Instead, the team clarified instructions and information, helping claimants have a more seamless and efficient experience.