Case Study

Summary

For many Veterans, disability benefits are a lifeline. They’re among the top five most-used U.S. Department of Veterans Affairs (VA) benefits, and they give Veterans more financial support than any other VA program. 

Given the far-reaching impacts of disability benefits, the VA Office of the Chief Technology Officer (OCTO) is prioritizing improvements to how Veterans apply for disability benefits on va.gov and how claims processors adjudicate claims. VA tapped Nava and prime contractor Agile Six to help improve one step of the disability application process, ultimately contributing to faster and more accurate payment of benefits. Our shared work contributes to OCTO’s mission of providing Veterans with benefits in minutes, not months. 

Approach

For this project, we focused on the va.gov page where Veterans enter the conditions they’d like to claim for benefits. Conditions can be anything from headache to right foot pain, and they help inform next steps in the claims adjudication process. Once a Veteran submits their application, claims processors sort similar conditions into categories, or classifications. 

Initially, we set out to help VA automate the classification process, which would free up time for claims processors who must otherwise manually classify conditions. During this work, we learned that some Veterans found the conditions submission page confusing, presenting an opportunity for VA to increase the usability and accessibility of the page. Based on this learning, we began to help VA improve the conditions submission page, in addition to continuing work on automating the classification process. 

Outcomes

With VA, we successfully automated the classification process, which can correctly classify 70% of conditions that Veterans submit online. To ensure accuracy and continuous improvement, agency staff review the automated classifications, and we set up a dashboard to monitor the service’s performance. We also review any changes that claims processors make to the automated classifications, which provides us with valuable feedback on accuracy.

In addition, we helped VA make changes to the conditions submissions page with an eye toward usability and accessibility. When Veterans enter text into the conditions field on this page, an algorithm produces a drop-down list of suggested conditions that the Veteran can choose from. For example, if a Veteran enters “tin” into the conditions field, the algorithm would likely suggest “tinnitus,” among other conditions. 

Previously, some Veterans found these suggestions confusing or irrelevant. That’s why we helped VA replace the old list of conditions with a new list that includes the most frequently claimed conditions, written in plain language. Importantly, every condition on the new list is classifiable. We also helped revise the search algorithm to return more relevant conditions based on what a Veteran enters, encouraging Veterans to choose from the classifiable conditions in the drop-down menu. With the new list and algorithm, Veterans select conditions from the drop-down 65% of the time — 10 percentage points more than the legacy list. 

Finally, we worked with VA to improve the instructions Veterans see when they submit conditions. This helps Veterans navigate the process more smoothly and promotes trust in VA and its systems. 

Process

Setting out to automate conditions classification

We began by conducting discovery research to understand condition classification. In this phase, we spoke to and shadowed claims processors and other VA experts to learn about their experience evaluating claims. We found that claims processors manually classify conditions, which is a necessary step in identifying which medical exams Veterans need to get for their claims.

Sometimes, a claims processor cannot accurately classify conditions because the Veteran does not provide specific enough condition information. In those cases, the claims processor has to inspect the Veteran’s medical records or call the Veteran for more information. This can be burdensome for the claims processor and can contribute to longer claim timelines. Lacking information on the Veteran’s conditions can also lead claims processors to select incorrect classifications. This can cause the system to suggest erroneous medical exams, extending the claim timeline by up to 30 days. 

Based on these learnings about the back end of va.gov, we decided to pursue a solution that would automate the classification process. We wanted to begin helping Veterans and claims processors as soon as possible, so we created a pilot program to process claims containing only one condition. By focusing on a subset of claims, we were able to iterate on the service and monitor its performance before expanding to all claims.

Redesigning the conditions list

During this process, we found it was difficult to classify many conditions because the way Veterans submitted them was inconsistent. Many Veterans found the instructions on the conditions submission page confusing, and the autosuggested conditions returned by the legacy algorithm were often unrelated, duplicative, or not classifiable, yet Veterans would still select them.

Before and after screenshots of the conditions submission page. 

As a result, we worked with VA to redesign the list of conditions to be more usable for Veterans and easily classifiable. We approached this by researching VA language and medical dictionaries, conducting data analysis of historic conditions submissions, speaking with claims processors, and interviewing relevant VA experts. Once the list was finalized, we tested it with Veterans and VA experts. These tests indicated that the conditions list worked well, so VA moved to launch it.

Enhancing usability and accessibility on the conditions submission page

We took several steps to help VA improve the usability and accessibility of the conditions submissions page. Using content strategy best practices, we helped re-write the instructions on the page to be more understandable for Veterans. We also built a custom algorithm to power the autosuggestions that Veterans see when entering conditions. 

With VA, we refined the page’s design to make it clearer that Veterans can select a condition from the dropdown list or enter their own free-text submission. When we tested the page with Veterans, 100% of participants were able to successfully enter free text, compared to only 50% with the legacy design. The refined design also meets current accessibility standards, and we partnered with accessibility experts to ensure the page works for people who use assistive technology.

Expanding impact by automating classifications for all claims

Our initial pilot only covered claims with one condition, or 20% of all disability claims. Once we completed the front-end work of redesigning the conditions list and optimizing the conditions submissions page, we were able to send 100% of claims through the automated service. 

Conclusion

Disability benefits are a critical resource for many Veterans, so it’s important for the online application process to be quick and easy and for benefits to be paid in a timely and accurate fashion. That’s why VA partnered with Nava to improve the condition classification process with an eye toward usability and accessibility. Though conditions classification is one specific facet of VA’s benefits system, it’s integral to OCTO’s mission of providing Veterans with benefits in minutes, not months.

Special thanks to Chloe Hilles for contributing to this article.