Helping Cyber Officers
Select Their Upper Echelon

Service
UX research & analysis, MVP Development
Stakeholder
Air Combat Command
Project Timeline
Jan 2023 -
July 2023
ROI
~300 person-hours saved

Project Overview

The Air Force prides itself on creating great leaders through its training, education, and character development. But how are these varied leaders selected to become the upper echelons of officers? That is the question the AF CyberWorx team has been working with the 17x cyber career field heads at Air Combat Command (ACC). Tasked directly by Brig Gen Blackwell, our most recent effort was centered around taking our research and understanding of the cyber ecosystem from a previous effort and using it to dig deeper into the actual selection process, known as Development Team (DT) Boards.

How might a system aggregate and present 17X billet, training, and education data
to enable DTs to make the best 17X talent management decisions more efficiently?


With the problem to attack fleshed out, we worked towards completing a set of design criteria that could be used for such a system, deep analysis into the current DT board process, and recommended process improvements.

Our previous knowledge and discussions with cyber officers was not enough on it's own, so our Team entered into a 6-month effort to fully understand the DT board process and develop a solution. AF CyberWorx, with me as the lead UX researcher and designer, began both primary and secondary research to ensure we had a complete picture of the current process, as well as succession planning as a whole in the Air Force

After our own seondary research in addition to user interviews of the DT board members and their Officer Assignment Teams (OAT), our Team began creating a full set of platform-agnostic design criteria that would later become the system requirements. In addition, we conducted primary and secondary research into existing Air Force systems that could potentially be used for this purpose.

What resulted of this effort was a report with actionable recommendations and design criteria, as well as an in-person outbrief to our ACC stakeholders. Our brief included the commander of ACC A6 and his Team, in addition to Gen Select Phillips, who replaced Brig Gen Blackwell.

Below you will find a more in-depth breakdown of our process and methods used throughout this project.

Project Execution

Due to the DT boards being a largely "hush-hush" affair, we knew the first step would be to deepen our knowledge of the space. We simultaneously launched an effort to understand industry best practices for succession planning with secondary research, and developing a stakeholder map through both primary and secondary research. Once we knew more about the players in the space, it was time to start asking questions!

Our Team conducted 12 user interviews with current and previous 17X Officer Assignment Team (OAT) members, Career Field Managers (CFMs), Talent Marketplace experts, a 6X career field expert, and a MAJCOM civilian talent management representative. A few of the questions we asked were the following:

  • How does the DT process differ from the Assignment team process?
  • What percentage of the process is done by the DT board itself vs OATs?
  • What data is driving what decisions?
  • What do you believe is the ideal outcome of the DT process?
  • How are decisions made for new ideas?

User interviews and secondary research helped us form a more complete mental model, but to really dig into how things are done vs how individuals say they are done was an important difference for us to explore. So combat this, our Team sent a UX researcher to observe the Spring DT board in-person. This provided invaluable insight into the workflow, conversations, and methods employed in the actual board. These insights were used largely to inform the creation of a process flow in addition to illuminating pitfalls and needs of the users in this process.

By combining the outputs and analysis of our user interviews with what our Team identified in the DT board itself, we crafted a series of design criteria broken down by epics, user stories, and acceptance criteria for each need of the system. This output will essentially translate directly into requirements that can be contracted out to developers for completion of the system itself.

With an understanding of what this system would need to accomplish as well as the user needs for this system, or Team then moved to recommendations for an actual platform to build this on. We explored 7 different data-based platforms with the ability to acces Air Force systems of records that would be needed for the DT board data. We recieved demos from the developers for all 7 platforms, and gave a top 3 recommendation for platforms that are able to connect to the appropriate data, are customizable to the workflow of the DT process, and are relatively low-code and low barrier of entry to allow an eventual handoff to the government. 

With the combination of our fulyl developed design criteria, platform recommendations, policy improvement recommendations, and available development paths delivered, our Team also had the good fortune of having a rapid MVP development group assist us in creating a notional MVP. Their Team rapidly developed a barebones version of the workflow and data integration that this DT board system needs. To bolster this effort, our CyberWorx Team created low-fidelity wireframes to assist in the buildout of the on-screen elements.

Our internally built MVP was a great proof of concept, but our Team wanted to better understand the thoughts OAT members would have and if they feel a system like this would lighten the load of their homework phase. We conducted 2 rounds of synchronous user testing with 5 OAT members and used each session's feedback to inform updates to the workflow, UI, and features of the MVP. These efforts helped round out not just the user research to inform decision making, but also ensure the design aspects for the upcoming continued MVP development would meet the user experience needs of those that will use the system.

Project Results

The result of this effort was in-depth design criteria that can rapidly be acted on to create a system that will save the OAT members ~300 person-hours of work each DT board, ensure current usability struggles are rectified, and provide the opportunity for more artful decision making when selecting the upper echelon of cyber officers in the Air Force.

~300

yearly person-hours saved

26

Design criteria identified

48

personnel interviewed
Case Studies

My Portfolio

Air Force Cyber Career Field Development Team (DT) Board System

~300 person-hours saved per board held

Air Force Development Team (DT) boards are responsible for selecting the upper echelon of Air Force officers. Through a research effort combined with system design based off user design criteria, my Team and I improved both the efficiency and efficacy of the DT board process.

Member Operations Training Analytics and Reports (MOTAR)

Cost-avoidance of ~$6.75 million

Innovation is the conduit for so many meaningful ideas, but it is also important to identify programs that fall short of their proposed intent. Enter MOTAR, a research-heavy initiative culminating in a business case analysis lite to inform decision-makers of the Air Education and Training Command.

81st Training Wing

Facilitated 55 leaders to create 1 united vision

Maryland State Library knew their user experience, particularly surrounding navigation, was flawed and wanted data to back it up. Our Team siezed the opportunity to create an updated navigation structure to go along with a modernized UI.