# 3. Research for Service Analysis (12 pgs)



# 3-1 Research for Service Analysis

![chapter-3-1 picture.png](https://faa.switchboxinc.com/uploads/images/gallery/2023-08/chapter-3-1-picture.png) Section 3 deals with the integration of HF prior to approval to initiate the formal Acquisition Management System (AMS) process and covers the topics indicated below. You can page through Chapter 3 to review these topics.

- P. 3-2: [<span style="color: rgb(53, 152, 219);">Triggers for developing proposed acquisition </span>](#page_19)
- P. 3-3: [<span style="color: rgb(53, 152, 219);">HF guidance when proposing small acquisition projects </span>](#page_20)
- P. 3-4: [<span style="color: rgb(53, 152, 219);">HF integration when proposing large acquisition projects</span>](#page_22)
    - P. 3-5: [<span style="color: rgb(53, 152, 219);">Completing Requirements of In Service Review (ISR) Checklist (FAA network user only)</span>](#page_24)
    - P. 3-6: [<span style="color: rgb(53, 152, 219);">Sample Requirements in ISR Checklist (FAA network user only)</span>](#page_25)
    - P. 3-7: [<span style="color: rgb(53, 152, 219);">Planning the HF Effort</span>](#page_26)
    - P. 3-8: [<span style="color: rgb(53, 152, 219);">Specifying HF Activities</span>](#page_27)
    - P. 3-9: [<span style="color: rgb(53, 152, 219);">Monitoring and Reviewing HF Activities and Deliverables</span>](#page_28)
    - P. 3-10: [<span style="color: rgb(53, 152, 219);">Specifying Contents of the Transfer Package</span>](#page_30)
    - P. 3-11: [<span style="color: rgb(53, 152, 219);">Sample Contents of a Transfer Package</span>](#page_31)
- P. 3-12: [<span style="color: rgb(53, 152, 219);">Conclusion: Research for Service Analysis</span>](#page_32)

# 3-2 Research for Service Analysis

**Triggers for Developing Proposed Acquisition.** There are a number of different sources of triggers that need to be documented in order to produce a request for approval by the [<span style="color: rgb(53, 152, 219);">Joint Research Council (JRC)</span>](https://ksn2.faa.gov/afn/jrcportal/SitePages/Home.aspx) to start the formal Acquisition Management System (AMS) process for a project or to define needs and preliminary requirements for a project to develop an enhancement within an existing program. See<span style="color: rgb(53, 152, 219);"> <span style="color: rgb(0, 0, 0);">Operational Needs Assessment (ONA)</span></span><span style="color: rgb(0, 0, 0);"> </span>for an example. Such sources include but are not limited to the following:

- Internal to FAA 
    - Air Traffic Organization ([<span style="color: rgb(53, 152, 219);">ATO</span>](https://www.faa.gov/about/office_org/headquarters_offices/ato))
    - Technology Development &amp; Prototyping Division (<span style="color: rgb(53, 152, 219);">[ANG-C5 (FAA network user only)](https://my.faa.gov/org/staffoffices/ang/ang_org/directorates/angc/angc5)</span><span style="color: rgb(53, 152, 219);">)</span>
- External to FAA - through a transfer package or recommendation submitted to the Program Management Organization ([<span style="color: rgb(53, 152, 219);">AJM</span>](https://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/pmo)) and Mission Support ([<span style="color: rgb(53, 152, 219);">AJV</span>](https://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/mission_support)) 
    - <span style="color: rgb(0, 0, 0);">Collaborative Decision Making ([<span style="color: rgb(53, 152, 219);">CDM</span>](https://cdm.fly.faa.gov/)</span><span style="color: rgb(0, 0, 0);">) Program</span>
    - [<span style="color: rgb(53, 152, 219);">MITRE</span>](https://www.mitre.org/our-impact/rd-centers/center-advanced-aviation-system-development)
    - [<span style="color: rgb(53, 152, 219);">NASA</span>](https://www.nasa.gov/)
    - [<span style="color: rgb(53, 152, 219);">RTCA</span>](https://www.rtca.org/)
    - [<span style="color: rgb(53, 152, 219);">SWIFT</span>](https://www.faa.gov/air_traffic/technology/swim/swift) <span style="color: rgb(53, 152, 219;">(System Wide Information Management (SWIM) Industry-FAA Team)</span>

# 3-3 Research for Service Analysis

**HF Guidance When Proposing Small Acquisition Projects.** This can be provided by the FAA Human Factors Division ([<span style="color: rgb(53, 152, 219);">ANG-C1</span>](https://www.hf.faa.gov/)) or the FAA Planning and Analysis Team (<span style="color: rgb(53, 152, 219);">[AJM-131 (FAA network user only)](https://ksn2.faa.gov/ajm/pmo/about-us/ajm1/ajm13/131/hf/Pages/Home.aspx)</span>). These offices can assist with identifying HF analyses relevant to defining a shortfall and in outlining possible solutions. And they can help ensure that these analyses are tailored to be consistent with the size, cost, and complexity of the solution being acquired ([<span style="color: rgb(53, 152, 219);">HF-STD-004A</span>](https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf)).

[![chapter-3-3 picture.png](https://faa.switchboxinc.com/uploads/images/gallery/2023-08/chapter-3-3-picture.png)](https://faa.switchboxinc.com/uploads/images/gallery/2023-07/chapter-3-3-picture.png)

# 3-4 Research for Service Analysis

**HF Integration When Proposing Large Acquisition Project.** For larger projects, significant Research for Service Analysis needs to be conducted in order to prepare an<span style="color: rgb(0, 0, 0);"> Operational Needs Assessment (ONA)</span> or to define needs and preliminary requirements for a project that is an enhancement of an existing program. Additional Research for Service Analysis may also be completed after a preliminar<span style="color: rgb(0, 0, 0);">y ONA </span>has been approved to move forward by the ATO Directors Forum or in response to the submission of a transfer package submitted to [<span style="color: rgb(53, 152, 219);">AJM-S</span>](https://www.faa.gov/about/office_org/headquarters_offices/ato/service_units/pmo) by an external organization such as [<span style="color: rgb(53, 152, 219);">NASA</span>](https://www.nasa.gov/) or [<span style="color: rgb(53, 152, 219);">MITRE</span>](https://www.mitre.org/our-impact/rd-centers/center-advanced-aviation-system-development).

**Preliminary Concept Exploration, Development and Evaluation.** For larger projects that merit significant Research for Service Analysis, concept exploration, development and evaluation is necessary. HF involvement in this research includes:

- Specifying HF considerations relevant to a preliminary shortfall analysis
- Defining users and their associated operational needs and constraints
- Defining use cases, scenarios and critical tasks
- Identifying important elements of the broader system environment
- Generating and evaluating design alternatives

Inclusion of HF input during this initial stage is important because production pressures during Solution Implementation tend to limit the extent to which design alternatives can be adequately considered and evaluated. This initial stage prior to initiation of the formal Acquisition Management System (AMS) process provides an opportunity to more fully define user needs and shortfalls, as well as to consider the broader use environment and evaluate design alternatives.

# 3-5 Research for Service Analysis

[![chapter-3-7 picture.png](https://faa.switchboxinc.com/uploads/images/gallery/2023-08/chapter-3-7-picture.png)](https://faa.switchboxinc.com/uploads/images/gallery/2023-07/chapter-3-7-picture.png)**<span style="color: rgb(0, 0, 0);">Completing Requirements of In Service Review (ISR) Checklist.</span>** Another example of important HF issues that are relevant for consideration during Research for Service Analysis is provided by the [<span style="color: rgb(53, 152, 219);">In Service Review (ISR) Checklist (FAA network user only)</span>](https://my.faa.gov/org/linebusiness/ato/safety/isd.html). The ISR Checklist is an FAA management tool that is used to ensure that important criteria have been met prior to fielding of an acquisition.

For large projects, early consideration of these specified HF criteria during Research for Service Analysis can help to ensure that critical HF issues are addressed, as production pressures during Solution Implementation may result in inadequate, cursory consideration of these factors.

# 3-6 Research for Service Analysis

**Sample Requirements in ISR Checklist.** There are a number of requirements in the [<span style="color: rgb(53, 152, 219);">ISR Checklist (FAA network user only)</span>](https://my.faa.gov/org/linebusiness/ato/safety/isd.html) that can be addressed in the Research for Service Analysis. Examples are provided below.

Item # 6.1.1 Have **human factors requirements, plans, analyses, and verifications been determined and documented** for the product in accordance with the <span style="color: rgb(53, 152, 219);">[FAA Human Factors Acquisition Job Aid](https://fast.faa.gov/docs/HFAcqJobAid.doc), [FAA Human Factors Design Standard (HF-STD-001B)](https://hf.tc.faa.gov/publications/2016-12-human-factors-design-standard/)</span>, <span style="color: rgb(53, 152, 219);">[Human Factors Engineering Requirements (HF-STD-004A)](https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf)</span>, <span style="color: rgb(53, 152, 219);">[FED-STD-795, Uniform Federal Accessibility Standards (UFAS)](http://everyspec.com/FED-STD/FED-STD-795_9884/)</span>, and other appropriate standards and conventions?

6.1.2 Have **usability issues and human factors risks** for all user (e.g., operators, maintainers, supervisors, administrators) populations been addressed for all intended sites?

Item # 6.1.3 Have the **human-system performance studies and analyses** (such as those related to workload, training, functional design, computer-human interface, staffing, safety and health, special skills and tools, work space, automation functions/roles, displays and controls, information requirements, display presentation, visual/aural alerts, input/output devices, communications, procedures, anthropometrics, documentation, and environment) been conducted to identify, document, and resolve/mitigate human factors issues?

Item # 6.1.6 Have steps been taken to **identify, eliminate, reduce, or mitigate the sources of human error** with the highest risks (combination of likelihood, severity, detection, and recovery)?

See additional HF criteria in Items 6.1.4, 6.1.5, 6.3.1, 8.4.2 and 8.6.10 of the [<span style="color: rgb(53, 152, 219);">ISR Checklist (FAA network user only)</span>](https://my.faa.gov/org/linebusiness/ato/safety/isd.html).

# 3-7 Research for Service Analysis

**Planning the HF Effort.** For large projects, there are several steps that are important to ensure that the organization conducting the Research for Service Analysis integrates HF into its work.

- **Step 1.** Has the FAA Human Factors Division ([<span style="color: rgb(53, 152, 219);">ANG-C1</span>](https://www.hf.faa.gov/)) or the FAA Planning and Analysis Team (<span style="color: rgb(53, 152, 219);"><span style="color: rgb(0, 0, 0);">AJM-131</span></span>) been consulted for guidance on how to effectively integrate HF into the Research for Service Analysis?
- **Step 2.** Has a qualified HF specialist been added as part of the FAA Systems Engineering Management Team managing the project?
- **Step 3.** Has this HF specialist on the FAA Systems Engineering Management Team defined the HF activities and deliverables that need to be included as requirements in the Statement of Work (SOW) for the organization conducting the Research for Service Analysis, tailored to be consistent with the size, cost, and complexity of the solution being acquired ([<span style="color: rgb(53, 152, 219);">HF-STD-004A</span>](https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf))?
- **Step 4.** If the Research for Service Analysis is competitively bid, has the HF specialist been involved in evaluating the submitted bids for the HF components? Has the organization conducting the research included a qualified HF specialist on its team and included a user group to provide input? Has this organization specified the appropriate HF activities and deliverables as part of its proposal?

# 3-8 Research for Service Analysis

**Specifying Human Factors Activities.** A large project should include a focus on workflow, usability, procedures, equipment, workload/staffing, human error and operational suitability. This can include both material (e.g., equipment) and non-material (e.g., procedures) needs, shortfalls and concepts and inputs needed by stages of the [<span style="color: rgb(53, 152, 219);">Acquisition Management System (AMS)</span>](https://fast.faa.gov/) process. Activities also should include consideration of the requirements for the <span style="color: rgb(0, 0, 0);">ONA</span> and [<span style="color: rgb(53, 152, 219);">ISR Checklist (FAA network user only)</span>](https://my.faa.gov/org/linebusiness/ato/safety/isd.html). **Keeping in mind the need to tailor the effort to reflect the nature and magnitude of the project** ([<span style="color: rgb(53, 152, 219);">HF-STD-004A</span>](https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf)), possible activities include:

- Ethnographic studies (on site observation/interviews of prospective users)
- Input from operational users and other stakeholders using questionnaires, focus groups, structured interviews and use case reviews
- Task, functional and user analyses, including development of use cases, scenarios and [<span style="color: rgb(53, 152, 219);">Critical Task Analysis Reports</span>](https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf) (see p. B-7), considering the broader hardware, software and distributed work context
- HF input to a preliminary CONOPS, with a separate CONOPS for the different user groups (including operations and maintenance) as needed
- Specification of the HF aspects of the ONA and associated set of preliminary functional requirements
- Design of storyboards, user interface mockups and functional prototypes
- Completion of formative evaluations of storyboards, user interface mockups and prototypes, including evaluations of alternative designs and design features using heuristic analyses, cognitive walkthroughs, usability studies and/or Human-In-The Loop studies (HITLs) to evaluate usefulness and usability

# 3-9 Research for Service Analysis

**Monitoring and Reviewing HF Activities and Deliverables.** For large projects, there are several steps that are important to ensure that the organization conducting the Research for Service Analysis effectively integrates HF into its work.

- **Step 1.** Has the HF specialist on the FAA Systems Engineering Management Team reviewed the details of proposed activities by the organization conducting the research and observed the completion of these activities?
- **Step 2.** Has the HF specialist on the FAA Systems Engineering Management Team reviewed the <span style="color: rgb(0, 0, 0);">Operational Needs Assessment (ONA)</span> and associated set of preliminary requirements based on the Research for Service Analysis?
- **Step 3.** Has the HF specialist on the FAA Systems Engineering Management Team reviewed the HF deliverables and the contents of the transfer package prepared by the organization conducting the Research for Service Analysis?

The following pages provide examples of HF components of sample transfer packages.

# 3-10 Research for Service Analysis

**Specifying Contents of the Transfer Package.** For large projects, the findings from the Research for Service Analysis can provide valuable insights for the stages in the formal AMS process that are not adequately communicated in an Operational Needs Assessment (ONA), Concept of Operations CONOPS or requirements. As one FAA Program Manager indicated:

“We found that the contractor \[who was implementing the solution\] was revisiting things we solved 2 years ago, so we decided to let the contractor see the prototype developed during concept development. We’d spent years doing the original concept exploration to develop the CONOPS and requirements. Why throw all that expertise away? There’s a lot that they have learned that isn’t captured in the CONOPS and requirements. Let’s give the contractor a head start. They will have new ideas too. I wish I had done this earlier. It would have saved a lot of money.”

The transfer package should include both documents and any storyboards or prototypes that have been developed during the Research for Service Analysis. The next page provides examples of HF content for a range of transfer packages.

# 3-11 Research for Service Analysis

**Sample Contents of a Transfer Package.** The table of contents of a sample transfer package developed as part of the Research for Service Analysis for an FAA acquisition is provided below. Note that many of the included artifacts have important HF content based on the results of the Research for Service Analysis. Note, however, that the contents of the transfer package should be tailored to be consistent with the size, cost, and complexity of the solution being acquired ([<span style="color: rgb(53, 152, 219);">HF-STD-004A</span>](https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf)).

- Tech Transfer Package Overview
- Discussion Outcomes (The information in the Discussions Outcomes section was gathered from operational users and other stakeholders using Questionnaires, Focus Groups, Use Case Reviews, Human Factors Working Group (HFWG) / Working Group (WG) sessions, and Data Administration (DA) Working Group sessions. It included 25-30 briefings based on early UI mockups and associated feedback.) 
    - Discussion Outcomes Intro and ConOps Overview briefing
    - Discussion Outcomes briefing for each topic
    - CSS-Wx Products
    - CSS-Wx Products Overview briefing
    - Textual Wx Products spreadsheet
    - Graphical Wx Products spreadsheet
- Use Cases 
    - Use Cases Overview briefing
    - Use Cases List and Details spreadsheet
    - Use Cases \[series of DOCs\] (60 focusing on Air Traffic users; 32 on Maintenance users and 1 on data administration)
- Preferred Architecture 
    - Preferred Architecture briefing
- Technical Analysis 
    - Technical Analysis Summary briefing
    - Supporting Items
- Enterprise Information Management (EIM) Package 
    - EIM Overview briefing
    - Position and Information Management Utilizing EIM document
    - Static Data List spreadsheet
    - Supporting Items
- Computer Human Interaction (CHI) Guidelines 
    - CHI Guidelines Overview briefing
    - CHI Guidelines document
- Concept of Operations (There were three very different user groups, so three CONOPS were developed. One for ATC, one to support national management for standardization across facilities while allowing some customization by facility, and one for enterprise-wide system management, which had both National and local components. They were described in separate CONOPS.) 
    - Solution CONOPS document
    - Data Admin CONOPS document
    - Maintenance CONOPS document
    - Screening Information Request (SIR) to Contract Award Summary of CONOPS Changes briefing
    - Maintenance CONOPS Overview briefing
- Support Personnel Synopsis
- Summary of Requirements Changes (SIR to Contract Award) 
    - SIR to Contract Award Summary of Requirements Changes briefing
- Tech Transfer Glossary

# 3-12 Research for Service Analysis

**<span style="color: rgb(0, 0, 0);">Research for Service Analysis: Conclusion</span>**

**<span style="color: rgb(0, 0, 0);">Triggers.</span>** Research for Service Analysis is triggered by identification of an operational need or opportunity to improve performance. In order to help ensure that an acquisition meeting such an operational need will be useful, usable and actually used, [<span style="color: rgb(53, 152, 219);">FAA Order 9550.8, Human Factors Policy</span>](https://www.faa.gov/documentLibrary/media/Order/9550.8.pdf) requires that HF considerations be addressed in the earliest phases, including this initial stage.

**HF Guidance.** The FAA Human Factors Division ([<span style="color: rgb(53, 152, 219);">ANG-C1</span>](https://www.hf.faa.gov/)) or the FAA Planning and Analysis Team (<span style="color: rgb(0, 0, 0);">AJM-131</span>) can be consulted to assist with identifying the HF analyses for defining a shortfall and outlining an operational concept for a solution. This includes advice on how to tailor the effort to be consistent with the size, cost, and complexity of the solution being acquired ([<span style="color: rgb(53, 152, 219);">HF-STD-004A</span>](https://hf.tc.faa.gov/publications/2016-09-human-factors-engineering-requirements/HF-STD-004A.pdf)).

**HF Management.** HF guidance can be provided **for small projects** to produce an <span style="color: rgb(0, 0, 0);">ONA</span>, as well as for larger projects that require significant research. It also can help ensure that information is generated to support later stages in the formal AMS process and address requirements of the [<span style="color: rgb(53, 152, 219);">ISR Checklist (FAA network user only)</span>](https://my.faa.gov/org/linebusiness/ato/safety/isd.html). **For larger projects**, the HF specialist appointed to be part of the FAA Systems Engineering Management Team can provide input on the HF requirements for inclusion in the SOW for the organization conducting the Research for Service Analysis, including HF activities and components of the transfer package, as well as providing overall management of the HF components of the research.