Method for Defining the Automation Level of an eVTOL

ABSTRACT

Optimizing pilot-automation collaboration necessitates early safety assessment of the aircraft, which involves a rigorous examination of potential scenarios. This article seeks to establish a comprehensive framework for safety analysis, exploring various levels of automation (LoA) within high-technology projects such as electric vertical takeoff and landing (eVTOL) aircraft. The method is divided into three phases: the first involves defining the operations concept; the second involves applying the systems theoretic process analysis (STPA) method; and the third encompasses determining the safest LoA. Furthermore, this article scrutinizes the landing operations of eVTOL aircraft in urban centers. The identified control actions (CAs) include flight monitoring, landing verification, speed selection, and maneuvering. Ultimately, a LoA categorized as automated decision making emerged as the safest. This entails the concurrent monitoring and generation of alternatives by both the pilot and the autopilot, with the final selection of the optimal alternative and its subsequent implementation exclusively entrusted to the autopilot. This work contributes by presenting evidence that safety analysis should commence at the level of individual CAs, rather than at a higher level encompassing complete operations or the entire aircraft. This approach aims to generate comprehensive, practical, and effective safety requirements.

Keywords
Aeronautics; Automation; Aerospace safety; Methodology

INTRODUCTION

Recent advancements in Advanced Air Mobility (AAM) have positioned electric vertical takeoff and landing (eVTOL) aircraft at the forefront of both academic research and commercial applications (Xiang et al. 2024). These aircraft are set to revolutionize urban air mobility (UAM) by enabling vertical takeoff and landing at vertiports, utilizing distributed electric propulsion systems to achieve quieter operations compared to traditional helicopters (Agustinho and Bento 2022; Franciscone and Fernandes 2023; Ribeiro 2023). Ensuring the safe operation of eVTOL aircraft across domains such as aerodynamics, control systems, structures, and power systems is essential for fully realizing the potential of UAM (Su et al. 2024).

NASA-funded research has identified significant risks in AAM, including adverse weather conditions, failures in eVTOL vehicles and components, and potential intrusions into designated air mobility corridors by non-cooperating aircraft (Thompson et al. 2022). Automation plays a pivotal role in modern aviation, offering benefits like workload reduction. However, challenges persist in optimizing human interaction with these systems, particularly regarding autopilot mode confusion during critical flight phases such as vertical navigation (VNAV), which poses complexity and confusion for pilots (Albano et al. 2022; Laarmann et al. 2023).

Regulatory bodies like the Federal Aviation Administration (FAA) and the European Union Aviation Safety Agency (EASA) play an essential role in developing certification standards for eVTOL aircraft guided by regulations such as 14 Code of Federal Regulations (CFR) Part 23-64 and CS-23-5, which evolve alongside technological advancements (Cardoso et al. 2022). The integration of advanced automation systems in eVTOL operations necessitates careful consideration of human factors, including pilot workload management, standardization of automation philosophies, and effective supervision to ensure operational safety and efficiency (Abreu-Júnior 2008; Polet et al. 2003; Theunissen and Suarez 2015).

This paper proposes a novel framework for enhancing safety in eVTOL projects, focusing on comprehensive hazard scenario analysis, determination of optimal levels of automation (LoA) for pilots and autopilots, and integration strategies to promote effective human-machine collaboration. By advancing understanding and practice in safety management, this framework aims to support the successful integration and operation of future eVTOL systems in urban environments.

PROPOSED METHODOLOGY

The method proposed in this study is divided into three phases, as shown in Fig. 1. The first involves defining the operations concept; the second involves applying the systems theoretic process analysis (STPA) method; and the third encompasses determining the safest LoA.

Figure 1
Method phases.

1^st Phase

In the 1^st Phase, the Concepts of Operations (ConOps) document aims to describe high-level requirements and expectations of system users. The following parts detail the ConOps phases (U.S. Department of Justice 2018; NASA 2019).

Description of envisioned system

This section offers a comprehensive overview of the system’s requirements, goals, and objectives, providing detailed expectations for its capabilities, behavior, and operations. It includes a functional-level description of the system’s components, involving users and operators. Additionally, it explores modes and configurations vital for the system’s lifecycle, spanning testing, training, operational, and disposal phases.

Physical environment

This section should describe the environment in which the system is expected to function, including integration, testing, and transport. Temperatures, pressures, radiation, winds, and other atmospheric, spatial, or aquatic conditions expected and outside the nominal must be considered.

Support environment

This section describes how the idealized system will be supported after it is in the field. Discussions should consider how the system will be maintained, repaired, replaced, economic analysis, and future upgrades.

Operational scenarios, use cases, and/or project reference missions

This section offers a comprehensive overview of the primary scenarios, use cases, and Design Reference Manuals (DRMs) linked to the idealized system. It presents a cohesive narrative along a linear timeline, addressing the system’s typical operation under regular circumstances. Additionally, it delves into scenarios that involve specific conditions that imply deviations from the norm, including failures, suboptimal performance, unexpected environmental variables, or operator errors. The focal point is on the identification of essential additional features or safeguards required for the system across diverse scenarios.

Impact considerations

This section extensively assesses the potential impacts of the system on the environment, organizational aspects, and scientific/technical domains. The discussion also covers organizational impacts, specifically addressing considerations related to operator recruitment and training.

Risks and potential problems

This section should describe any risks and possible problems associated with the development, operations, or disposal of the envisaged system. It also includes concerns about the project schedule, required support staff, or implementation approach.

In general, it was considered that the eVTOL aircraft is equipped with eight electric propellers and relies on high-voltage batteries for power. It utilizes electric motors with 12 rotors and features helicopter-like skids for landing gear. It produces a noise level of 70 dB at a height of 500 feet and can reach a maximum altitude of 6,600 feet. Operating at a cruise speed of 150 mph, it has a range of 60 miles.

As eVTOL aircraft technology is still in its early stages, with generally low to medium technology readiness levels (TRL), only a few ConOps are available. For instance, Franciscone and Fernandes (2023) and Ribeiro et al. (2023) emphasize the potential of eVTOLs in UAM and stress the need for robust ConOps development. These plans should tackle the unique challenges of integrating eVTOLs into urban infrastructure, ensuring operational safety, security, and efficiency. However, due to limited information on these evolving aircraft, these points need to be studied.

2^nd Phase

Based on the data and information obtained from ConOps, a context for the study is established. However, the relationship between ConOps and a method for analyzing hazard scenarios can be deepened, and this will be addressed in the 2^nd Phase.

Risk management in complex sociotechnical systems involves modeling with unique attributes, including multiple, non-linear, and simultaneous factors (Bjerga et al. 2016). The study of dynamic contexts within systemic models aims to elucidate diverse activities, emphasizing preventive measures against organizational pressures. Accidents, stemming from multiple factors, trace back to Heinrich’s Domino theory in 1931 (Johnson 2011). Over the years, notable methods like Jens Rasmussen’s Accimap (Rasmussen 1997), Shorrock’s functional resonance analysis method (FRAM) (Shorrock 2007), and Nancy Leveson’s systems theoretic accident model and process (STAMP) (Leveson 2002) have emerged for risk analysis in complex systems (Borges 2019).

The STAMP theory, particularly its hazard analysis approach, STPA, stands out for identifying potential causal scenarios comprehensively, considering hierarchical relationships within the system (Borges et al. 2021).

STPA is a hazard analysis method focused on early accident detection and mitigation in complex system design (Kunio 2021). Leveson (2011) outlines its four steps: identifying accidents, hazards, and safety constraints; developing a control structure model; pinpointing unsafe control actions (UCAs); and proposing safety requirements. It is an iterative process applicable across the system design lifecycle, offering a structured approach to proactively identify safety concerns beyond traditional methods (Oginni et al. 2023).

The approach goes beyond the capabilities of traditional methods, providing insights into the interaction between system components and capturing hazards within the broader context. Advantages in applying STPA to assess the risk of operational traffic modifications in Brazil have already been demonstrated (Rodrigues et al. 2022).

3^rd Phase

Despite its success, a research gap exists in selecting the appropriate LoA for a control action (CA) in complex systems, especially for early aircraft safety analysis and design (Borges et al. 2021).

Identifying works with similar objectives involved conducting bibliometric research. No work was found relating ConOps, STPA, and LoA.

Figure 2 shows the number of documents found per database and the filters applied using the keywords. In the end, 168 documents were selected for analysis, with the keywords “Level of Automation” and “LoA,” in the database Scopus and Web of Science.

Figure 2
Bibliometric research.

The references span from 1997 to 2024. Among them, the three most cited documents are as follows:

• “Level of automation effects on performance, situation awareness and workload in a dynamic control task” by Endsley and Kaber, in 1999, with 735 citations (averaging 28 citations per year).

• “The effects of level of automation and adaptive automation on human performance, situation awareness, and workload in a dynamic control task” by Kaber and Endsley, in 2007, with 542 citations (averaging 25 citations per year).

• “Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety” by Endsley and Kaber, in 2004, with 188 citations (averaging six citations per year).

The first article introduces intermediate LoA with a taxonomy of 10 LoA across four roles, aiming to sustain operator engagement and enhance situational awareness in real-time control tasks. The second paper expands on this by studying LoA effects on performance, situational awareness, and workload, introducing intermediate LoAs for engagement and adaptive automation (AA) for workload management. The third paper focuses on outside-of-the-loop (OOTL) performance issues in human supervisory control, emphasizing consequences such as vigilance decrements, complacency, knowledge loss, and restricted manual control. Overall, these contributions define LoA, emphasizing the increasing role of machines in decision-making, referencing the 10 levels proposed by Sheridan and Verplank, in 1978 (Parasuraman et al. 2000).

Echoing this theme, Endsley and Kaber (1999), whose highly influential table underpins this work, identified four core functions inherent within the categories established by Sheridan and Verplank (1978). These functions encompass:

• Monitoring: involves systematically scanning screens or data to capture and assess the status of the system.

• Generating: formulating various options or strategies to achieve predetermined goals or objectives.

• Selecting: making decisions to choose the most appropriate option or strategy from the generated possibilities.

• Implementation: refers to the actual execution and realization of the chosen option or strategy.

For the connection between STPA and determining which LoA is safer, it will be necessary to analyze hazard scenarios, studying who would be better in command of the CA – the pilot (P), the autopilot (AP), or both (P/AP) – according to Table 1.

Table 1
Table of LoA.

Automation level		Roles
		Monitor	Generate	Select	Implement
1	Manual control (MC)	P	P	P	P
2	Action support (AS)	P/AP	P	P	B
3	Batch processing (BP)	P/AP	P	P	AP
4	Shared control (SHC)	P/AP	P/AP	P	P/AP
5	Desicion support (DS)	P/AP	P/AP	P	AP
6	Blended decision making (BDM)	P/AP	P/AP	P/AP	AP
7	Rigid system (RS)	P/AP	AP	P	AP
8	Automated decision making (ADM)	P/AP	P/AP	AP	AP
9	Supervisory control (SC)	P/AP	AP	AP	AP
10	Full automation (FA)	AP	AP	AP	AP

• Source: Adapted from Endsley and Kaber (1999).

When searching for the keywords “STPA” and “LoA,” the article titled “A systems-theoretic approach to hazard identification of marine systems with dynamic autonomy” was discovered. This article, authored by Yang et al., was published in 2020.

The article proposes an approach for identifying hazards using the STPA method and analyzing unsafe transitions between different LoA in systems. The approach is based on three LoA, ranging from Manual to Autonomous Control, which differs from the model proposed by Endsley and Kaber (1999).

However, there is still a lack of a method that guides the analysis of this system study until the selection of the LoA. The proposed method will be essential for identifying and refining the aircraft’s safety, particularly in the future eVTOL environment. The V-Model for the avionics domain is depicted in Fig. 3, based on the Guidelines for Development of Civil Aircraft and Systems (ARP4754A) (SAE 2024), illustrating the interaction between avionics development and safety processes.

The V-Model is divided into two parts, namely “System Allocation” and “System Integration”. Additionally, it is composed of stages from the ConOps until the system is operational and maintenance is required. Each stage is described below:

• ConOps: this stage outlines the high-level description of how the system will be used and its intended functionality.

• High-level requirements: these requirements are derived from the ConOps and define the overall objectives and capabilities of the system.

• System-level requirements: detailed requirements are developed based on the high-level requirements, specifying the functions, performance, and interfaces of the system as a whole.

• Subsystem requirements: these requirements further decompose the system-level requirements into specifications for individual subsystems or components.

• Detailed component design: the design of each system component is elaborated upon, including software, hardware, and interfaces.

• Implementation: this stage involves the actual development and construction of the system components based on the detailed design specifications.

• Component verification: each individual component is tested to ensure that it meets its specified requirements and functions correctly.

• Subsystem verification: subsystems are integrated and tested to verify that they function correctly together and meet their interface requirements.

• System verification: the integrated system is tested as a whole to verify that it meets all system-level requirements and functions as intended.

• High-level system verification: this stage ensures that the overall system satisfies the high-level requirements and objectives defined in the ConOps.

• Operation and maintenance of the system: once verified, the system is deployed for operational use. Operation and maintenance activities include monitoring, troubleshooting, and repairing the system to ensure continued functionality and performance.

• Traditionally, this model is employed in the iterations performed to meet each baseline. In addition to iterations, concepts such as phasing, goal setting, periodic assessments, role definition, and traceability (forward and backward) are traditionally included in these development processes (Taibi et al. 2015).

Figure 3
Avionics V-Model based on ARP4754A (SAE 2024) and Berkeley University of California (2024).

CASE STUDY

The sequence of results will respect the order of the proposed method.

1^st phase: define the ConOps

Description of envisioned system

At this stage, two rounds of face-to-face interviews were conducted, each involving two pilots from the Flight Research and Testing Institute (IPEV). These pilots possess extensive flying experience, totaling more than 1,500 flight hours, including proficiency in helicopters—an aircraft type more analogous to the eVTOL. In the first round, the goal was to establish a study context with questions such as those in Table 2.

Table 2
Interviews with pilots to gather information.

Interviews with pilots to gather information
1	What are the routine actions for landing helicopters in terms of safety?
2	What level of automation is available on the aircraft (e.g., H-36)?
3	What incidents have occurred during the landing of a highly automated aircraft? Could you describe the process until you reach the hazard?
4	What are the possible accidents and hazards in these cases? For example, accident “death or injury to people” and hazard “bird attack”.
5	What mitigating measures can be taken? To avoid an accident, what actions could or could not be taken?
6	In the case of quick decisions in a time of emergency landing, what skills/procedures/knowledge are important?

• Source: Elaborated by the authors.

The 1^st Phase of the proposed method is to understand the context of the study. Various approaches were taken to research eVTOL, including examining articles (cited in this article) and scientific publications, reaching out to project engineers at Empresa Brasileira de Aeronáutica (EMBRAER), and conducting interviews with pilots from the IPEV.

Embraer is a manufacturer of commercial jets and the world leader in the segment of up to 130 seats. Eve Air Mobility was created by Embraer to produce the eVTOLs.

The core mission of IPEV is to conduct flight testing, applied research, and specialized personnel training with a commitment to excellence, scientific rigor, and safety.

In ConOps, the following actors could be studied as shown in Table 3.

Table 3
Interviews with pilots to gather information.

Stakeholders and their objectives
1	Agência Nacional de Aviação Civil (ANAC) is a federal regulatory agency that holds the vital responsibility of overseeing civil aviation operations in Brazil. Its primary focus encompasses the regulation of both economic aspects and technical safety within the aviation sector.
2	Eve Air Mobility is a Brazilian company that plans to produce eVTOL aircraft, as well as UAM infrastructure.
3	Passengers will be the users of the air taxi service in the state of São Paulo, Brazil, who seek convenient transportation between cities.

• Source: Elaborated by the authors.

In this study, the goals and objectives of the predicted system were identified.

Mission needs: provide an optimized and safe intermunicipality transportation service.

Goal 1: provide an optimized means of transport that connects the states of Brazil.

Objectives are presented in Table 4.

Table 4
Interviews with pilots to gather information.

Objectives for Goal 1: To provide an optimized means of transportation that connects the states of Brazil
1	The passenger collection point must be close to one or more other means of transport (bus stop, subway etc.).
2	The duration of the journey en route should be a maximum of 3 hours.
3	The duration of the journey en route should be a maximum of 3 hours.
4	The duration of the journey en route should be a maximum of 3 hours.
5	The vehicle must carry 4 passengers.
6	The aircraft shall have eight electric propellers.
7	The aircraft must be powered by two or more high-voltage batteries.
8	The aircraft must be powered by two or more high-voltage batteries.
9	The aircraft must have electric motor power (12 rotors).
10	The aircraft must have helicopter-type two-skid landing gear.
11	The aircraft must have a noise level of 70 dB at 500 feet.
12	The aircraft must reach a maximum altitude of 6,600 feet.
13	The aircraft shall have a cruising speed of 150 mph and a range of 60 miles (96 km per hour).

• Source: Elaborated by the authors.

Goal 2: ensuring comfort and safety to passengers in the face of operational risks.

Objective are presented in Table 5.

Table 5
Interviews with pilots to gather information.

Objectives for Goal 2: To ensure comfort and safety for passengers in the face of operational risks
1	The operation or flight phase of the aircraft must receive the safest LoA between 1 to 10.
2	The sensor must identify objects at a safe distance to deflect.
3	The pilot or autopilot must maintain communication with one or more ground support centers.

• Source: Elaborated by the authors.

Physical environment

The environment in which the aircraft is expected to operate is an urban center; in Brazil, the context is a flight between São Paulo and São José dos Campos. With a humid subtropical climate, characterized by a notably dry winter and a very rainy summer. In this article, the approach and landing phase of the flight will be studied, as shown in Fig. 4.

Figure 4
Flight phases.

Support environment

After identifying the main actors, some of their possible interests were identified, as shown in Fig. 5.

Figure 5
Stakeholder interest in ConOps.

The planning and control of the operation will be conducted by the company EVE, encompassing both the automation of the aircraft and the logistics for user access to the service.

Operational scenarios, use cases, and/or design reference missions (DRM)

One operational scenario, in a common flight context, will involve the aircraft flying between SJC and SP, facing varying weather conditions such as fog, rain, or sun. As mentioned earlier, the context under study is eVTOL, so four main tasks can be identified, which can be performed by both the pilot and the autopilot, as shown in Fig. 6.

Figure 6
Use case.

In adverse contexts, the aircraft flies in heavy rain and lightning, close to buildings, close to other aircraft, close to moving objects (drone, kite, balloon), or obstacles (power wire, pole).

Impact considerations

eVTOL has the potential for both positive and negative impacts.

Positive impacts

• Improved Urban Mobility: eVTOLs can provide faster and more flexible transport solutions, especially in congested urban areas, contributing to reduced traffic and improved mobility.

• Time Efficiency: By avoiding ground traffic, eVTOLs can offer faster journeys, saving time for passengers.

• Accessibility in Remote Areas: They can be used to provide transport in remote or difficult-to-access areas, improving connectivity and facilitating access to essential services.

• Reducing polluting emissions by using clean energy sources, eVTOLs have the potential to reduce greenhouse gas emissions compared to traditional modes of transport.

• Technological Development: Research and development around eVTOLs drive advances in technologies related to electric aviation, batteries, and automation, contributing to technological innovation.

Negative impacts

• Necessary infrastructure: Successful implementation of eVTOLs requires appropriate infrastructure, including helipads, charging stations, and appropriate regulations, which can be a logistical challenge.

• Environmental Impact of Battery Production: Manufacturing batteries for eVTOLs can have a significant environmental impact, especially if associated waste and material extraction are not properly managed.

• Safety and Regulatory Issues: Introducing a new form of air transport requires robust regulations and adequate safety measures to prevent accidents and ensure the safety of passengers and people on the ground.

• High Initial Cost: eVTOLs can have a high initial cost, which can limit their accessibility and widespread adoption until costs decrease over time.

• Noise: Noise generated by eVTOLs, especially during vertical takeoff and landing, can be a concern in urban areas, impacting residents’ quality of life.

Risks and potential problems

Some of the main risks associated with eVTOLs include:

• Operational Safety: Risks include air collisions, mechanical failures, software errors, and other factors compromising eVTOL safety.

• Noise: Electric motor-generated noise poses risks to the hearing health of ground-level individuals.

• Regulation and Certification: Challenges involve developing efficient regulations and certification processes ensuring safety standards for eVTOL aerial operations.

• Air Traffic Integration: Risks related to the safe integration of eVTOLs with conventional air traffic, covering route coordination, collision avoidance, and airspace management.

• Infrastructure: Adequate infrastructure, like helipads and charging stations, presents logistical and investment challenges for supporting eVTOL operations.

• Climate Conditions: Adverse weather, including strong winds, storms, and fog, poses additional risks to eVTOL safe operation.

• Battery Failures: Risks include electrical battery failures like overheating or sudden malfunctions, posing safety threats.

• Costs and Viability: Risks pertain to high development, production, and operation costs impacting eVTOL economic viability.

• Public Acceptance: Public resistance or fear regarding the presence and operation of eVTOLs, whether due to safety concerns, ticket prices, or other factors, can be a challenge.

In this sense, it is essential that industry, regulators, and other stakeholders proactively address these risks to ensure that the integration and operation of eVTOLs are safe, efficient, and accepted by society. Continued research, effective regulation, and technological advances play critical roles in mitigating these risks.

Then, from ConOps, the engineer needs to analyze which LoA will be used for which aircraft or flight phase to define the high-level requirements, and STPA will help in this work.

2^nd Phase: apply the STPA method

Define the analysis proposal

Based on this information, the application of the STPA begins with the definition of the analysis proposal, which in this study was the safety analysis of the eVTOL landing in an urban center. Subsequently, the study identified the following losses:

• L-1: Loss of life or injury to individuals.

• L-2: Loss of property.

• L-3: Damage to the aircraft.

• L-4: Loss of reputation.

In addition, it is important to mention, as hazards: H-1 = lack of power to fly the desired vertical profile; H-2 = bird strike; H-3 = collision with balloons, kites, drones, etc.; and H-4 = hitting an obstacle (antenna, power wire, etc.).

Identify controller, controlled process, CAs, and feedback

The second step in STPA involves modeling the control structure. Figure 7 illustrates a high-level control structure of the system, where both the pilot and the autopilot serve as controllers, and the eVTOL aircraft acts as the controlled process. This depiction facilitates the identification of specific CAs for more in-depth analysis.

Figure 7
Model of control structure with pilot and autopilot.

The pilot’s decision-making is influenced by external factors like adverse weather and in-cabin conversations, alongside airline protocols and training. Conversely, the autopilot operates uninfluenced by external factors, constrained by programmed behaviors and sensor data. Both share responsibilities like monitoring the flight, checking the landing site, selecting speed, and executing maneuvers. The pilot controls actuators, engages/disengages the autopilot, and receives data from displays. The autopilot commands actuators, receives data from actuators and sensors, and relies on actuator commands and sensor data for operation in the eVTOL.

Identify UCAs in the four types

For this step, the second round of interviews was carried out with three pilots from IPEV (different from the first round). The questions were:

• What unsafe actions could occur if certain tasks are performed in manual or automation mode? Please consider the following tasks:

  • Monitoring flights;

  • verifying landing sites;

  • choosing speed;

  • maneuvering.

• Of the above tasks for landing, which do you consider most critical and why?

• If these tasks were autonomous (with no possibility of human intervention), would it be safer?

  At this phase, a specific context was chosen for further study based on the information collected and interviews, with the following characteristics.

• Vertistops above a metropolis, like São José dos Campos, in the state of São Paulo, Brazil.

• Visual meteorological conditions (VMC).

• Daytime.

• The crew will consist of one pilot and four passengers.

• The flight phase is landing.

Table 6 presents CAs and UCAs.

Table 6
UCAs.

Nº	CA	Not providing	Providing causehazard	Too early/too late	Stop too soon/applying too long
1	Monitor flight	1.1 The pilot does not monitor flight during the landing. [H2, H3, H5] [L1-L4]
2	Verify landing site		2.1 The pilot verifies an unsafe landing site. [H2, H3, H5] [L1-L4]
3	Choose speed	3.1 The pilot does not choose a safe speed during landing. [H1, H2, H3, H5] [L1-L4]
4	Maneuver	4.1 The pilot does not handle the controls with confidence. [H2, H3, H5] [L1-L4]
5	Monitor flight	5.1 The autopilot does not fully monitor the area surrounding the aircraft during the landing. [H2, H3, H5] [L1-L4]
6	Verify landing site		6.1 The autopilot monitors the flight unsafely during landing. [H2, H3, H5] [L1-L4]
7	Choose speed				7.1 The autopilot does not select a slowdown soon enough to avoid impact with an object during the landing. [H2, H3, H5] [L1-L4]
8	Maneuver				8.1 Autopilot does not maneuver (right, left) in time to clear hazards during landing. [H2, H3, H5] [L1-L4]

• Source: Elaborated by the authors.

Identify loss scenarios and safety requirements

Subsequently, the loss scenarios of each UCA (why the UCA would happen) and respective safety requirements were identified (Table 7).

Table 7
UCAs with respective loss scenarios and safety requirements.

CA	UCA	Loss scenarios	Safety requirements
1	1.1	The pilot does not monitor the flight because he is inattentive or trusts that he will be warned of any moving object or obstacle nearby.	If the pilot does not monitor the flight due to being inattentive, any breach of safety minimums must be promptly detected, and appropriate measures need to be taken to prevent collisions or potential hazards.
2	2.1	The pilot verifies an unsafe landing site because he does not know the region and it is an emergency landing.	If the pilot finds an unsafe landing site, ground personnel must monitor the route and alert the pilot to the best landing sites.
3	3.1	The pilot does not choose a safe speed during landing, because he is under pressure to finish the experience.	If the pilot does not choose a safe speed during landing, then the ground team must monitor the route and alert the pilot.
4	4.1	The pilot does not confidently maneuver the controls because he suffers from a loss of situational awareness.	If the pilot does not confidently maneuver the controls, then the ground team must monitor the route and alert the pilot.
5	5.1	The autopilot does not fully monitor the area around the aircraft during landing, because there are too few sensors to cover the total area.	The autopilot does not fully monitor the area around the aircraft during landing, and then sensors need to be located at strategic points to pick up different angles if an object approaches.
6	6.1	The autopilot monitors the flight without safety because the maps of the region are out of date.	The autopilot monitors the unsafety flight, and then the ground team must monitor the route and alert the pilot.
7	7.1	The autopilot does not select a slowdown in time to get away from an object during the landing, because the object is out of sensor range.	The autopilot does not select a slowdown in time to get away from an object during the landing, and then the sensors need to be located at strategic points to capture different angles if an object approaches.
8	8.1	Autopilot does not maneuver in time to clear hazards during landing because the object is out of sensor range.	Sensors need to be located at strategic points to capture different angles if an object approaches.

• Source: Elaborated by the authors.

3^rd Phase: Analyze the safest LoA

Analysis of the safest LoA for each CA and corresponding scenario

For the selection of the safest LoA, an analysis of the scenarios is carried out (Table 8). The engineer or product analyst will point out the arguments that led them to choose between the pilot, the autopilot, or both.

Table 8
Analysis of scenarios.

no	CAs	Responsible	Function	Scenarios	Better
1	Monitor flight	Pilot	Monitoring	1.1 and 5.1	P/AP
In this scenario, the pilot neglects monitoring the flight, either due to inattentiveness or reliance on the assumption that any nearby moving object or obstacle will trigger a warning. Concurrently, the autopilot monitors the flight, but the object is beyond the sensor range. In such a situation, it is essential for both the autopilot and the pilot to engage in monitoring. The autopilot offers the advantage of continuous vigilance without distractions, while the pilot contributes the benefit of long-range vision.
2	Verify landing site	Pilot	Generating	2.1 and 6.1	P/AP
In this scenario, the pilot considers an unsafe landing site as an alternative due to unfamiliarity with the region, especially during an emergency landing. Simultaneously, the autopilot monitors the flight without ensuring safety due to outdated maps of the region. In such a case, during routine operating conditions, the autopilot would navigate to the landing site. However, in the scenario of an emergency landing, it would be more appropriate for the pilot in command to analyze and determine the safest area for landing.
3	Choose speed	Pilot	Selecting	3.1 and 7.1	AP
In this scenario, the pilot may not select a safe landing speed due to the pressure to conclude the workday, and the autopilot may not adjust the speed safely in time to clear hazards during landing because the object is beyond the sensor range. In such a situation, the autopilot would possess the precision to control the speed (increasing and decreasing) in the safest manner, thus avoiding a sudden stop. Meanwhile, the pilot would focus on monitoring and discerning the object. Additionally, it is recommended to have sensors covering various areas around the aircraft
4	Maneuver	Pilot	Implementing	4.1 and 8.1	AP
In this scenario, the pilot may struggle to confidently maneuver the controls due to a loss of situational awareness, and the autopilot may not maneuver in time to clear hazards during landing because the object is beyond the sensor range. In such a case, the maneuvering of the aircraft (guiding the directions) would be more effectively performed by the autopilot, especially in situations where the pilot experiences a loss of situational awareness. It is recommended to incorporate sensors that capture different angles of the area around the aircraft.

• Source: Elaborated by the authors.

Comparison between the safest level of autonomy for each CA and the information outlined in the automation table

Throughout this study, if the scenario analysis option selected in Table 9 did not align with the Safest Level of Autonomy (LoA) presented in Table 1, which are already established in the field and widely used to define LoA, a reevaluation of the scenarios would be conducted. This occurred during the analysis, specifically in the “Generating” and “Control Action - Verify landing site” functions. Initially, controlling only by the pilot seemed beneficial, but it did not align with LoA 8 (the closest option). Upon reanalyzing the scenario and considering information from the interviews, it was concluded that having control by both the pilot and the autopilot was beneficial.

Table 9
Selection of the safest preliminary LoA.

No	CAs	Function	LoA
			1	2	3	4	5	6	7	8	9	10
1	Monitor flight	Monitoring	P	P/AP	P/AP	P/AP	P/AP	P/AP	P/AP	P/AP	P/AP	AP
2	Verify landing site	Generating	P	P	P	P/AP	P/AP	P/AP	AP	P/AP	AP	AP
3	Choose speed	Selecting	P	P	P	P	P	P/AP	P	AP	AP	AP
4	Maneuver	Implementing	P	P/AP	AP	P/AP	AP	AP	AP	AP	AP	AP

• Source: Elaborated by the authors.

The analysis based on the table presented by Endsley and Kaber (1999) results in LoA 8 (automated decision-making) as the best choice, given its alignment with the scenarios under consideration (as shown in Table 4).

Conclusive analysis of the LoA for each designated function

In the context of flight monitoring, a shared command approach is preferable, wherein the pilot issues warnings about potential obstacles and checks blind spots in scenarios that pose possible hazards. Both the pilot and the autopilot can perform functions such as “‘generating alternatives” and “checking the landing site.” The autopilot exhibits greater precision in tasks involving “selecting the best alternative” and “choosing speed.” Additionally, it proves more accurate and effective in executing the “implementation” function and controlling the “maneuver” action, especially in common scenarios.

Given the variability in human decision-making, error prevention procedures become essential, while programming the autopilot necessitates thorough scenario exploration. Early identification, classification, and mitigation of hazards play an essential role in enhancing both pilot decision-making and autopilot programming. Although automation positively impacts human performance under routine conditions, completely removing all functions poses challenges in recovering performance during system failures.

The study underscores the pivotal role of the pilot in addressing unforeseen scenarios during eVTOL deployment. In situations where the autopilot encounters unscheduled scenarios, the pilot can intervene to correct the flight path. Furthermore, in the event of a pilot error during the approach, executing a go-around remains a viable option. The interviews emphasize the importance of the “generating” function as the most critical, showcasing the pilot’s expertise in analyzing critical flight conditions and generating alternative strategies.

CONCLUSION

This article presents a new analysis method that ranges from the ConOps to the selection of the appropriate LoA for a given CA. It contributes and explores a combination of methods not previously investigated in academia and addresses an essential concern for the eVTOL industry, necessitating thorough analysis and practical implementation for operational safety.

The study specifically delves into safety considerations regarding automation levels and hazard scenarios to determine the safest LoA. The method has potential applications during the high-level design phase in the development of eVTOL. The research question, “What is the best method for selecting the safest LoA for operation?” is addressed through a three-phase approach, applying the study of ConOps, STPA, and LoA. Findings, particularly from interviews, indicate that LoA 8 (automated decision-making) is the safest for the analyzed case.

Limitations of the research include its focus on an eVTOL landing operation in the upcoming years, with the aircraft still in development and limited technical information available. Time constraints during the information-gathering process with available pilots were also encountered. While the article provides valuable insights, it lacks consideration of how cognitive load, attention, and fatigue can influence findings. Future research should explore these potential influences. Another recommendation for future research is to explore a group of LoAs, which could be beneficial rather than focusing solely on one LoA. For example, considering a combination of LoAs 8 and 9 when tasks such as “choose speed” and “maneuver” are performed solely by the autopilot could provide insights into advancing automation in monitoring and generating.

In scenario studies, the necessity for pilot knowledge and skills for operation suggests that the current scenario is not ready for full autonomy. The method was tested for eVTOL landing over a metropolis, and different contexts, locations, and flight phases necessitate new analyses and scenarios using the same method. It is concluded that the safest LoA for each CA entails specific safety requirements for operation.

The success of the method is evident in its ability to identify the safest LoA through a systematic approach, particularly highlighting the decision-making shared by the pilot and the autopilot in the specific study of each CA.

The contributions of applying this method to the eVTOL industry are significant, offering a comprehensive system for assessing and selecting the most appropriate automation levels. This method ensures enhanced operational safety, addresses critical concerns, and paves the way for the development and implementation of safer eVTOL systems.

ACKNOWLEDGEMENTS

We are grateful for the support received by the IPEV for supporting this work; five pilots participated in interviews for data collection.

• Peer Review History: Single Blind Peer Review.

• FUNDING

  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
  Finance Code 001

DATA AVAILABILITY STATEMENT

All datasets were generated or analyzed in the current study.

REFERENCES

• [NASA] National Aeronautics and Space Administration (2019) Appendix S: concept of operations annotated outline. [accessed Jul 05 2023]. https://www.nasa.gov/seh/appendix-s-concept-of-operations
  » https://www.nasa.gov/seh/appendix-s-concept-of-operations
• [SAE] SAE International (2024) Guidelines for Development of Civil Aircraft and Systems – ARP4754. [accessed Jul 05 2023]. https://www.sae.org/standards/content/arp4754a/
  » https://www.sae.org/standards/content/arp4754a/
• Abreu-Júnior CE (2008) Automação no cockpit das aeronaves: um precioso auxílio à operação aérea ou um fator de aumento da complexidade no ambiente profissional dos piltotos? R Ação Ergon 3(2):6-15. https://revistaacaoergonomica.org/article/627d7785a9539511202397b2
  » https://revistaacaoergonomica.org/article/627d7785a9539511202397b2
• Agustinho JR, Bento CAM (2022) Operational requirements analysis for electric vertical takeoff and landing vehicle in the Brazilian regulatory framework. J Aerosp Technol Manag 14. https://doi.org/10.1590/jatm.v14.1269
  » https://doi.org/10.1590/jatm.v14.1269
• Albano LM, Fregnani JATG, Andrade D (2022) Analysis of automation mode confusion with Brazilian airline pilots. J Aerosp Technol Manag 14. https://doi.org/10.1590/jatm.v14.1280
  » https://doi.org/10.1590/jatm.v14.1280
• Berkeley University of California (2024) Connected Corridors Program. UC Berkeley. [accessed Jul 05 2023]. https://connected-corridors.berkeley.edu/#:~:text=Connected%20Corridors%20is%20a%20collaborative,managing%20transportation%20corridors%20in%20California
  » https://connected-corridors.berkeley.edu/#
• Bjerga T, Aven T, Zio E (2016) Uncertainty treatment in risk analysis of complex systems: the cases of STAMP and FRAM. Reliab Eng Syst Saf 156:203-209. https://doi.org/10.1016/j.ress.2016.08.004
  » https://doi.org/10.1016/j.ress.2016.08.004
• Borges SFS (2019) Integração de métodos para análise de riscos em projetos de pesquisa aeroespaciais (master’s thesis). São José dos Campos: Instituto Tecnológico de Aeronáutica. In Portuguese.
• Borges SFS, Albuquerque MAF, Cardoso-Junior MM, Belderrain MCN, Costa LEL (2021) Systems theoretic process analysis (STPA): a bibliometric and patents analysis. Gest Prod 28(2). https://doi.org/10.1590/1806-9649-2020v28e5073
  » https://doi.org/10.1590/1806-9649-2020v28e5073
• Cardoso, SHSB, Oliveira MVR, Godoy JRS (2022) eVTOL certification in FAA and EASA performance-based regulation environments: a bird strike study-case. J Aerosp Technol Manag 14. https://doi.org/10.1590/1806-9649-2020v28e5073
  » https://doi.org/10.1590/1806-9649-2020v28e5073
• Endsley MR, Kaber DB (1999) Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics 42(3):462-492. https://doi.org/10.1080/001401399185595
  » https://doi.org/10.1080/001401399185595
• Endsley MR, Kaber DB (2004) Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety. Process Saf Prog 16(3):126-131. https://doi.org/10.1002/prs.680160304
  » https://doi.org/10.1002/prs.680160304
• Eve Air Mobility (2024) Mobility Reimagined. Eve Air Mobility. [Jun12 2024]. https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.eveairmobility.com%2F&psig=AOvVaw0tVsKCcWExDByabJ4JfE-z&ust=1720048849393000&source=images&cd=vfe&opi=89978449&ved=0CBEQjRxqFwoTCJiSibi_iYcDFQAAAAAdAAAAABAE
  » https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.eveairmobility.com%2F&psig=AOvVaw0tVsKCcWExDByabJ4JfE-z&ust=1720048849393000&source=images&cd=vfe&opi=89978449&ved=0CBEQjRxqFwoTCJiSibi_iYcDFQAAAAAdAAAAABAE
• Franciscone BG, Fernandes E (2023) Challenges to the operational safety and security of eVTOL aircraft in metropolitan regions: a literature review. Journal of Airline Operations and Aviation Management (1):45-56. https://doi.org/10.56801/jaoam.v2i1.2
  » https://doi.org/10.56801/jaoam.v2i1.2
• Johnson A (2011) Examining the foundation: were Heinrich’s theories valid? Do they still matter? Safety+Health 210;(1). https://www.safetyandhealthmagazine.com/articles/6368-examining-the-foundation
  » https://www.safetyandhealthmagazine.com/articles/6368-examining-the-foundation
• Kaber DB, Endsley MR (2007) The effects of level of automation and adaptive automation on human performance, situation awareness, and workload in a dynamic control task. Theor Issues Ergon Sci 5(2):113-153. https://doi.org/10.1080/1463922021000054335
  » https://doi.org/10.1080/1463922021000054335
• Kunio Y (2021) Introduction of system safety analysis method (STAMP/STPA) in the development of the PCB inspection system. Omron Technic 53:006EN 2021.5.
• Laarmann L, Thoma A, Misch P, Röth T, Braun C, Watkins S, Fard M (2023) Automotive safety approach for future eVTOL vehicles. CEAS Aeronaut J 14(2):369-379. https://doi.org/10.1007/s13272-023-00655-0
  » https://doi.org/10.1007/s13272-023-00655-0
• Leveson NG (2002) A new approach to system safety engineering. Cambridge: Massachusetts Institute of Technology. https://doi.org/10.1.1.139.3388
  » https://doi.org/10.1.1.139.3388
• Leveson NG (2011) Engineering a safer world: systems thinking applied to safety (engineering systems). Cambridge: MIT Press. https://direct.mit.edu/books/oa-monograph/2908/Engineering-a-Safer-WorldSystems-Thinking-Applied
  » https://direct.mit.edu/books/oa-monograph/2908/Engineering-a-Safer-WorldSystems-Thinking-Applied
• Oginni D, Camelia F, Chatzimichailidou M, Ferris TLJ (2023) Applying system-theoretic process analysis (STPA)-based methodology supported by systems engineering models to a UK rail project. Safety Sci 167:106275. https://doi.org/10.1016/j.ssci.2023.106275
  » https://doi.org/10.1016/j.ssci.2023.106275
• Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern A:Syst Hum 30(3):286-297. https://doi.org/10.1109/3468.844354
  » https://doi.org/10.1109/3468.844354
• Polet P, Vanderhaegen F, Amalberti R (2003) Modelling border-line tolerated conditions of use (BTCU) and associated risks. Safety Sci 41(2-3):111-136. https://doi.org/10.1016/S0925-7535(02)00037-1
  » https://doi.org/10.1016/S0925-7535(02)00037-1
• Rasmussen J (1997) Risk management in a dynamic society: a modelling problem. Safety Sci 27;(2-3):183-213. https://doi.org/10.1016/S0925-7535(97)00052-0
  » https://doi.org/10.1016/S0925-7535(97)00052-0
• Ribeiro JK, Borille GMR, Caetano M, Silva EJ (2023) Repurposing urban air mobility infrastructure for sustainable transportation in metropolitan cities: a case study of vertiports in São Paulo, Brazil. Sustain Cities Soc 98:104797. https://doi.org/10.1016/j.scs.2023.104797
  » https://doi.org/10.1016/j.scs.2023.104797
• Rodrigues RG, Fulindi JB, Oliveira DBP, Moraes AO, Marini-Pereira L (2022) Safety analysis of GNSS parallel runway approach operation at Guarulhos International Airport. J Aerosp Technol Manag 14. https://doi.org/10.1590/jatm.v14.1260
  » https://doi.org/10.1590/jatm.v14.1260
• Sheridan, TB, Verplank WL (1978) Human and computer control of undersea teleoperators. Paper presented 1978 14th Annual Conference on Manual Control. NASA; Washington, D.C., USA. https://ntrs.nasa.gov/citations/19790007441
  » https://ntrs.nasa.gov/citations/19790007441
• Shorrock S (2007) Barriers and accident prevention. Ergonomics 50(6):961-962. https://doi.org/10.1080/00140130600971077
  » https://doi.org/10.1080/00140130600971077
• Su J, Huang H, Zhang H, Wang Y, Wang F (2024) eVTOL performance analysis: a review from control perspectives. IEEE Trans Intell Vehicl. https://doi.org/10.1109/TIV.2024.3387405
  » https://doi.org/10.1109/TIV.2024.3387405
• Taibi D, Lenarduzzi V, Dieudonné L, Plociennik C (2015) Towards a classification schema for development technologies: an empirical study in the avionic domain. International Journal on Advances in Software 8(1-2):125-135. https://api.semanticscholar.org/CorpusID:58794341
  » https://api.semanticscholar.org/CorpusID:58794341
• Theunissen E, Suarez BR (2015) Choosing the level of autonomy: options and constraints. NATO Communications and Information Agency. The Hague: Netherlands. Autonomous systems: issues for defence policymakers; p. 34. https://www.researchgate.net/publication/282338125_Autonomous_Systems_Issues_for_Defence_Policymakers#pfbf
  » https://www.researchgate.net/publication/282338125_Autonomous_Systems_Issues_for_Defence_Policymakers#pfbf
• Thompson EL, Taye AG, Guo W, Wei P, Quinones M, Ahmed I, Biswas G (2022) A survey of eVTOL aircraft and AAM operation hazards. Paper presented AIAA AVIATION 2022 Forum. AIAA; Reston, USA. https://doi.org/10.2514/6.2022-3539
  » https://doi.org/10.2514/6.2022-3539
• U.S. Department of Justice (2018) Concept of operations (CONOPS). [accessed Jul 05 2023]. https://www.justice.gov/archive/jmd/irm/lifecycle/appendixc9.htm
  » https://www.justice.gov/archive/jmd/irm/lifecycle/appendixc9.htm
• Xiang S, Xie A, Ye M, Yan X, Han X, Niu H, Li Q, Huang H (2024) Autonomous eVTOL: a summary of researches and challenges. Green Energy and Intelligent Transportation 3(1):100140. https://doi.org/10.1016/j.geits.2023.100140
  » https://doi.org/10.1016/j.geits.2023.100140
• Yang X, Utne IB, Sandoy SS, Ramos MA, Rokseth B (2020) A systems-theoretic approach to hazard identification of marine systems with dynamic autonomy. Ocean Eng 217:107930. https://doi.org/10.1016/j.oceaneng.2020.107930
  » https://doi.org/10.1016/j.oceaneng.2020.107930

Edited by

Section editor: Eric Njoya https://orcid.org/0000-0002-1799-9469

Publication Dates

• Publication in this collection
  23 Sept 2024
• Date of issue
  2024

History

• Received
  05 Aug 2023
• Accepted
  20 June 2024