Requirements formally document what a system must achieve in terms of behavior, performance, and functionality to meet product and business objectives. This encompasses both nominal operation and the system’s reliability and resilience in handling faults or scenarios beyond the defined Operational Design Domain (ODD).
In addition to defining system expectations, requirements establish clear ownership for individual components, fostering accountability and coordination across teams. They also provide the foundation for a Verification and Validation (V&V) program, enabling a structured approach to testing and ensuring the system meets its requirements under all intended conditions.
Iterating on requirements throughout the design process is critical to account for evolving insights, technical challenges, and stakeholder needs. Regularly validating requirements against business goals and operational constraints helps avoid misalignment and downstream integration issues. Teams can refine requirements to balance performance, cost, and technical feasibility, ensuring all subsystems work cohesively.
By serving as a shared framework across disciplines, well-crafted requirements reduce ambiguity, mitigate risks, and support efficient, predictable development cycles.
Some challenges and lessons learned related to requirements for a safety critical robotics system include:
Over-decomposition of Requirements:
Safety-critical systems require a structured decomposition of requirements into tiers, breaking the system into smaller subsystems. This process evolves as the product and the Verification and Validation (V&V) strategy mature.
A key challenge is determining the appropriate level of decomposition. For instance, should every function within every library running on the system have its own specific requirements? Over-decomposition can lead to excessive V&V work without adding meaningful value.
To assess whether a lower-level requirement is necessary, consider the following:
- Testing Scope: Can the proposed requirement be adequately tested at a higher level?
- Triage Efficiency: If failures occur, will it still be easy to trace the issue without the additional requirement?
- Analysis Redundancy: Does the proposed requirement address risks, behaviors, or scenarios already captured by higher-level requirements or their associated testing and analysis?
- Design Flexibility: Will the additional requirement unnecessarily constrain design or implementation choices? Overly prescriptive requirements can stifle innovation and adaptability.
Decomposition of Supporting Software Components
Determining the scope of requirements involves carefully considering which components directly influence the system’s performance, safety, and functionality. While it’s essential to capture requirements for mission-critical or safety-sensitive elements, not all supporting components warrant the same level of formal requirement development. In addition to the software running on the deployed system, robotics products often include supporting components, such as:
- Cloud-Based Infrastructure: Software or services in the cloud that support robot operations, such as fleet management, data aggregation, or remote monitoring.
- Offline Pipelines: Tools for generating or preprocessing static assets like maps, sensor calibration data, or machine learning models.
- Development and Testing Tools: Software used to debug, test, or simulate the system during development.
- Human Interfaces: Interfaces or dashboards that allow humans to interact with the system, including operational controls or maintenance tools.
- Third-Party Dependencies: External libraries, frameworks, or tools integrated into the system.
Focus should be placed on those components that have a direct, measurable impact on the deployed system or its operations, ensuring that unnecessary V&V work is avoided. To assess whether a component warrants dedicated requirements, consider the following criteria:
- Impact on System Behavior: Does a failure in the component directly affect the deployed system’s performance, safety, or functionality? Components that do not influence the system’s behavior in deployment may not need formal requirements.
- Independent Verification of Outputs: Are the component’s outputs independently verifiable? For instance, if a map or model meets all deployment requirements, the upstream processes generating them may not need exhaustive testing.
- Risk of Hidden Dependencies: Does the component introduce risks or dependencies that could propagate failures? For example, third-party libraries may require some level of validation to ensure compatibility and reliability.
Under-decomposition of Requirements
Decomposing system-level requirements into subsystem-specific requirements can be challenging. Some system behaviors may be tightly coupled with a single subsystem, making it difficult to distinguish between system-level and subsystem-specific behavior. In other cases, it may be difficult to differentiate between system-level behavior and the behavior of specific subsystems, such as motion planning.
However, it is essential to perform this decomposition to ensure effective testing and validation. Subsystem-level testing allows for isolating specific deficiencies within a subsystem, such as motion planning, when it is provided with ideal inputs. This focused approach can uncover issues that might not be visible at the system level.
Conversely, system-level testing involving multiple subsystems (e.g., motion planning, localization, and perception) can reveal integration-related deficiencies that only surface when subsystems are tested together.
Effective Requirement Development and Review
Since requirements both define what your product should do and serve as the foundation for a robust V&V process, multiple teams are vested in their development. Generally, the following roles are involved in the requirements development and refinement process.
- System Safety Engineer: responsible for the administration of the requirement documentation. They ensure that the requirements language meets style and quality standards, traceability is captured between requirements, and driving the requirements review process.
- Engineering: responsible for the implementation of the requirement as well as developing the test suite to verify that the product is meeting the requirement. Since requirements are often contracts between teams, multiple engineers could be impacted by the requirement discussions.
- Test Engineer: responsible for generating a test strategy for the requirements, determining the proper test modality and test conditions appropriate for requirement verification.
It’s helpful to have a single DRI for each requirement. This provides both accountability and historical context. If there are questions about a requirement long after its initial development, it’s helpful to understand the context of why the requirement was initially developed. If the requirement is owned by a team instead of an explicit individual, this context can get lost over time.
When drafting a requirement, some best practices help ensure that the text meets minimum criteria to reduce the number of iteration cycles. The requirement should be written in a way that it can be directly tested. It should be atomic, meaning that it represents a single concept. The requirement should be unambiguous. Lastly, any acronyms in the text should be defined in the requirement itself or a general glossary.
Also, given the cross functional nature of the requirements development and maintenance process, live reviews are often the most efficient way to review and approve requirements. Asynchronous work can help get baseline alignment or prepare draft text, but it’s hard to distill all the conversations into a concise requirement that would facilitate easy offline review. Generally, getting everyone together for review and alignment goes faster. Multiple review cycles can drag out the process, resulting in a loss of momentum.
Instability in requirements causes larger downstream impacts. If requirements are constantly shifting, then the implementation, test modalities, test creation, and test failure analysis are all also unstable resulting in delays to complete the V&V process. We want to ensure that any requirements discussions are rapidly closed out. We also want to minimize as much as possible, large last minute requirements changes
Balancing Requirement Iteration and Stability
In an ideal product development process, a comprehensive set of requirements would drive engineering efforts from the outset. However, early development often prioritizes rapid iteration and achieving basic functionality for limited deployments over formal requirements definition.
As the product transitions from research and prototyping to a production-quality system, the system architecture and core functionality stabilize. This stabilization provides an opportunity to formally document the system’s desired behavior, performance, and functionality, enabling the V&V process to proceed effectively before the next release.
Requirements can be sourced from various existing documentation, such as Interface Control Documents, design documents, or Product Requirement Documents. Once a baseline set of requirements is established, insights gained from testing—whether through real-world deployments (online) or simulated environments (offline)—can be used to refine and iterate on the requirements.
Importantly, requirements should not only be created or modified but also evaluated for relevance over time. Some requirements may become obsolete as the system design or performance matures. Each requirement introduces downstream V&V costs, including test creation, coverage verification, failure triage, and result adjudication. While implementing a requirement may seem straightforward, the cumulative cost of verification can be significant.
To achieve an efficient and effective V&V process, it’s critical to focus on high-impact requirements that directly contribute to product reliability, safety, and performance. This ensures that resources are directed toward areas that provide the greatest value in advancing the system toward deployment.
Additionally, the downstream V&V costs associated with each requirement make it essential to minimize the frequency and scope of late-stage requirement changes. Even minor changes can necessitate updates to test creation, tooling, and failure adjudication, which can collectively delay the release timeline. Ensuring stability in requirements during the final stages of development helps streamline the V&V process and reduces the risk of unanticipated delays.