Intelligent test automation gives Orion spacecraft a boost

public://webform/writeforus/profile-pictures/_w5a1538.jpg
Antony Edwards, CTO , Eggplant

The role of testing has expanded far beyond making sure that an e-commerce website can cope with a traffic surge during the holiday shopping season. In the world of aerospace and defense (A&D), and for the Orion spacecraft project in particular, software testing is mission-critical.

To confirm that all aspects of the mission are vigorously tested, NASA is using intelligent test automation in the Orion space program to ensure that onboard software and equipment works as expected and doesn't have any faults. 

Orion, still in development, is a human spacecraft for deep-space missions. It's scheduled to launch in 2019, although successful test flights have already taken place.

Space may be the ultimate challenge for test automation strategies, but test engineers in any industry can benefit from realigning their approach to testing around the core steps to intelligent automation. Put simply, if it's helping humans travel to deep space, it's probably worth exploring how it can help your organization.

World Quality Report 2018-19: The State of QA and Testing

The ultimate in mission-critical testing

Whether they make software to fuel space exploration, for aircraft, or for command-and-control systems, companies and contractors in the A&D sector are held to higher standards for accuracy, quality, reliability and delivery.

Complicating the situation is the fact that software is becoming increasingly complex: Systems including weapons, command and control, spacecraft, and radar are all now running millions more lines of code. This requires A&D QA teams to move away from focusing solely on code compliance to ensure operator safety.

With automated testing, A&D organizations can test hundreds of scenarios quickly and no longer need to spend time manually testing software. Automated testing can increase and scale alongside technology, learning and adapting to new processes and systems as the software becomes more intricate.

AI and automation have the potential to significantly speed up technology advancements and safety in the business of space exploration. Additionally, automated intelligent testing ensures that the software delivers the required outcomes both on land and in space.

Intelligent testing in space

NASA is using an intelligent approach to testing in the development of the Orion spacecraft as it readies the craft to travel into deep space.

To monitor the status of the spaceflight and provide instructions to the crew, the cockpit has software-based digital displays that replace hundreds of pounds of paper documentation. 

NASA engineers use automated testing to ensure that the cockpit displays accurate information so that, for example, astronauts operating Orion or doing a spacewalk see accurate information. This is truly the definition of mission-critical.

Before automated testing, it took many person-hours to manually verify that the software behind spacecraft displays was working as designed. A more automated, robust, thorough verification method was needed to remove the risk of human error.

Testing doesn't stop at launch. Once Orion is in space, the mission will use intelligent testing to continue monitoring the mission's progress and to identify recommended improvements. Mission control will track the spaceflight and provide instructions to the crew, but intelligent testing on the craft gives astronauts a way to monitor vehicle status and to control the vehicle independently, especially in emergency situations such as during a loss of communication.

[ Webinar: Agile Portfolio Management: Three best practices ]

Software testing challenges

Software testing in A&D frequently consumes a large portion of product development costs across vast, multiyear projects, with ambitious contract delivery dates. That's why the industry is rethinking its approach to testing and exploring how intelligent automation can help solve some of the issues it faces.

As in commercial business, A&D organizations face pressure to cut software testing costs. Ensuring that legacy systems are compatible with modern systems is another critical issue that many organizations are struggling to navigate with legacy testing tools.

Testing dynamic GUIs presents another challenge, since the user interfaces are predominantly complex, animated, and touch-enabled. Traditional testing tools focus on text boxes, labels, buttons, and tabs, which is not effective for testing a plane on a radar screen.

Aerospace also faces a shortage of talent, so having highly skilled and hard-to-replace engineers carry out mundane manual testing is neither productive nor effective. Refocusing them on developing new capabilities and adding value is critical. Intelligent automation is helping address these software testing challenges.

Six steps to intelligent automation

NASA and other A&D organizations must follow these six core steps to maximize mission success. 

1. Tap into domain experts

This will help you model code-less user journeys and desired outcomes. Bring experts in to help build the models, the predicted user journeys, and critical user journeys to drive test automation.

2. Pursue AI-assisted test automation

Once you have your initial models in place, bring in artificial intelligence software as the brain of the program. Use it to execute defined scripts and requirements, and then, based on its understanding, have it take over generating user journeys. It can also be left in an always-on state to look for bugs. It will then package the results into an archive for developers.

3. Automation should be non-invasive 

Any automation tool you use should connect to any system, including proprietary hardware and system software, without requiring modifications to the system under test.

4. Test the UX, not just code

It is not enough to simply test code. Test the experience as well to see if the software is actually delivering what the user needs. This is especially crucial for graphical information systems and situation awareness, where information is complex, highly graphical, and must be absorbed quickly.

5. Deploy predictive analytics

AI-assisted and automated testing can support predictive analytics to determine launch readiness by testing many scenarios. It helps teams predict quality issues that might occur, intelligently navigate applications, and identify and resolve issues quickly. 

6. Continue with production UX monitoring and analytics

Once the software goes live, you need to continue monitoring it in production to fix bugs and add capabilities.

Embrace intelligent automation

By adopting this automation strategy, testing in A&D moves from being compliance-driven to making sure testing is meeting and exceeding desired mission outcomes.

At NASA, software testing can be a matter of life or death. To maximize mission outcomes, testing needs to pivot from being compliance-driven to guaranteeing that it is meeting and exceeding its desired result. By deploying intelligent automation, NASA will be able to accelerate the delivery and quality of its complex mission-critical software systems—and so can you.

Intelligent automation allows testing to move from being a manual, compliance-driven activity to one that helps drive business success. If your current testing strategy is not directly supporting your business objectives, it may be time to learn from the A&D industry and embrace intelligent automation.

Image: Courtesy NASA Kennedy