Should I Add Recommendations to My Warning System? The RCRAFT Framework Can Answer This and Other Questions About Supporting the Assessment of Automation Designs

Automation is widespread in interactive applications, promising multiple benefits to users, including enhancing comfort, safety, security and entertainment. Implicitly or explicitly, automation is now a critical design option for interactive application designers. Unfortunately, despite its long use (especially in safety-critical systems) assessing the benefits and the drawbacks of design alternatives including automation remains a craft activity, unsupported by conceptual frameworks or tools. In order to address this problem, we present the RCRAFT framework. The framework considers five attributes of automation: Resources, Control Transitions, Responsibility, Authority, and System Functions and User Tasks. We show how these attributes support the assessment of designs involving automation. Furthermore, adding the RCRAFT concepts to task models makes it possible to evaluate automation properties such as transparency, congruence and controllability in addition to usability. We demonstrate the utility of our approach in a case study, comparing the design for an existing Flight Warning System currently deployed in Airbus A350 against a redesigned version which incorporates functionality to provide recommendations to pilots handling unusual situations. We demonstrate that the RCRAFT framework helps in highlighting the implications of different design alternatives by making the impact of proposed changes on both users' work and the required properties of automated components explicit.