Methods: Experiment Design, Mixed Model Statistical Analysis
Tools: Heart Rate Monitor, SAS, Matlab

Background

This study’s goal is to evaluate the applicability of existing human performance moderator functions (HPMFs) to human performance in peer-based human-robot teams. HPMFs are equations derived from empirical data to predict human performance due to specific performance factors, such as fatigue, mental workload or temperature. Interaction between humans and robots in peer-based teams can be dramatically affected by human performance. Our research is focused on determining if existing human performance moderator functions for task workload apply to peer-based human-robot interaction and if not, how such functions must be modified.

Method

The evaluation involved the validation of a series of human performance models, created using the Army Research Lab’s IMPRINT Pro Software. The models depict multiple team-based scenarios involving first responders (both robot and human) to disaster events. The performance models provide a prediction of the HPMFs (e.g., workload) according to current HPMFs and this prediction is evaluated with real people and robots in test scenarios.

During the evaluation, two different conditions were studied. First, human-only teams completed the scenarios where one individual in the team will be an evaluator and the other human will be the participant. The participant and the evaluator will partner to complete the task. The evaluator gave the participant instructions on assessing victims’ injuries (triage) based on the information reported by the participant. Second, a human participant was paired with a robot, both of whom were supervised remotely by a human evaluator. The robot and the participant partnered to complete the victim assessments. Performance of the participants was compared between the two conditions and to the performance models’ predictions.

Participant assessing victim injuiry while communicating withi human teammate through walkie-talkie:

Participant assessing victim injuiry under the verbal instruction of robot teammmate:

Experiment video of Human-Human condition:

Experiment video of Human-Robot condition:

Results

Both the models and the empirical evaluation found that the human-robot condition generally results in lower workload than the human-human condition. As well, the empirical results generally mirror the model results for each condition, with the human-robot peer-based team model slightly overestimating the measured workload. This research provides initial support for the applicability of a current workload HPMF (for human-only teams) to human-robot peer-based teams.

Related publications:

Harriott, C. E., Buford, G. L., Zhang, T., & Adams, J. A. (In Press). Measuring human workload in a collaborative human-robot team. Journal of Human-Robot Interaction.

Levin, D.T., Harriott, C., Paul, N., Zhang, T, & Adams, J. A., (2013). Cognitive dissonance as a measure of reactions to human-robot interaction. Journal of Human-Robot Interaction, 2(3), 1-17.

Harriott, C. E., Zhang, T., & Adams, J. A. (2013). Assessing physical workload for human-robot peer-based teams. International Journal of Human-Computer Studies, 71(7-8), 821-837.

Harriott, C. E., Buford, G. L., Zhang, T., & Adams, J. A. (2012). Assessing workload in human-robot peer-based teams. In Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI ‘12). ACM, New York, NY, USA, 141-142.

Harriott, C. E., Buford, G. L., Zhang, T., & Adams, J. A. (2012). Human-human vs. human-robot teamed investigation. In Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI ‘12). ACM, New York, NY, USA, 405-406.

Levin, D. T., Adams, J. A., Harriott, C. E., & Zhang, T. (2011). Cognitive Dissonance as a Measure of Reactions to Human-Robot Interaction. In Proceedings of Robotics: Science and Systems (RSS) 2011 Workshop on Human-Robot Interaction, University of Southern California, Los Angeles, CA.

Harriott, C. E., Zhang, T., & Adams, J. A. (2011). Evaluating the applicability of current models of workload to peer-based human-robot teams. In Proceedings of the Sixth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI ‘11). ACM, New York, NY, USA, 45-52.

Harriott, C. E., Zhang, T., & Adams, J. A. (2011). Predicting and validating workload in human-robot teams. In Proc. of the 20th Annual Conference on Behavior Representation in Modeling Simulation, Sundance, Utah.