Exact Maximum Entropy Inverse Optimal Control for Modelling Human Attention Switching and Control
Felix Schmitt, Hans-Joachim Bieg, Dietrich Manstetten, Michael Herman, Rainer Stiefelhagen
IEEE Conference on Systems, Man and Cybernetics 2016, Budapest, Hungary, October 9 – 12, 2016
Abstract:
Maximum Causal Entropy (MCE) Inverse Optimal Control (IOC) has become an effective tool for modelling human behaviour in many control tasks. Its advantage over classic techniques for estimating human policies is the transferability of the inferred objectives: Behaviour can be predicted in variations of the control task by policy computation using a relaxed optimality criterion. However, exact policy inference is often computationally intractable in control problems with imperfect state observation. In this work, we present a model class that allows modelling human control of two tasks of which only one be perfectly observed at a time requiring attention switching. We show how efficient and exact objective and policy inference via MCE can be conducted for these control problems. Both MCE-IOC and Maximum Causal Likelihood (MCL)-IOC, a variant of the original MCE approach, as well as Direct Policy Estimation (DPE) are evaluated using simulated and real behavioural data. Prediction error and generalization over changes in the control process are both considered in the evaluation. The results show a clear advantage of both IOC methods over DPE, especially in the transfer over variation of the control process. MCE and MCL performed similar when training on a large set of simulated data, but differed significantly on small sets and real data.
@INPROCEEDINGS{Schmitt2016SMC,
author={Felix Schmitt and Hans-Joachim Bieg and Dietrich Manstetten and Michael Herman and Rainer Stiefelhagen},
booktitle={2016 IEEE Conference on Systems, Man and Cybernetics (SMC)},
title={Exact Maximum Entropy Inverse Optimal Control for Modelling Human Attention Switching and Control},
year={2016},
month={October},
}