Currently, there is a challenge in deriving explanations of autonomous decision-making behavior. In particular, the challenges inherent in interactions between humans and intelligent machines, where transparency and trust are essential to facilitate successful human-machine teaming. IFT is spearheading the effort, based on deep reinforcement learning, to address the fact that autonomous decision-making agents can affect future states based on their current actions. IFT is also addressing the challenge of reasoning over long-term human-machine collaborative objectives of underlying missions. This effort will lead to a better understanding of learning outcomes and more effective machine learning algorithms. It will be applicable to military scenarios by enabling human-interpretable behavior explanations in human-in-the-loop decision processes. The commercial sector, such as healthcare or financial systems, where stringent interpretability and accountability are required will also benefit from this work.