AI-Enabled Computed Vision Air-to-Air Refueling Performance Assessment and Automated Pilot Debrief System

Air-to-air refueling (AAR) is a critical Department of the Air Force and Department of the Navy capability enabling global mobility, long-range strike, combat airpower projection, and sustained aviation operations. However, aerial refueling remains one of the most technically demanding and safety-critical aviation tasks performed by aircrews. Successful execution requires precise aircraft positioning, coordinated control inputs, accurate closure rate management, continuous situational awareness, and effective crew coordination between tanker and receiver aircraft.Current aerial refueling training and post-flight assessment processes rely heavily on instructor observation, pilot self-assessment, manual video review, and subjective evaluation of performance. These approaches provide limited ability to quantitatively assess the thousands of individual performance variables that influence successful refueling operations. Existing methods do not provide pilots with objective, moment-by-moment feedback on aircraft positioning, closure rates, control inputs, deviations from desired procedures, or trends in individual performance over time.This topic seeks to develop, demonstrate, and transition an artificial intelligence (AI)-enabled computed vision capability that (post-flight) automatically analyzes aerial refueling operations and provides objective, data-driven pilot performance feedback. The desired capability will mature existing advances in autonomous aerial refueling analytics and apply them toward training, readiness, and safety improvement by transforming aerial refueling video and available aircraft performance data into actionable post-flight debrief products.The developed system shall provide an automated “hot wash” which analyzes aerial refueling events, identifies performance deviations, and generates intuitive visualizations that allow pilots and instructors to better understand execution quality and areas for improvement. The system shall reduce reliance on subjective recollection and manual review by providing objective measurements of aircraft performance throughout the refueling profile.The capability shall focus on analysis of tanker-mounted video data, such as KC-46 aerial refueling camera systems, while maintaining an architecture capable of expanding across additional tanker and receiver platforms. The vendor may pick (with Government concurrence) the first 3-5 receiver platforms to use in developing this system. Future applicability includes KC-135, C-17, all fighter receiver aircraft, naval aviation communities, autonomous aerial refueling systems, and other aviation training environments requiring advanced performance analytics.The desired solution shall leverage artificial intelligence, machine learning, computer vision, and advanced human-machine interfaces to provide capabilities including:• Automated detection and tracking of tanker (boom and hose) and receiver aircraft during aerial refueling operations.• Measurement of relative aircraft position, movement, velocity, and alignment throughout the refueling engagement.• Assessment of aircraft position within the approved refueling envelope.• Analysis of closure rates, approach vectors, aircraft stability, and deviations from nominal procedures.• Ability to correlate video with other aircraft data if/as available (e.g. telemetry, fuel transfer rate, fuel state, and pilot inputs) to evaluate performance drivers.• Generation of objective performance scores and trend analysis across multiple training events.• Creation of visual performance assessment products for receiver pilot post-flight review of their performance during refueling, including graphical overlays, heat maps, event timelines, and immersive or interactive debrief environments.• Support for instructor review, adjudication, and refinement of automated assessments during initial deployment.The system should evaluate innovative approaches for integrating automated analytics with existing or future aviation training environments, including virtual reality (VR), augmented reality (AR), mission rehearsal systems, simulator environments, or other computer-based training tools.The solution must address operational challenges associated with military aviation data, including secure processing of controlled aviation video, automated data protection, and protection of sensitive operational information. The capability should consider approaches for minimizing processing burden on operational units while enabling rapid generation of post-flight assessments.The Government seeks solutions which provide an operationally relevant prototype supporting aerial refueling training organizations. Successful completion of this effort will provide Air Force and joint aviation communities with a next-generation training and assessment capability that improves pilot proficiency, increases the training value of each refueling event, reduces equipment damage and maintenance impacts, and enhances overall mission readiness.The desired end state is an operationally demonstrated capability ready to implement for at least one type of receiver aircraft and that can support transition into existing aviation training ecosystems and provide a scalable foundation for future AI-enabled aviation performance assessment across the Department of Defense.