- Turkish Journal of Science and Technology
- Vol: 18 Issue: 1
- A Hierarchical Reinforcement Learning Framework for UAV Path Planning in Tactical Environments
A Hierarchical Reinforcement Learning Framework for UAV Path Planning in Tactical Environments
Authors : Mahmut Nedim ALPDEMİR
Pages : 243-259
Doi:10.55525/tjst.1219845
View : 7 | Download : 4
Publication Date : 2023-03-29
Article Type : Research
Abstract :Tactical UAV path planning under radar threat using reinforcement learning involves particular challenges ranging from modeling related difficulties to sparse feedback problem. Learning goal-directed behavior with sparse feedback from complex environments is a fundamental challenge for reinforcement learning algorithms. In this paper we extend our previous work in this area to provide a solution to the problem setting stated above, using Hierarchical Reinforcement Learning (HRL) in a novel way that involves a meta controller for higher level goal assignment and a controller that determines the lower-level actions of the agent. Our meta controller is based on a regression model trained using a state transition scheme that defines the evolution of goal designation, whereas our lower-level controller is based on a Deep Q Network (DQN) and is trained via reinforcement learning iterations. This two-layer framework ensures that an optimal plan for a complex path, organized as multiple goals, is achieved gradually, through piecewise assignment of sub-goals, and thus as a result of a staged, efficient and rigorous procedure.Keywords : hierarchical reinforcement learning, tactical UAV path planning, machine learning, path optimization