中文摘要：

自主系统中，agent通过与环境交互来执行分配给他们的任务，采用分层强化学习技术有助于agent在大型、复杂的环境中提高学习效率。提出一种新方法，利用蚂蚁系统优化算法来识别分层边界发现子目标状态，蚂蚁遍历过程中留下信息素，利用信息素的变化率定义了粗糙度，用粗糙度界定子目标；agent使用发现的子目标创建抽象，能够更有效地探索。在出租车环境下验证算法的性能，实验结果表明该方法可以显著提高agent的学习效率。

英文摘要：

Agent interacts with the environment to perform their assigned tasks in autonomous systems.Using hierarchical rein-forcement learning technology helps the agent to improve learning efficiency in the large and complex environment.This paper put forward a new method to find subgoal.It used the rate of change of pheromone which ants leaved in ergodic process to define the roughness,and used the roughness to define the sub-goals.It used the found subgoals to create abstract agent in order to explore more effective.The experimental results show that this method can significantly improve the learning performance.Authentication algorithm in a taxi environmental performance,experimental results show that this method can significantly improve the learning efficiency of agent.