基于状态空间划分的预测状态表示模型学习算法.doc

文档介绍：
基于状态空间划分的预测状态表示模型学
习算法#
刘云龙*
5
10
15
20
25
30
35
40
(厦门大学自动化系)
摘要:预测状态表示(Predictive State Representations, PSRs)是解决局部可观测问题的有效
方法,但现有研究往往以整个状态空间为问题背景或仅能获取系统的局部模型,限制了
PSR 方法的应用范围。本文提出了一种基于状态空间划分的 PSR 模型的学习算法,首先根
据训练数据,利用 Landmark 将状态空间划分为多个子状态空间,进而确定每个子空间 PSR
模型以及缺值数据的获取方式,最后利用子空间 PSR 模型构建整个系统的 PSR 模型,实现
对任意事件的预测。所提方案降低了获取模型的难度,提高了预测的精度。通过在 Cheese
Maze 问题上的应用,验证了所提算法的准确性和有效性。
关键词:预测状态表示;局部模型;状态空间划分
中图分类号:TP282
An algorithm for Learning Predictive State Representations
Based on State Space Partitioning
Liu Yunlong
(Department of Automation, Xiamen University, 361005, Xiamen)
Abstract: Predictive State Representations (PSRs) are powerful methods of modeling dynamical
systems by representing state using only observable data. Much progress has been made since
PSRs were proposed. However, current techniques in PSRs focus on learning a model on the
entire state space, or learning a local model of the system, which limit the application of PSRs. In
this paper, an algorithm for learning the PSR model of a system based on state space partitioning
is proposed. First, according to the training data, the entire state space is partitioned into sub-state
spaces using landmarks of the environment, and then traditional technique is used to learn every
sub-state space’s PSR model, which is easier than learning the model on the entire state space. We
then show