We investigate the problem of using function approximation in reinforcement learning where the agent’s policy is represented as a classiﬁer mapping states to actions. High classiﬁcation accuracy is usually deemed to correlate with high policyquality. But this is not necessarily the case as increasing classiﬁcation accuracy can actually decrease the policy’s quality. This phenomenon takes place when the learning process begins to focus on classifying less “important” states. In this paper, we introduce a measure of state’s decision-making importance that can be used to improve policy learning. As a result, the focused learning process is shown to converge faster to better policies1.