We propose ‘ﬁlter forests’ (FF), an efﬁcient new discriminative approach for predicting continuous variables given a signal and its context. FF can be used for general signal restoration tasks that can be tackled via convolutional ﬁltering, where it attempts to learn the optimal ﬁltering kernels to be applied to each data point. The model can learn both the size of the kernel and its values, conditioned on the observation and its spatial or temporal context. We show that FF compares favorably to both Markov random ﬁeld based and recently proposed regression forest based approaches for labeling problems in terms of efﬁciency and accuracy. In particular, we demonstrate how FF can be used to learn optimal denoising ﬁlters for natural images as well as for other tasks such as depth image reﬁnement, and 1D signal magnitude estimation. Numerous experiments and quantitative comparisons show that FFs achieve accuracy at par or superior to recent state of the art techniques, while being several orders of magnitude faster.