In this paper, we propose an algorithm for view-dependent nonuniform sampling for image-based rendering (IBR). Given a set of virtual views, the positions of the capturing cameras are rearranged in order to obtain the optimal rendering quality. The resulting arrangement of the cameras is effectively non-uniform sampling of the plenoptic function. We formulate the above sampling problem as a recursive weighted vector quantization problem, which can be solved efficiently. Experimental results show that the non-uniform sampling scheme renders much better images than traditional uniform sampling methods.