The rendering performance of an image-based rendering (IBR) system is determined by the number of images and the amount of geometrical information used. In this paper, we propose a layered lumigraph representation that is configured for optimized rendering performance based on the rendering platform (e.g., processor speed, memory) and output image resolution. The layered lumigraph is produced by classifying all pixels into a number of depth layers. Based on prior work on plenoptic sampling analysis, the layered lumigraph is constructed to achieve the same rendering quality along the minimum sampling curve by balancing the number of images and depth layers. For a given rendering platform, the best rendering performance can be obtained by choosing the optimal number of images and depth layers. Moreover, the layered lumigraph is capable of level-of-detail (LOD) control using the same image geometry trade-off. Therefore, the layered lumigraph fully exploits the inherent constraints between the number of images, depth complexity, and output resolution. Finally, a backward warping technique is designed to efficiently render the layered lumigraph by taking advantage of texture mapping hardware.