Generalized Assorted Pixel Camera

Fumihito Yasuma is with the Sony Corporation, Japan. He was a visiting researcher at Columbia Uni- versity and supported by Sony Corporation. This work was conducted at the Computer Vision Laboratory in the Department of Computer Science at Columbia University. Abstract We propose the concept of a generalized assorted pixel (GAP) camera, which enables the user to capture a single

image of a scene and, after the fact, control the trade-off between spatial resolution, dynamic range and spectral detail. The GAP camera uses a complex array (or mosaic) of color filters. A major problem with us- ing such an array is that the captured image is severely under-sampled for at least some of the filter types. This leads to reconstructed images with strong aliasing. We make four contributions in this paper: (a) We present a comprehensive optimization method to arrive at the spatial and spectral layout of the color filter array of a GAP camera, (b) We develop a novel anti-aliasing algorithm for reconstructing the under- sampled channels of the image with minimal aliasing, (c) We demonstrate how the user can capture a single image and then control the trade-off of spatial resolution to generate a variety of images, including monochrome, high dynamic range (HDR) monochrome, RGB, HDR RGB, and multispectral images and (d) Finally, the per- formance of our GAP camera has been verified using extensive simulations that use multispectral images of real world scenes. A large database of these multispectral images is being made available at http://www1.cs.columbia.edu/CAVE/ projects/gap_camera/ for use by the research community. Index Terms - Color filter array, assorted pixels, sub-micron pixels, demosaicing, color reproduction, dynamic range, signal to noise ratio, multispectral imaging, skin detection. 1 1 Introduction Most color image sensors use a color mosaic which is an assortment of different spectral filters. A color mosaic usually consists of three primary colors (e.g., RGB). One reason for the use of tri-chromatic filter arrays is that tri-chromatic sensing is near-sufficient in terms of colorimetric color reproducibility. It is also commonly assumed that this pixel as- sortment is the only practical way to sense color information with a single semiconductor image sensor 1 . Recently, new image sensing technologies have emerged that use novel pixel assort- ments to enhance image sensing capabilities. For high dynamic range (HDR) imaging, a mosaic of neutral density filters with different transmittances have been used [9] [8]. A new approach to high sensitivity imaging builds upon the standard Bayer mosaic by using panchromatic pixels [6] that collect a significantly larger proportion of incident radiation. Color filter arrays (CFAs) with more than three colors have been proposed to capture multispectral images [14] [2]. In this paper, we introduce the notion of a generalized assorted pixel (GAP) camera, that uses a mosaic with a...

Website: www1.cs.columbia.edu | Filesize: -
No of Page(s): 31
Download Generalized Assorted Pixel Camera: - Department of Computer ....pdf