|
From
•>>February 2005
Peyman Milanfar answers
a few questions about this month's emerging research front
in
field of Engineering: Engineering
Article: A computationally efficient superresolution image reconstruction algorithm
Authors: Nguyen, N;Milanfar,
P;Golub, G
Journal: IEEE TRANS IMAGE PROCESSING, 10: (4) 573-583, APR 2001
Addresses:
KLA Tencor Corp, Milpitas, CA 95035 USA.
KLA Tencor Corp, Milpitas, CA 95035 USA.
Univ Calif Santa Cruz, Dept Elect Engn, Santa Cruz, CA 95064 USA.
Stanford Univ, Sci Comp & Computat Math Program, Stanford, CA 94305 USA.
|
|
Why do you think your
paper is highly cited?
|
“The paper describes a method for producing a high-resolution image from a collection of low-resolution images of a scene, having been captured while the camera and/or scene undergo small motions.”
|
|
Our paper described a method of constructing a high-resolution
image from a collection of low-resolution, moving images of the same
scene (called super-resolution). Work in this area has been ongoing
for at least 20 years in the engineering community. But only in the
last few years, in part due to the timing and content of our
publication, has the relevance of this body of work begun to be
widely recognized. This recognition has also come with the rapid
proliferation of digital imaging, and the realization that some of
the hardware complexities of a digital camera can be reduced, and
compensated for, with back-end software processing. Our work
addressed the relevant computational issues in solving this problem,
which had hitherto not been fully addressed. As such, it triggered a
strong renewed interest in this area, as it demonstrated the
practicality of a novel solution, particularly in light of more
significant computing power available inside digital cameras. Since
then, several special issues of technical journals have concentrated
on this topic, and related conferences on the subject have been held
or are planned on this topic over the course of the next few years.
Does it describe a new discovery or new methodology that's
useful to others?
We described the image super-resolution problem in the framework
of computational linear algebra, and presented a solution based on
advanced methods in numerical analysis. Our work built upon the
pioneering earlier works of Tsai, Irani, Elad, Golub, and others. It
has spurred others to consider this problem anew in a vast array of
applications from medical imaging to video surveillance, and also to
analytical chemistry.
Could you summarize the significance of your paper in layman's
terms?
The paper describes a method for producing a high-resolution
image from a collection of low-resolution images of a scene, having
been captured while the camera and/or scene undergo small motions.
For instance, imagine trying to hold a video camera still in your
hand while taking a short clip of images of a scene of interest; or
think of a microscope that has not been vibration-isolated,
capturing a sequence of images of a sample, which itself may not be
perfectly stationary. The small motions that often inevitably occur
enable the reconstruction of a high-resolution composite image from
the captured frames. This high- resolution image reconstruction
process has come to be known as "computational
super-resolution."
Super-resolution is possible due to two fundamental ingredients.
The first is called "aliasing." In many scenarios, it is
the case that the number of picture elements (pixels) in an image is
in general not sufficient to fully represent all the fine details
present in the scene being imaged. Adding imperfect optics to the
mix results in a mixing of spatial frequencies called aliasing,
which is related to the more commonly known "Moiré"
patterns. A common example of this is seen when the news anchor on
television wears a pinstripe shirt. With insufficient resolution,
the stripes appear to flicker, change shape, and even slightly
change direction, creating a disturbing visual experience. This so
called aliasing effect can happen in time as well, when the number
of frames captured per unit of time is not sufficiently high to
resolve a fast-moving phenomenon. In a context familiar to most,
wheels on a cart appear to slow-down, stop, or even move backwards
in some movie clips.
The second fundamental ingredient alluded to earlier that makes
super-resolution possible is motion. The presence of motion between
the "aliased" frames allows the mixed-frequency components
to be separated, and restored to their original form in a
high-resolution composite image. Somewhat counter-intuitively, one
might say that if your video camera is poor, to construct a better
image, you could shake it a little while taking the pictures! I
invite the interested reader to see some examples of
super-resolution, using our latest methods, on the Web.
How did you become involved in this research?
I began working on this problem more than seven years ago, when I
read some of the early literature, and introduced it to my colleague
Gene Golub at Stanford, where I was a consulting Professor. Nhat
Nguyen began working with us, and made this the topic of his Ph.D.
thesis, which he completed in the year 2000. I have since continued
to work in this area at the University of California at Santa Cruz,
where along with several remarkable students (Sina Farsiu and Dirk
Robinson), and our collaborator Michael Elad at the Technion, we
have developed newer algorithms for super-resolution which have
surpassed the quality and computational efficiency of the method
described in the 2001 paper. I invite the interested reader to visit
our website where additional information about our recent
theoretical progress, the software implementations, and examples of
performance, can be found on the Web.
Peyman Milanfar
Associate Professor of Electrical Engineering
University of California at Santa Cruz
Santa Cruz, CA, USA
|
Return to Emerging Research Fronts | Return
to Special Topics main menu
|
