Project: Depth from Defocus
Results
The final report and output data are available.
20080417
Assuming the lens focal length is available and a subject focus distance is known, the lens distance can solved for using the lens law. The lens I am using has a focal length of 70mm. For a focus distance of 1.27m, the lens can be estimated to be 74.3mm in front of the image plane. For a focus distance of 1.95m, the lens is 72.7mm in front of the image plane.
I have begun implementing the solver in Matlab, however, there are no results to show from this yet. The primary cause of this is the difficulty in integrating the mathematics into usable Matlab code. At this point I have most of the geometric estimations done and am beginning work on the Hankel transform.
20080403
I have obtained images of varying focus with most of the needed information to complete the depth from defocus calculations. I have used these images to estimate the depth from defocus to test the accuracy of my approach before attempting the final implementation. This is done by taking two images of different focus with known subject distance. The distance of objects can then be estimated by finding the approximate blur radius of objects out of focus.
The formula used is[5](8):
D0 = {F * Di} / {2Rf + Di - F}
Where D0 is the depth from the image plane, F is the focal length, f is the F number, Di is the distance from the lens to the image plane, and R is the estimated blur radius.
Mouse over to see far/close focus
The above images were taken with a Canon 300D with a Canon 70-200mm EF lens. The focal lenth was 70mm, the F stop was 4. The 300D CCD is 22.7x15.1mm large with an effective resolution of 3072x2048. Thus, for a 300D, there are 2048/15.1 or 135.6 pixels/mm on the image plane. The distance farthest focus plane is 1.94m from the image plane, the closest is 1.27m (obtained from the EXIF data).
Mouse over to see actual/estimate blur of closest box
The blur amount on the closest box can be simulated with a gaussian blur of 70px radius. Assuming that the focal length and distance from the lens to the image plane are nearly the same, the depth can be estimated by the above formula to be 1.18m.
While this is fairly close to the actual distance of 1.27m, there are issues that must be resolved. As the blurred radius approaches 0, the distance does not approach the know distance of the focused plane. I believe this is due to not having the actual measurement of lens to image plane distance.
I will be looking at this issue over the next few weeks, as well as implementing the actual solver. A more accurate measurement of lens to image plane could be obtained geometrically from the known distances, or by solving using the know focus distance. I will also be looking to see if it can be obtained by other EXIF values in the image files.
While most think of stereo vision as the primary means to reconstruct depth from a set of images, there are other methods that can solve the same problem. One of these is using the relative focus and defocus of an image to estimate the depth of the scene. The goal of depth from defocus is to estimate the values in
I(x) = \int{I_0(e) g(x-e) de
where I is the given image intensities, I_0 is the actual scene intensities, and g is a blurring function. That is, the original scene texture/radiance values are blurred by some amount based on known camera properties. For my project, I plan to implement a depth from defocus algorithm similar to the one described in [1].
All code related to the actual depth analysis part will be my own. I will like use external math libraries to aid in the solving of the various equations. I plan on obtaining the sample images via my own cameras, although this is depending on what focus information is available in the EXIF. The actual values in EXIF depend on camera and lens used. If I cannot obtain my own data, I will seek data from other researchers.
ReservationsMy primary reservations are centered on obtaining datasets. Since most cameras do not provide a discrete method of adjusting the focus, it may be difficult to capture images with known, repeatable focus values. All other reservations are based on implementation details, such as integrating a Fourier library or accurately solving the PDE associated with the methods.
References
[1] Y. Xiong and S. Shafer, Depth from focusing and defocusing, In Proc. of the Intl. Conf. of Comp. Vision and Pat. Recogn., pages 68-73, 1993
[2] P. Favaro and S. Soatto, Learning depth from defocus, Proc. IEEE European Conference on Computer Vision, 2002
[3] A. Pentland, A new sense for depth of field, IEEE Trans. Pattern Anal. Mach. Intell., 9:523-531, 1987
[4] H. Jin and P. Favaro, A variational approach to shape from defocus, Proc. IEEE European Conference on Computer Vision, 2002
[5] J. Ens and P. Lawrence, An investigation of methods for determining depth from focus, IEEE Trans. Pattern Anal. Mach. Intell., 15:97-108, 1993