Frequencies as the Shadow has them

CSE 558 Project Three – Spring, 2006
by
Leith Caldwell & Nathaniel Jacobson

The challenge

The goal of this project was to create characterizations of shadow boundaries through careful capture and analysis. The motivation for creating these characterizations was to determine the effects of direct and indirect illumination on edge profiles for subsurface scattering simulations.

The how

Example capture image of our target.

Example capture image of our target.

In order to determine the range of shadow boundary profile curves, we captured a series of shadow boundary photographs from various environments under different illumination conditions using a specific target. In addition to the camera we used a tripod for stability and an exposure meter for establishing absolute photometric calibration.

The target we created was adhesive paper on glass with a checkerboard pattern using 30mm × 30mm squares. The pictures were taken as close to the target as possible in order to maximize the spatial resolution, with our aim as being at least 10 pixels per millimetre resolution in each image.

Prior to capturing the shadow boundary images on our target, we captured a series of different exposure images of the same scene in order to determine the camera's response curve.

In addition to the images, we recorded information on the time, date and place the images were taken. In order to further specify the lighting conditions, we also recorded approximations of the distances from the target surface to the light source and from the target surface to the occluding object as well as any information on what kinds of light source and occluding objects were in the scene. The measurements taken have an estimated error margin of ±10% with an upper bound of 10mm.

Furthermore, we experimented with including other occluding objects in order to create as high a contrast shadow profile curve as possible in different environments.

The checkerboard geometric calibration pattern on our target was used to unwarp the captured images so that we could analyze the image as an orthographic view of the target surface. We analyzed the images of the target surface and determined the frequency content of the shadow boundary profile. Because the shadow boundary should be approximately uniform across the image, we combined many single line profiles of the shadow boundary to create a single oversampled profile. The oversampled profiles have 4× the original spatial resolution of the camera's sensor array. Effective resolution will still be dependent on the camera's optics.

During our analysis, we also compared the contrast ratios between the illuminated and shadowed regions on the target. The values from the illuminated and shadowed regions were extracted by hand in HDR Shop as an average over selected areas in the image.

One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.One of the capture scenes.

Several of our capture scenes.

The camera we used was a Canon EOS-1D Mark II with an original CCD image resolution of 3104 × 2072 pixels. We used a Canon 100mm focal length macro-lens and had the camera mounted on a standard tripod. The target was generated by printing a 30mm×30mm checkerboard pattern onto an adhesive shipping label (Avery 18126) and attaching the label to a 104.6mm(4") × 152.4mm(6") pane of glass.

The challenges we came across were:

Changes to the original plan

During the development of this project, a few things became clear about our original plan that needed to change.

Photometer
Originally we used the photometer to guide us in our selection of shot exposure time. We discovered after our first round of photography that the ISO of the photometer and that of the camera were different, which meant that the exposures we had been using resulted in several images with saturation issues. We resolved this by matching the ISOs in the next capture session.

Unwarping of images
In order to calibrate the frequency calculations, the images needed to be unwarped to create a 'straight above' view point. This was done using a perspective warp with bilinear interpolation, mapping the four corners of the 3 × 3 checkerboard to the extreme corners of a new 2072 × 2072 image.

Original image.

Original image.

Unwarped.

Unwarped.

The result

After all that, we have... these!

Below is a graphical represntation of the various shadow profile boundaries that we captured, color coded by light source. The curves have been normalized and aligned at their midpoints in intensity. From this graph, we can see the effect of having visible gaps between pixels from the captured projector images resulting in the oscillating pattern in the curve.

Shadow border profiles.

Shadow border profiles.

From the graph below (a side-view of the graph above), the range of the shadow profile boundaries is clear. Since the distance to occluder axis has been removed from this graph it is difficult to see that it is the most correlating property of the gradient of the curve.

Shadow border profiles.

Shadow border profiles.

The graph below shows the contrast ratios of the shadow boundaries that we captured, again color coded by light source. Each light source is clearly clustered to different parts of the graph, but still within the range of 2 to 10. The exception to this is the projector, with a slightly higher ratio of 20. Further captures are expected to fall within this range.

  Sun Projector CFL LED
Highest 8.86 20.00 6.67 9.88
Lowest 2.50 20.00 6.67 3.33
Contrast ratios plotted with ratios of 1 and 10 as lines.

Contrast ratios plotted with ratios of 1 and 10 as lines.

The graph below shows the spatial frequency response of the captured shadow profiles, showing only the frequency range where there is significant magnitude below the Nyquist rate. It can be seen that at least 90% of all of the curves is at spatial frequencies below 3 cycles per mm. For the projector response, three harmonic peaks are visible due to the oscillatory nature of the profile.

Spatial frequency of curves below Nyquist.

Spatial frequency of curves below Nyquist.

The following waterfall plot shows the same spatial frequency information, plotted against distance to occluder, and including spatial frequencies that are derived using superresolution. The most notable feature is a frequency peak at the original Nyquist rate. This may be related to the effects of the bilinear interpolation used in the perspective warp.

Spatial frequency of curves.

Spatial frequency of curves.

There is always more you can do

There are a number of things that could be implemented to extend the scope of the project. These include...

Probably the most notable thing here is that more complete characterizations and trends will require a much larger set of input images in order to come up with anything statistically validated.

Resources

Bouguet Toolkit – toolkit for geometric camera calibration.
sfrmat 2.0 – matlab function for spatial frequency response measurements.

Acknowledgements

Thanks to Michael Goesele for assigning us an opportunity to research something new and extend our domain knowledge.