The 3D Object Reconstruction from 2D Slices - Image Preprocessing.
Michal Fano, Martin Polák
Faculty of Mathematics and Physics
Main topic discussed in this paper is the practical application of image processing systems, particularly those resolving the problem of image restoration from 2D slices, which are acquired via CCD cameras connected to various types of optical devices and subsequently processed by computer.
Keywords: convolution, objective, numerical aperture, in-focus plane, out-of-focus plane, inverse filtering, PSF.
The human being is not able to handle all of the received information. The information obtained at the very first stages of human observation of the real world is always accompanied by, according to previous sentence, 'surplus' data. The area of image preprocessing serves as instrument for suppressing such undesirable features of image, which is sometimes equal to emphasizing the desired ones either by empirical or theoretical methods. Thereinafter we focus on describing some image preprocessing techniques.
We begin with listing of several image-depreciating effects:
· glare and shading of objects
· 3D to 2D space projection
· limited resolution of devices
· additive and multiplicative noise
· radiation induced by examined object
· motion of object
Very often an image contains much more information than can be perceived. Details are invisible due to noise or other obstacles. Some of image degradations are introduced by human factor. This includes improperly focused optical devices, construction defects etc. Our intention is to describe some of them and to try to eliminate their influence on the image. This can be achieved by using of explicit information about optical devices.
2. Image reconstruction techniques
"Renovation" is an image preprocessing technique, which tries to reduce the image defects by procedures based upon estimation of defect characteristics. While it is obvious that reconstructed image is merely an approximation of the original, the goal is to minimize the differences. Considering the viewpoint from which solution arises, we can split image reconstruction techniques into deterministic and stochastic. Deterministic techniques are only suitable for image that contains none or irrelevant amount of noise. The reconstructed image is a result of applying the inverse transformation to the acquired image (i.e. the image obtained by a physical device). On the other hand stochastic methods are based upon starting estimation of original image and successive iteration which must satisfy convergence condition. However, in both cases it is essential to have the degrading transformation properly modeled.
Considering the type of availability of information, we recognize a priori and a posteriori information. A priori knowledge can be obtained e.g. during scanning of moving objects. In such cases ablation of defects (motion blur) can be accomplished using information about such physical parameters as direction and speed of movement. Defects detected during image analysis are considered a posteriori.
2.1. Image reconstruction where the degrading transformation is known
In order to solve the problem of reducing the image defects we firstly are obligated to formally describe the process of degradation. This can be done by following formula
g(x,y) = s+ v(x,y), (1)
where g(x,y) represents resulting image, f(x,y) the original image, s - nonlinear function, v(x,y) the noise. This model is often simplified, neglecting nonlinearity of s(x,y) and constraining h(x,y,a,b) to the time-invariant convolutional core, which is used to describe the defects of the optical device. Simplified formula
g(x,y) = f(x,y) Ä h(x,y) + v(x,y), (2)
represents the fundament for image restoration algorithms. The design of h(x,y) called Point Spread Function will be explained in details later in the paper.
Objective lens of device is focused on small area of examined specimen. However acquired image contains light from non-focal sources. As a result of this effect, image information coming from in-focus plane is fairly blurred by light diffraction. While PSF was defined for both in-focus and out-of-focus plane we are able to calculate resulting image by summing the convolution of in-focus data with in-focus PSF and the convolution of out-of-focus data with out-of-focus PSF. Let us consider simple case, when merely two out-of-focus planes contribute to blurring of the image from in-focus plane [Agar89]:
g(x,y)f(x,y)h(x,y)+f(x,y)h(x,y) +f(x,y)h(x,y), (3)
where functions f(x,y), f(x,y), f(x,y) are actual images. Functions h(x,y), h(x,y), h(x,y) are in-focus and out-of-focus PSF. As the actual image data from individual planes are unknown, we will use their approximation. Hence g(x,y) f(x,y) and g(x,y) f(x,y). Therefore data from in-focus plane can be calculated using modified equation
f(x,y)(g(x,y)-c.(g(x,y)h(x,y)+ g(x,y)h(x,y))) h(x,y), (4)
where constant c is determined empirically, function h(x,y) describes filter for in-focus plane. This process is also called nearest-neighbours algorithm [Agar89].
Figure 1: Scheme of the nearest-neighbours algorithm with two out-of-focus planes, using CTF unction, meanwhile the Fourier transformation is applied to each slice
By modification of this procedure we obtain no-neighbours algorithm [Monck92]
f(x,y) (g(x,y)-c.(g(x,y)h(x,y) +g(x,y)h(x,y))) h(x,y), (5)
assuming that f(x,y) g(x,y) and f(x,y) g(x,y).
2.1.1. Non-iterative methods of inverse filtering
Techniques of inverse filtration are derived from basic model defined by (2). For effective applying of inverse filtration the assumption of minimal noise is necessary.
Further we take advance (in terms of calculation) of fact, that convolution can be calculated as multiplication in spectral domain (i.e. after applying of DFT):
G(u,v) = F(u,v).H(u,v) + V(u,v) (6)
We can obtain the original image by filtering with a function inverse to H(x,y):
F(u,v) = G(u,v).H(u,v) – V(u,v).H(u,v) (7)
In cases when noise is relevant, it appears in equation (7) as an additive error, which has its maximal amplitudes mostly in higher frequencies of the spectrum. By neglecting noise we obtain:
(u,v) = G(u,v).H(u,v) (8)
Afterwards we apply linear operator Q to the F to improve the flexibility of the found approximation. Our task is to find the minimum of J(F) using the method of Lagrange multipliers.
J() = + .(), (9)
where is Lagrange multiplier.
= .G(u,v), (10)
Equation (10) serves as a base for finding suitable linear operator Q and successive determination of g. Well-known Wiener filter is often used. Estimation of statistical properties of image noise is a criterion for the choice of the most appropriate operator.
where k is obtained empirically.
2.1.2. Iterative methods of inverse filtering
Principal idea of iterative procedures used in inverse filtration is to estimate and gradually improve the quality of approximation of the original image. To minimize the distance of related images is one of the possibilities. Let us use following relation between the original and measured image
g(x,y) = f(x,y) Ä h(x,y), (12)
D(x,y) = (13)
D (x,y) is the measure of quality of estimation in the k-th step of iteration. o(x,y) - represents the approximation of f(x,y) in the k-th step of iteration. In terms of previous formula we have to minimize D(x,y). Now we define the k-th step of iteration as
o (x,y) = o(x,y).w(x,y), (14)
where w is vector of correction. Choice of suitable correction vector is necessary. We notice some well-known proposals of the correction vector:
· Jansson & van Cittert
w(x,y) = 1 + ..( g(x,y) - (h(x,y)o(x,y)) ) (15)
w(x,y) = (16)
· Richardson & Lucy
w(x,y) = h(x,y)() (17)
Aforementioned methods can be extended with possibility of noise reduction so that in each step of iteration we apply Gaussian filter to the processed image. Hence modified Meinel's algorithm is:
w(x,y) = L(x,y) (18)
The formulae listed above belong to very repeatedly used in the field of image preprocessing. h(x,y) is called Point Spread Function.
3. Point Spread Function
Degradation and blurring of optically acquired image is described via PSF. PSF is a mathematical description of how the points from an out-of-focus plane are transformed to the measured image. Hence this function is individual for each optical device, depends on object glass used and is influenced by numerical aperture, by errors and aberrance of the used lenses and by index of refraction of used immerse media. PSF is an approximation of theoretical PSF and it is obtained either considering aforementioned physical parameters in theoretical optics model or can be measured empirically.
3.1. Theoretical approach
Image degradation effect can be computed as an approximation of the original function with aid of theoretical optics. There exists numerous methods of simulating PSF; they mostly arise from the basic theoretical model described in works of Castleman [Cast79] and Agard [Agar84]. Theoretical PSF is considerably different than real PSF in the way that it represents ideal model of optical devices and it is not able to fully describe the variations of physical parameters of practically used optical devices. In spite of that we can reach very substantial image improvements. PSF is modeled via CTF, which is its equivalent in frequency domain.
CTF depends on physical parameters of optical device, such as:
· numerical aperture
· refractive index of medium
· x/y-dimension of pixel
· image size
· wavelength of emitted light
S(q)=(2b - sin 2b)jinc[(1 – )], (19)
where q = ,
jinc(x) = 2J1(x) / x ,
and J1(x) is Bessel function. This function is usually included in the math library of a C language compiler.
b = cos-1 () ,
w = - df - Dz cos a + (df2 + 2dfDz + Dz2cos2 a),
where d is focal distance of the objective lens in micrometers and Dz is distance from in-focus plane in micrometers. From (19) follows that planes equally distant from out-of-focus planes are equivalent.
a = sin-1(NA / h),
Parameter a is the widest angle of incident light rays, relative to the optical axis, that can be captured by the objective lens. NA is the stated numerical aperture on the objective lens and h is the index of refraction of the immersion medium recommend by the manufacturer of the objective lens.
fc = 2 NA / l = 2 h sin a / l ,
where fc is the inverse of the resolution limit, which is the highest spatial frequency that can be passed by the objective. Parameter l is the wavelength of emission light.
Figure 2: CTF NA=1.4, df =800 mm, h = 1.515, Xdim = 0.295858 mm, Ydim = 0.295858 mm, distance of defocus 1 mm, l = 0.488 mm
3.2. Empirical approach
In various cases of reconstruction of depreciated display image it is satisfactory to know PSF as an abstract approximation of the original PSF, but in some other cases, as the aberrance and others anomalies of individually used lenses bring about cancellation of image, that we cannot abolish by present approximation, it is necessary to obtain PSF by an empirical manner. We can define PSF as a transformation of a light emitting point, whose magnitude is smaller than resolution of the optical system, so we chose a point light source for examined object. Measuring out these image parameters yields PSF, that itself more approaches to original. If the optical system does not contain any anomalies (so we can discuss ideal optical device), then examined point shows itself as a luminous spot (appearing at display image as a pixel) at in-focus plane, while there are no spots at out-of-focus planes. At factual conditions the light emitting point appears as a circular aggregation of light spots, with fading intensity depending on the radius. Circular aggregation increases in radius with clearance of in-focus plains.
Figure 3: Empirical PSF
4. Future work
Work presented in this paper provides compact tool for processing of image obtained via CCD camera connected to optical devices. Algorithms use either theoretical or empirically measured approximation of point spread function. Merely small number of physical parameters is considered in existing models of point spread function. Considering more parameters there appears the possibility of more accurate modeling of point spread function for individual optical systems. This seems to be vast and very rich field, which offers amount of opportunities for research.
The aim of this work was to collect various methods of image preprocessing, their improving and implementation. These techniques try to give human instruments for better understanding, storage and further processing of acquired images.
We would like to thank Dušan Chorvát for fruitfull discussions, for his support and assistance during experiments. We also appreciate material support from International Laser Center in Bratislava.
[Gonz87] Gonzales R.C., Wintz P.: Digital Image Processing, Addison-Wesley, Mass., 1987
[Ruži95] Ružický E. , Ferko A. : Počítačová grafika a spracovanie obrazu, Sapienta, Bratislava, 1995
[Šonk92] Šonka M. , Hlaváč V. : Počítačové videní, GRADA, Praha, 1992
[Agar84] Agard D. A. : Optical sectioning microscopy: Cellular architecture in tree dimensions, Annu.Rev.Biophys.Bioeng, 1984
[Cast79] Castelman K. R. : Digital Image Processing, Englewood Cliffs, New Jersey: Prentice-Hall, 1979
[Monc92] Monck J. R. , Oberhauser A. F. et al.: Thin-section ratiometric Ca images obtained by optical sectioning of fura-2 loaded mast cells, J.Cell Biol., 1992
[Keat99] Keating J. , Cork R. J. : Improved Spatial Resolution in Ratio Images Using Computational Confocal Techniques, A Practical Guide to the Study of Calcium in Living Cells, Volume 40 of Methods in Cell Biology, Academic Press, 1999