Introduction
Microscopic investigation of human organs and tissue was once restricted to the two-dimensional (2D) level. However, to describe an entire sample using 2D image data, many individual images should be collected and examined. This type of investigation is analogous to the Chinese idiom “The Blind Men and the Elephant.” In this famous idiom, the blind man touching the elephant’s leg claims that the elephant is a tree. Meanwhile, another blind man touching the elephant’s belly claims that the elephant is a wall. This idiom hints at the significance of examining the multiple aspects and regions of samples when performing full-scale studies in the life sciences.
In the field of tissue engineering, biomimetic and bioengineered scaffolds are used for stem cell differentiation and organ/tissue regeneration. However, materials scientists have been unable to manufacture an ideal scaffold material because many histological aspects of the human body remain unknown. Three-dimensional (3D) reconstruction technologies for visualizing human organs and tissue from a microcosmic perspective, such as micro-computed tomography (microCT), microscopic magnetic resonance imaging (microMRI), and confocal laser scanning microscopy (CLSM), are revolutionizing histological evaluation in the life sciences.
Recently, micro-morphological image reconstruction and evaluation technologies have been applied to regenerative medicine. Many types of 3D imaging and visualization techniques are providing new insights by displaying high-resolution images of tissue microstructures [ 1]. However, apart from CLSM and synchrotron-based microCT, few 3D techniques can attain the sub-micron resolution of 2D light microscopic images [ 2]. Histological sectioning is a valuable method that can be used to produce accurate high-resolution representations of almost any type of tissue [ 3, 4]. However, histological sectioning is time-consuming and difficult to stitch into panoramic images. The process also runs the risk of sample loss, impairment, uneven dyeing, and distortion during histological processing. 3D calculations from 3D reconstructed images can provide the precise distribution of a specific tissue component. Traditionally, this information is obtained from enzyme-linked immunosorbent assay, western blot, and other techniques. Which method is the best is difficult to determine, but 3D quantitative calculation technology provides a promising new solution.
Although magnetic resonance imaging (MRI) and computed tomography (CT) have been significantly improved and are becoming more convenient for 3D reconstruction, they do not yield satisfactory resolution for life science research. We only discuss XY resolution in this paper. Some emerging 3D imaging and visualization techniques are summarized in Table 1.
Tab.1 Descriptions and comparisons of several 3D imaging and visualization techniques |
| XY resolution | Application | Advantages | Drawbacks | References | |
|---|---|---|---|---|---|
| X-ray microtomography (microCT) | 300–700 nm | In vivo analysis and 3D imaging (embryos, bone, tumor, stomatology, granular, and porous materials) | Non-destructive; different perspective imaging | No ideal methods for specific staining | [2,5,6] |
| Microscopic magnetic resonance imaging (microMRI) | Approximately 10 μm | 1. In vivo monitoring and 3D imaging | Non-destructive; | Limitations of physical resolution | [2,7] |
| 2. Untreated or living biological specimens can also be used | higher soft tissue contrast | ||||
| Orthogonal-plane fluorescence optical sectioning (OPFOS) microscopy | 20 μm | 1. Ideal for the analysis of the intact mammalian cochlea | 1. Different series of 2D images from the same specimen can be acquired | 1. Inevitable damage to tissue microstructures | [4,8,9] |
| 2. Limited resolution | |||||
| 2. Quantitative measurements can be obtained | 2. Less time-consuming | 3. Not suitable for in vivo studies | |||
| Optical projection tomography (OPT) | A resolution of microns to tens of μm | 1. 3D visualization of soft tissue, cells, protein distribution, and gene expression patterns in biomedical specimens | 1. Suitable for in vivo studies | Difficult to control the distribution and provide sufficient fluorescent or colored stains | [10–12] |
| 2. Suitable for 1–10 mm-thick specimens | 2. Fills the imaging gap between MRI and confocal microscopy | ||||
| Confocal laser scanning microscopy (CLSM) | 0.5–1 μm | Cell behavior, nerve endings, distribution of biomolecules, analysis of 3D pore structures, and other biomechanical parameters in porous nano/microfibrous biomaterials | Suitable for non-invasive in vivo detection and quantification | 1. Maximum specimen thickness is approximately 100 µm | [2,13–15] |
| 2. Distribution and sufficient fluorescent markers | |||||
| 3D reconstruction of serial physical sections | Sub-micron resolution | Almost all tissue ranging from bones to soft tissue | 1. High resolution and large specimen size | 1. Comparatively laborious | [16] |
| 2. Dyeing of the target components is easy to control | 2. Not suitable for in vivo studies |
MicroCT is a 3D imaging technique based on the absorption dependency of X-rays, which has a resolution of 300–700 nm. This technique is widely used for in vivo analysis or 3D imaging of embryos, bone, tumors, stomatology, and granular and porous materials. Many contrast agents can be used for better imaging, including gold nanoparticles, liposomal iodine nanoparticles, inorganic iodine, and phosphotungstic acid. The advantages are non-destructive, non-invasive, and diverse perspective imaging. One of its main drawbacks is that no ideal methods are available for specific staining of tissue components or gene products.
MicroMRI has a resolution of approximately 10 μm. It is suitable for in vivo monitoring and non-destructive 3D imaging, such as the analysis of basilar artery, cranial nerves, bone, cartilage, ligament, blood vessels, tumors, gene expression, and topographic analysis of the shape of eyes. Untreated or living biological specimens can also be used for 3D reconstruction because microMRI has high soft tissue contrast. However, the main drawback of this technique is its physical resolution limit of about 10 μm.
Orthogonal-plane fluorescence optical sectioning (OPFOS) microscopy is a whole-specimen imaging technique that is ideal for tissue 3D reconstructions of complex features, such as intact mammalian cochlea. Quantitative measurements can be achieved, and the technique is usually less time-consuming. OPFOS microscopy has a resolution of 20 μm. Different series of 2D images from the same specimen can be acquired by OPFOS microscopy, making it superior to light microscopy (LM) in 3D reconstruction. However, OPFOS is not suitable for in vivo studies. For 3D reconstruction of the intact cochlea, soft tissue around the inner ear (neural tissue) must be removed, which can cause serious damage to the microstructures.
In optical projection tomography (OPT), images are obtained through the absorption or emission of visible light. OPT has a resolution of microns to tens of microns, and is good for 3D visualization of soft tissue, cells, protein distribution, and gene expression patterns in biomedical specimens. OPT has been used to analyze β-cell mass distribution in pancreatic islets, vertebrate embryo development, and HEV growth in immunoreactions, and map the distributions of molecular agents in whole mice hearts. OPT fills the imaging gap between MRI and CLSM. However, controlling the distribution and sufficiency of optical fluorescent or colored stains is difficult in OPT.
In CLSM, 3D images are obtained by generating optical sections through an object. CLSM can reach a resolution of 0.5–1 μm [ 17], and currently supplies a substantial portion of novel morphological findings at histological scales. CLSM can be used to analyze cell behavior, nerve endings, distributions of biomolecules, and 3D pore structure or other biomechanical parameters in porous nano/microfibrous biomaterials. This technique is generally suitable for non-invasive in vivo detection and quantification. Nevertheless, the maximum specimen thickness for CLSM is, in practice, not greater than 50 μm, which is usually insufficient for 3D analysis of many biological samples. Moreover, specific fluorescent markers for some components can be difficult to find, and equal distribution of the markers throughout the samples is likewise difficult.
Some other techniques are not included in Table 1, but also allow for serial sectioning imaging, such as transmission electron microscopy (TEM), serial block-face scanning electron microscopy (SBFSEM), autoradiography, imaging mass spectrometry (IMS), and two photon-excited fluorescence laser scanning microscopy (2PLSM).
In TEM, a beam of electrons is transmitted through an ultra-thin specimen (50–100 nm), interacting with the specimen as it passes through. An image with a significantly higher resolution (0.1–0.2 nm) than that of LM is formed from the interaction of the electrons transmitted through the specimen. TEM has been applied to both physical and biological sciences, such as nanotechnology, semiconductor research, cancer research, virology, and materials science. Denk et al. [ 18] reported using SBFSEM to reconstruct 3D tissue nanostructures. The data sets were obtained by automated block-face imaging combined with serial sectioning inside the chamber of a scanning electron microscope. This technique has sufficient resolution (6.7 nm achieved by Denk et al.) to identify synapses and trace even the thinnest axons, which makes it a good method for nervous system research. However, sample preparation is complex in both TEM and SBFSEM. Sections are required to be at hundreds of nanometers thick. The microstructures may be changed during the preparation process. Moreover, the field of view is relatively small, raising the possibility that the region analyzed may not be characteristic of the entire sample.
Autoradiography has been used to determine the tissue (or cell) localization of a radioactive substance, which is either bound to a receptor or enzyme, introduced into a metabolic pathway, or hybridized to a nucleic acid. Thus, autoradiography is a good combination of morphological and metabolic research, and excels at marking the target components. This technique has not been widely applied to 3D reconstruction.
IMS is a technique used in mass spectrometry to visualize the spatial distribution of metabolites, biomarkers, compounds, peptides, or proteins by their molecular masses. Andersson et al. [ 19] created 3D volume reconstructions of matrix-assisted laser desorption-ionization IMS data from rat brain. Therefore, scientists can acquire not only large proteomic and genomic data sets generated from tissue homogenates, but also information on the spatial localization of their encoded products.
2PLSM [ 20] uses nonlinear light-matter interactions to generate signal contrast, and is well suited for deep tissue imaging [ 21] when combined with in vivo fluorescence labeling techniques. Many types of specimens at depths of up to 1 mm [ 22] can be analyzed in detail without damaging the tissue microstructures.
The manual reconstruction of histological sections dates back to the late nineteenth century. The first step of this procedure is the preparation of the specimen, which includes dehydration or decalcification, trimming, embedding, serial sectioning (slicing or grinding), and section staining to make the target components visible. Depending on the registration method, one may choose to add artificial fiducial points for registration.
The second step is image registration (or alignment), in which two adjacent slices are matched one-to-one by rotating, translating, and stretching to produce an accurate 3D model. By adding additional stains, changing the optical magnification, or reducing slice thickness, the spatial resolution of the model can be improved significantly. Nevertheless, the limits of time and labor for this process should be evaluated.
The last step is image processing (image segmentation and 3D reconstruction), which produces the final 3D model of the target organ or tissue.
3D reconstruction technology for serial sections has many applications in the life sciences, including the Visible Human Project, histological morphology observations, and diagnosis of diseases. Take the story of “The Blind Men and the Elephant” as an example. Apart from the elephant’s height, weight, temperature, and appearance, we can now obtain information on each organ, each tissue component, some molecules, and even the expression of certain genes. We are no longer blind.
Overall workflow for 3D reconstruction
Preparation of target tissue or organs
The first step in most procedures is sample preparation. The preparation of the specimen is a crucial step that has a direct influence on the quality of 3D reconstruction. In this section, we provide some important suggestions.
First, one must not damage the structure of the specimen. Knives and scissors should be sufficiently sharp and held vertically when trimming the specimen. Using a blunt instrument or cutting the tissue by dragging should be avoided. Second, PBS or another cell isotonic solution should be used to rinse the specimen. Using distilled water, tap water, or any other liquid detergent is not recommended. Third, fresh specimens must be either fixed or prepared for frozen sectioning as soon as possible. Specimens should not be exposed to the air for too long or allowed to dehydrate.
Processing methods slightly differ for specimens containing bony structures, such as the inner ear or cochlea. The most common processing method for bony structures involves cutting the specimen into thin slices of proper thickness using a microtome after decalcification. The other method for processing bony structures is the microgrinding technique. Thin layers of equidistant slice thickness are removed using a special grinding tool developed by Rau et al. [ 4]. After each microgrinding step, serial cross-section images are obtained for digital photo documentation. Cavity organs, such as the heart and lungs, should be perfusion fixed before removed from the body.
Artificial fiducial markers for registration (used when necessary)
Tissue blocks are cut into many slices that must be aligned for 3D registration and reconstruction. Artificial registration (fiducial) markers [ 23] can aid in one-to-one mapping for accurate registration. Fiducial markers should be cylindrical, easy to cut along with the slice, non-deformable, and contrasting with the surrounding tissue. Various materials, such as hairs, nerves, medical suture lines, and cactus thorns embedded in drill holes perpendicular to the reference plane, as well as “V” type grooves made on the surface of the specimen, have been used as registration markers. In addition, laser drilling and tinged cylindrical pins composed of plastic, epoxy resin, or protein can help ensure accurate image registration. Rau [ 4] achieved satisfactory registration results using cylindrical plastic pins (LEGO Group) as registration markers in the 3D reconstruction of the inner ear.
However, fiducial markers have fallen out of favor in recent years because of their many shortcomings. In our experiment, V-shaped grooves, mouse sciatic nerves, surgical cotton sutures, and medical absorbable sutures served as vertical registration markers through skin specimens (Fig. 1 A: V-shaped grooves; B: mouse sciatic nerves; C: surgical cotton sutures; D: medical absorbable sutures). Each section was stained by HE. However, the results were unsatisfactory. Fiducial markers can destroy the tissue structures, leading to man-made deformations. Although these materials are easily cut, they are also easily lost during section preparation because of their poor adhesion to the tissue. The shape of the marker also varies dramatically between adjacent slices. Fiducial markers can damage the blades, and create cutting artifacts if they are torn through the specimen.
Fig.1 HE-stained skin sections with different fiducial markers. A V-shaped groove made by cutting part of the epidermis (A) and mouse sciatic nerves (B). A surgical cotton suture (C) and medical absorbable sutures (D) were used as vertical registration markers through skin specimens. We can clearly observe man-made structural deformations (A, B) and missing suture fibers (C, D). The shapes of the markers vary dramatically between adjacent slices. |
Satisfactory image registration can be achieved using computer algorithms and physiological tissue structures that naturally exist in specimens, such as blood vessels, nerves, and organ edges.
Sample preparation: fixation, embedding, slicing, and staining
The sample preparation procedures for 3D reconstruction are highly similar to the procedures used for histological research. Distortions and small irregularities are often introduced during the preparation of specimens for LM slides. Although some of these artifacts can be neglected, their influence should be minimized using the MIDAS alignment and adjustment procedure described by Hofman [ 8]. The MIDAS alignment procedure is part of the IMOD software package, which is a set of image processing, modeling, and display programs used for tomographic reconstruction and 3D reconstruction of EM serial sections and optical sections (http://bio3d.colorado.edu/imod/).
Fixation
Proteins will not dissolve in water or other organic solvents after degeneration and solidification by treatment with a fixation fluid. Samples usually harden after fixation.
Many fixation methods exist. Most samples can be directly immersed in fixation fluid. To ensure proper fixation, the volume of the fixation fluid should be more than 40 times the volume of the specimen.
The choice of fixation fluid depends on the type of tissue, target components, and staining methods. One of the most widely used fixation fluids is 4% formaldehyde solution, which spreads quickly through tissue, causing the specimen to shrink dramatically. The contraction ratio should also be considered for high precision computation.
Embedding
After completing the steps for dehydration and transparency, the specimens are still too soft to slice. Embedding can be used to address this problem.
Paraffin wax, epoxy resin, and collodion are widely used as embedding agents. Most samples for optical microscopy can be embedded in paraffin wax, whereas epoxy resin is an appropriate embedding agent for electron microscopy. Hydroxyethyl methacrylate, also known as glycol methacrylate, is the most common water-soluble resin-embedding agent. Collodion embedding is most commonly used for large, hollow, or decalcified tissue, such as embryos, the inner ear, and eyeballs.
Tissue freezing [ 24] is also commonly used for sectioning prior to histochemical or immunohistochemical staining. Correct processing of frozen tissue causes less damage to microstructures, which ultimately improves the accuracy of the 3D model. Tissue samples should be frozen as soon as possible to reduce the formation of ice crystals. The slicing knife in the cryostat should be sharp and cannot be reused.
Slicing
Serial cross-sections are usually cut using a microtome. During sectioning, attention should be given to several points. First, every section should be numbered in the slicing direction from the base to tip. In case of a missing or damaged section, Woodward [ 25] suggested taking a duplicate of the previous section to close the gap. Second, when slicing and stretching the sections, one should avoid shriveling or wrinkling the tissue. Patience is essential, and sectioning should be practiced before attempting to section valuable specimens. Third, the interslice thickness should be consistent. The thicker the slice, the more artifacts can be obtained. An increase in interslice thickness causes more information to be lost because of the lower similarity and consistency between the two adjacent slices. The precision requirements of a specific 3D reconstruction project should be considered when determining whether to correct for the shrinkage of the specimen.
Staining
The staining procedures are designed to distinguish components of interest from the surrounding tissue using easily visualized colors. Choosing an appropriate staining method is important. For example, arrector pili muscle fibers and collagen cannot be distinguished by HE staining. Masson’s trichrome, which stains these structures with different colors, is a more appropriate stain in this case. Homogeneity of section staining should be ensured.
Acquiring the serial section image file
Acquiring serial section image files can transform the data from morphological structures into computer identifiable information [ 26]. Photographic documentation can be obtained using a digital CCD camera [ 27] attached to a microscope with sufficient magnification. The serial images should be obtained using exactly the same protocol in each imaging direction (i.e., equal light intensity, image resolution, zoom factor, and pixel size) [ 28]. In this paper, we are only concerned with light micrographs.
The natural contours of tissue and organs, such as the skin epidermis and liver capsule, are of vital importance for registration and processing. However, imaging an entire tissue section with high resolution is usually difficult. Large high-resolution panoramic images can be generated by stitching several smaller images together in their overlapping areas.
Image stitching is the process of producing a panorama or larger image by combining multiple images while retaining the original resolution. The stitching process consists of four steps, as described by Yang [ 29]; these steps are relative position estimation, features extraction, image registration, and image fusion. Seamless image stitching has long been a strict requirement for producing a panoramic image because visual artifacts may be produced by severe intensity discrepancies and structural misalignments [ 30]. Many methods have been designed to improve panorama quality. These methods typically work by minimizing false edges [ 31], applying feature extraction algorithms, such as the Scale Invariant Feature Transform (SIFT) [ 32], or implementing speeded-up robust features [ 29].
Commercially available software, such as PhotoStitch by Canon [ 33], Paint Shop Pro by Corel, Autostitch by Cloudburst Research Inc. [ 34], and Adobe Photoshop, can stitch consecutive adjacent images into a panorama with no loss of resolution. These photo-stitching techniques have been well integrated with microscope technology. Digital images captured by digital cameras [ 35] and flatbed scanners [ 36] have also been used for stitching. A Full Automatic Digital Slide Scanning System that performs at a magnification of 400–1000 times is commercially available. This system significantly reduces the labor associated with stitching, and substantially improves the accuracy of the stitching results.
Image registration
Image registration is the process of capturing separate images and mapping them one-to-one to achieve alignment of the designated areas for optimal integration into a 3D model. Image registration is a key enabling technology for 3D reconstruction, as well as for other medical image analysis techniques. This technology has benefited from almost 30 years of development [ 37].
Pre-processing before registration
Obvious artifacts or defects, such as dust, creases, or staining precipitates, should be removed from the image background. Image stacks can also be contrast-enhanced by setting the white point to achieve a completely homogenous white background, as described by Handschuh [ 2]. Ideally, the voxels should be exactly cubic.
Image noises usually come from variations in the incident photon flux, random fluctuations in the signal intensity, and interfering signals from the electronic system of the captor device [ 38]. Several methods for noise reduction include neighborhood averaging, median filtering, and Wiener filtering. MATLAB software has been widely used for noise reduction (http://www.mathworks.cn/).
Images acquired by photographic devices usually have unfaithful colors with the original object mainly because of the undersampling of the limited color filters. White balance (WB) is the usual compensation procedure for a correction of colors, in which the color coordinates of an achromatic object become those expected under a reference illuminant. Zaraga et al. [ 39] presented a new method by tunable spectral responsivities for WB.
Image restoration by deconvolution [ 40] is necessary for high quality microscope images that are usually degraded because of noises or the diffraction and random distribution of light. Several deconvolution software packages are commercially available, such as 3D Huygens Deconvolution and Analysis Software from the website of Scientific Volume Imaging (http://www.svi.nl/). An experienced microscopist will be of great assistance in setting appropriate microscope parameters to enhance image deconvolution.
Registration procedure
The registration procedure geometrically aligns two different images (i.e., the reference and sensed images). The procedure for image registration usually consists of four steps: feature detection, feature matching, mapping function design, and image transformation and resampling (Fig. 2) [ 41].
Fig.2 Flow diagram for image registration.Step 1: Feature detection. Manual or automatic detection of salient and distinctive objects, such as edges, contours, line intersections, closed-boundary regions, and corners, etc. Step 2: Feature matching. Establishing the correspondence between the features detected in both the sensed and reference images. Step 3: Transform model estimation. Establishing the type and parameters of the mapping functions. Step 4: Image resampling and transformation. Transforming the sensed image using the mapping functions. |
After the four-step procedure, scientists should evaluate the registration accuracy (localization errors, matching errors, etc.). If the registration accuracy is satisfactory, the registered image can be exported; if not, the previous steps should be corrected.
Lippolis et al. [ 42] achieved automated microscopy image registration from prostatic tissue specimens following these four steps. Prostatic tissue samples obtained from either biopsy or radical prostatectomy were sectioned at a thickness of 4 µm. Two consecutive sections were stained: the first section was used for HE, and the other section was used for immunohistochemistry of p63 and AMACR. Registration was achieved using the following principle: landmarks were detected in both images by SIFT, and the point correspondences were found and aligned (Fig. 3). The registration results were then evaluated visually and quantitatively.
Fig.3 Alignment of adjacent sections from prostatic tissue. Left: HE; Right: p63/AMACR. The arrows link the matching inliers on the two images after rotation and scaling of the right image. In a perfect alignment, the arrows would be parallel. However, this expectation is unrealistic in practice. Cited from Lippolis et al.’s Ref. [42], with permission from BioMed Central. |
Manual alignment methods have traditionally been applied for registration. Randell [ 43] reported the “best visual fit method,” in which the section to be aligned is rendered translucent, rotated, and translated until its contours and structures overlap with the previous section as flawlessly as possible. Paish [ 44] adopted the cross-correlation coefficient for manual alignment using serial processing software. Although manual alignment has been applied for many years, it still has many shortcomings. The main limitation of manual alignment of serial section image stacks is the limited scope and number of images for reconstruction that result from the time-consuming nature of the reconstruction process. Missing or destroyed sections can be eliminated, and the preceding sections can be duplicated.
Rigid registration and non-rigid registration
Historically, image registration can be classified as either “rigid” or “non-rigid” (Table 2) [ 45]. Rigid registration simply rotates and translates the section with respect to each other to obtain correspondence. The distance between any two pixels in the image remains unchanged before and after rigid registration transformation. Non-rigid registration is used if the alignment of the structures cannot be accomplished without localized stretching of the images. Non-rigid registration incorporates affine, projective, and curved transformation. Image registration processes can also be classified as either area-based or feature-based.
Tab.2 Comparison of rigid registration and non-rigid registration |
| Type of transformation | Distance between any two pixels in the image | Methods | Limitations | |
|---|---|---|---|---|
| Rigid registration | Rotation and translation | Unchanged | Head-and-hat algorithm [46] | Not suitable for non-rigid or nonlinear objects, which are common in the human body |
| Non-rigid registration | Affine [47], projective [48], and curved transformation | Changed | Based on spatial transformation (Thin Plate Splines [49], Basis Function); | Comparatively time-consuming, not as robust, and fewer evaluation methods |
| Based on pseudophysical models (Elastic Model [45], Viscous Fluid Model [50], Optical Flow Model) |
Image segmentation
After image registration, segmenting the desired regions from the image series is necessary for the ultimate reconstruction. Image segmentation is designed to extract a particular “region of interest” (ROI) from an image by automatically or assertively delineating the anatomical structures and other ROIs [ 51]. Image segmentation has an important function in 3D reconstruction and many medical imaging applications.
In the early days of segmentation, medical technicians would sketch the contours slice by slice using a mouse or trackball (Fig. 4). This process is highly time-consuming and suffers from intra- and inter-observer variabilities [ 50].
Fig.4 Transverse section of the follicular unit at the largest dimension of the sebaceous gland. Blue margin, sebaceous lobules located in the angular position; yellow margin, sebaceous lobules located in the counter-angular position. APM, arrector pili muscle; HF, hair follicle; SG, sebaceous gland. Masson’s trichrome stain. Scale bar= 200 µm. Cited from Song et al.’s Ref. [26], with permission from John Wiley and Sons. |
Many computer-aided segmentation algorithms have been proposed to improve the accuracy and computational speed of the segmentation methods. These algorithms are typically based on thresholds, clustering techniques, or deformable models. The knowledge of doctors and medical professionals is crucial for computer experts to design effective algorithms. Medical images are usually influenced by noise or partial volume effects [ 52]. Therefore, a priori knowledge on the biomechanical behavior of tissue structures or details on the imaging environment has an important function in image segmentation.
Ma [ 50] described a detailed classification scheme for segmentation algorithms. The applications for each algorithm, along with their advantages and disadvantages, are summarized below (Table 3).
Tab.3 Algorithms for medical image segmentation |
| Segmentation algorithm | Classification | Applications |
|---|---|---|
| Based on thresholds | Edge-based algorithms | Wavelet transform, Canny edge detection [53], Sobel edge detection, and Laplacian edge detection |
| Region-based algorithms | Region growing algorithms [54] | |
| Hybrid algorithms | Watershed algorithms [55] | |
| Based on clustering techniques | Supervised classification algorithms | K-nearest neighbor classifiers, maximum likelihood algorithms, supervised artificial neural network, support vector machine, active shape model, and active appearance model |
| Unsupervised classification algorithms | CM algorithms, fuzzy C-means (FCM) algorithms, iterative self-organizing data analysis technique algorithm, and unsupervised neural networks | |
| Based on deformable models | Parametric deformable models | Snake method [56] |
| Geometric deformable models | Geodesic active contour algorithm |
Segmentation algorithms can be classified into three categories, namely, thresholds, clustering techniques, and deformable models. The first group includes edge-based, region-based, and hybrid algorithms. Edge-based algorithms are preferred when the structures of interest have distinct quantifiable features. However, edge-based algorithms are seldom used alone, and are usually used as a pre-processing step [ 57]. Hybrid algorithms are preferred when the images have low noise or the desired regions have a relatively high signal-to-noise ratio. Complete segmentation can be achieved, but the algorithm tends to over-segment.
The second group can be divided into supervised and unsupervised classification algorithms. Supervised classification algorithms are preferred when the structures of interest are regular and not influenced by noise [ 58]. Segmentation accuracy can be greatly enhanced if properly modeled. Unsupervised classification algorithms are preferred for tumor detection in positron emission tomography [ 59]. Both algorithms in this group are constrained because of the large shape variations of organs and other structures in medical images.
The third group can be divided into parametric deformable models and geometric deformable models. Parametric deformable models are preferred when the structures of interest are thin, have open boundaries, or when the algorithms need to operate in real-time with the drawbacks of computational complexity and sensitivity to initial conditions. Geometric deformable models are preferred when structures have large shape variety or complicated topology, as in the case for tumor detection and cardiac segmentation. The geometric properties can be implicitly calculated, and topological changes can be easily handled. However, selecting the proper parameters for the algorithm can be time-consuming.
3D imaging and visualization
3D visualization technology is applied after registration and segmentation. The main purpose of 3D visualization is to generate a pictorial representation of serial 2D images.
3D visualization technologies can be classified as Surface Rendering (SR) and Volume Rendering (VR). SR was the first method used for 3D image visualization. It involves shining a virtual light source onto an object, and is quick and requires limited labor. However, a great amount of detailed information is lost during the rendering process. VR constructs an image projection by simulating the absorption and emission of light in an image stack along each ray path to the eye [ 60]. The 2D images are generated in a manner similar to viewing semi-transparent 3D data from a chosen point of view. Each individual voxel is assigned a transparency color [ 61]. Given that each pixel is colored and illuminated, VR achieves complete preservation of 3D grayscale and color image information [ 2].
VR has been widely applied to visualize tomography data in three dimensions. However, VR is rarely used for serial sections. As of this writing, SR remains the most popular method for representing LM serial section data in three dimensions. However, the use of VR is expected to increase because of its superior reconstruction performance.
Applications
Scientists have studied morphological information on different organs or tissue using 3D reconstruction technology. Table 4 summarized some applications of 3D reconstruction.
Tab.4 Applications of 3D reconstruction |
| Applications | Organ/tissue | Thickness/ number of sections | Fixation | Stain method | Displayed components | Software |
|---|---|---|---|---|---|---|
| 1 | Human scalp skin | 6 µm/200–350 | 10% neutral buffered formalin | Masson’s trichrome | Arrector pili muscles, hair follicles, sebaceous glands | “Reconstruct” software |
| 2 | Human lower limb skin | 5 µm/5 | 4% paraformaldehyde | Podoplanin Immunohistochemistry | Lymphatic vessels | Amira Version 5.0 |
| 3 | Human median nerve | 20 µm/4650 | Liquid nitrogen | Acetylcholinesterase histochemical staining | Motor nerve fibers and sensory nerve fibers | Adobe Photoshop CS2 software; 3D nerve visualization system (China) |
| 4 | Rat liver | 15 µm/146 | Liquid nitrogen | Glucose-6-phosphatase (G6Pase) histochemical staining | Glucose-6-phosphatase (G6Pase) | Histol (Germany) |
| 5 | Duck lung | 0.3 µm/194 | 2.5% glutaraldehyde solution | Toluidene blue stain | Air capillaries (ACs) and blood capillaries (BCs) | Corel Photopaint V.08® (Corel Corporation) |
Song et al. [ 26] reported the 3D reconstruction of serially sectioned scalp skin (Fig. 4) to describe the morphological relationship between arrector pili muscles and sebaceous glands (Fig. 5). First, 10% neutral buffered formalin-fixed skin specimens were cut at a thickness of 6 µm for 200–350 sections. Second, the sections were stained with Masson’s trichrome to distinguish the arrector pili muscles, hair follicles, sebaceous glands, and collagen fibers. “Reconstruct” software was used to reconstruct 3D images (Table 4, application 1).
Fig.5 Reconstructed three-hair follicular units. (a1, b1, and c1) Lateral views; (a2, b2, and c2) superior oblique views; (a3, b3, and c3) superior views. Blue, hair follicles; red, arrector pili muscle; yellow, sebaceous gland. Cited from Song et al.’s Ref. [26], with permission from John Wiley and Sons. |
Wu et al. [ 63] reported the 3D reconstruction of lymphatic vessels from serially sectioned lower limb skin (Fig. 6). Five sections of 4% paraformaldehyde-fixed skin specimens were cut at a thickness of 5 µm. Lymphatic vessels were identified with podoplanin antibody by immunohistochemistry techniques. Amira Version 5.0 software (Visage Imaging GmbH, Berlin, Germany) was used to reconstruct 3D images (Table 4, application 2).
Fig.6 Reconstructions of a 3D lymphatic capillary network of human lower limb skin specimens using five individually stained 5 mm serial sections with podoplanin. The reconstructed lymphatic vessels are shown in yellow. Cited from Wu et al.’s Ref. [63], with permission from John Wiley and Sons. |
Sun et al. [ 64] reported the 3D reconstruction of the human median nerve to implement the 3D visualization of sensory and motor fascicles (Fig. 7). Liquid nitrogen-fixed human median nerve specimens were cut at a thickness of 20 µm for 4650 sections. The motor and sensory nerve fibers were identified using acetylcholinesterase histochemical staining. 3D nerve visualization system (Institute of Computing Technology, China Academy of Sciences, Beijing, China) and Adobe Photoshop CS2 software were used for reconstruction (Table 4, application 3).
Teutsch et al. [ 65] reported the 3D reconstruction of parenchymal units in rat liver (Fig. 8). Liquid nitrogen-fixed rat liver specimens were cut at a thickness of 15 µm for 146 sections. Glucose-6-phosphatase activity was histochemically demonstrated to visualize the borders of individual parenchymal units. A computer program (Histol, LIST-electronic, Darmstadt, Germany) was used for reconstruction (Table 4, application 4).
Fig.7 (A) Transection of the median nerve at the place where the median nerve passed through the pronator teres. Among the 17 fascicles, six motor fascicles, six sensory fascicles, and five mixed fascicles can be found. (B) 3D visualization of a segment of the median nerve at this site. The direction of the arrow is from proximal to distal of the median nerve. Red represents the sensory fascicles; yellow represents the motor fascicles; and green represents the mixed fascicles. Cited from Sun et al.’s Ref. [64], with permission from John Wiley and Sons. |
Fig.8 3D reconstruction of the secondary parenchymal unit and draining central venular tree. (A) Front view of the secondary unit. (B) Back view. Numerals indicate the primary units that are color-coded. Portal tracts are marked in red (p1, p2, and p3). (C) Front view of the draining central venular tree. (D) Back view. Numerals indicate the different central venular branches that are color-coded to correspond to the primary units. The stem of the central venular tree empties into a sublobular vein (sv). Cited from Teutsch et al.’s Ref. [65], with permission from John Wiley and Sons. |
Woodward et al. [ 25] reported the 3D reconstruction of the lung of an adult muscovy duck, Cairina moschata, to elucidate the microstructure of the terminal respiratory units of the avian lung (Fig. 9). The lungs were fixed with 2.5% glutaraldehyde solution buffered in phosphate (osmolarity: 350 mOsm·L–1, pH 7.4) at a pressure head of 3 kPa (25 mmHg) by intratracheal instillation, and sliced transversely along the costal sulci. The samples containing parabronchi were embedded in epoxy resin (epon/araldite). Subsequently, the samples were cut at a thickness of 0.3 µm for 194 sections using glass knives. The structures of interest in each section were identified with toluidene blue stain. A commonly available computer graphics program (Corel Photopaint V.08®, Corel Corporation) was used to reconstruct 3D images (Table 4, application 5).
Large volumes of data generated in 3D reconstruction may be troublesome particularly for scientists working in the field of nerve system research. High-performance computing (HPC) and efficient computational analysis of entire data sets are highly necessary. HPC often uses multiprocessors or a cluster organization of several computers to enhance computation speed. However, computational methods of large-scale neuroanatomical data sets remain a challenge. Helmstaedter et al. [ 66] reviewed the computational methods on neuronal data generated by electron microscopy or LM. Reconstructions of the morphology of individual neurons and entire circuits can be achieved. They also summarized useful software for the analysis of large-scale neuroanatomical data sets focusing on EM data reconstructions of neurons. Helmstaedteret al. [ 67] recently reported the dense reconstruction of 950 neurons in the mouse inner plexiform layer, and their mutual contacts using the KNOSSOS program [ 68] (http://www.knossostool.org).
Fig.9 (A, B) Integrated (combined) reconstruction of the blood vessels (red) and airways (blue) (A) and that of the vasculature (B): the area of the reconstruction is close to the parabronchial lumen. Labels are similar to show the topographic locations of the same structures. AC, air capillary; BC, blood capillary; If, infundibulae; stars, points at which ACs interconnect. Evidently, the ACs and BCs are not mirror images. Scale bar, 10 µm. (C, D) Structure of the ACs. The ACs are heterogeneous, rather rotund structures that anastomose openly or via narrow tubules (stars). The spaces between the ACs are almost completely filled by BCs. If, infundibulum. (D) An AC and interconnection (star) are delineated with dashed lines. Scale bars: (C) 20 µm; (D) 10 µm. Cited from Woodward et al.’s Ref.[25], with permission from John Wiley and Sons. |
Conclusions
3D imaging and visualization techniques have important functions in scientific research, allowing scientists to study tissue structures from a universal perspective. Although many new 3D imaging and visualization techniques exist, 3D reconstructions of LM image sections is still one of the best methods for obtaining high-resolution reconstructions of almost any type of tissue. With the exception of CLSM and synchrotron-based microCT, few 3D techniques can match the sub-micron resolution of LM image sections. LM image reconstruction can be used with almost any tissue, ranging from bones to soft tissue. In addition, this technique has no maximum size limit for a specimen. The entire human body can even be cut into slices and reconstructed.
We are happy to observe increasing progress toward improving 3D reconstruction. The use of digital pathology panoramic scanning microscopy, as well as automatic block-face image capturing methods, such as surface imaging microscopy and episcopic fluorescence image capturing [ 69, 70], make this process less time-consuming. New methods for LM sectioning techniques [ 71] are also likely to reduce the distortion rate and increase efficiency. Although these drawbacks have yet to be eliminated at this time, 3D reconstruction of LM image sections will be an irreplaceable tool in the years to come.
Outlook
Tissue-engineered scaffold
We believe that 3D reconstruction techniques will help scientists and engineers to manufacture highly bionic tissue-engineered scaffold materials. “Highly bionic” is difficult to define, but the structure and composition of tissue engineering scaffolds should mimic those of normal tissue. DermACELL®, an allograft material composed of acellular dermal matrix, serves as a good biocompatible scaffold for skin regeneration and scar prevention [ 72]. To some extent, the more similarities between tissue-engineered scaffolds and normal tissue, the better results in repair and replacement. With the help of 3D reconstruction techniques, anatomists can acquire large-scale histological data for human organs and properly display the structural information. Materials scientists can then use these data to generate accurate large-scale models for improving the fabrication of bioengineered tissue scaffolds.
3D printing
3D printing is a process of making a 3D solid object from a digital model (usually the STL format) using an additive process, in which successive layers of material are laid down in different shapes. 3D printing technology has been around since the 1980s, but became widely available commercially in the early 2010s. The digital model for 3D printing can be generated by a 3D reconstruction software, such as Amira Version 5.0 or Materialise Mimics 10.01.
We believe that 3D reconstruction and 3D printing will have important functions in bionic manufacturing, including tissue-engineered scaffolds. The reasons are as follows.
First, 3D printing machines can print materials layer by layer, allowing 3D models of target organs to be reconstructed from a series of serial image sections. The 3D models can be imported into the printer control system without any transformation or reprogramming, which may cause lower accuracy or distortion. Therefore, with 3D reconstruction and 3D printing technology, rebuilding tissue and organs in accordance with how they are naturally assembled should be possible. This feature would be advantageous for manufacturing tissue-engineered scaffolds. Second, after tissue-engineered scaffolds have been fabricated, seeding them with stem cells is difficult. Although perfusion seeding methods can improve the seeding rate, cells usually distribute on the surface of the scaffold. The application of 3D printing techniques may provide a solution for this problem. Scaffolds and stem cells can be printed simultaneously using multiple printer heads, allowing stem cells to be evenly distributed throughout the scaffolds.