A variety of biological molecules in almost any cell, organism, and tissue can now be imaged in large part due to advancements in microscopy equipment and improvements in labelling techniques. However, what has remained challenging is that often these images may be a challenge to interpret or rigorously evaluate. Particularly, it can be difficult to examine cells of interest if overlapping, shading, or colocalization may occur (Booij et al., 2019; Wilkinson, 1994).

 

Several factors may be responsible for the limited use of any current use of software that involves detailed image analysis. Particularly, customization of the software is often required to achieve optimal results for each specific laboratory’s purpose (Morris et al., 2010). Furthermore, while there are certain cases where images can be easily separated between foreground and background, sometimes images taken from microscopy may push the limits of detection, such as when there are significant amounts of non-specific signals in cells, or where intracellular signals are not much different in property than the extracellular signal (Garain et al., 2006). These conditions may occur due to labelling probes lacking specificity or the probes having inherently a weak signal relative to existing endogenous molecules. Finally, an additional factor may consist of overlapping molecules amongst a 3D space that appears washed in a 2D image, resulting in the mixing of colors and distortion of pixel values (Despotovi? et al., 2015).

 

These problems can often be corrected through the combination and modification of existing software, either through the program being available open-source or incorporating a multi-step pipeline that utilizes multiple analysis software. However, a singular software tool that can effectively and intuitively be used by non-programmers to reliably segment different sets of photos from fluorescence signals does not exist.

 

DeatJ was developed to provide customizability in segmentation on top of the existing functionality of ImageJ through modification and combination of different existing plugins that are further personalized by the user. This paper aims to describe the operation and use of the DeutJ software.

Implementation

 

DeutJ was developed using ImageJ version 1.72 and Java version 8 (Schneider et al., 2012). The codebase was built off of open-source ImageJ software. To use the DeutJ program, one needs to be able to run java executables. When launching DeutJ, the user must select the folder that contains the images required for analysis, and will be prompted to select all files used for analysis. Once selected, parameter values will be entered into a dialog box, which includes the size limits of objects, the masking algorithm, and any preprocessing adjustments such as brightness or contrast modifications. Once the parameters are entered, the macro will search through the selected folder and one by one produce segmentations of those images, saving them in the same folder as before.

 

Images are first converted into the 8-bit format. Then, the program conducts a series of normalizations and local background subtractions that improve the process of separating the foreground from the background before thresholding. This is done to improve the masking accuracy by aligning the image histogram in conjunction with the threshold algorithm. First, a contrast correction factor (CCF) will be calculated for each pixel of the unprocessed 256-bit image and stored as a floating point. This is done to achieve a standardized saturation percentage of 35%, though this value is adjustable by the user. The CCF will then be applied to all pixels in the image, to produce a final photo with a contrast between the range of -256 and +256, which will be printed on the screen. Negative values indicate that there has been a decrease in contrast, while a positive value will indicate an increase in contrast. The CCF is calculated as follows:

 

 

 

 

Thirdly, within individual regions of interest, a Sternberg’s local background subtraction is conducted to further reduce shading effects (Sternberg, 1983). Sternberg’s method of subtraction is particularly effective for uneven shading and correcting for large spatial variations in background intensity. An example of before and after an image has been background subtracted is displayed in Figure 2. This method is advantageous as most contemporary thresholding methods use a global subtraction process, which performs poorly when provided with unevenly shaded images.

 

These three pre-processing adjustments serve to reduce background artifacts, shading, and unevenly focused images, and handle a wide array of tagged samples in our lab. Previously, only fluorescence confocal microscope images could be reliably masked, while DeutJ can now mask samples taken with light microscopy or inverted microscopes, despite the presence of significant shading. Figure 3 shows an example of the DeutJ program compared with ImageJ’s “automatic thresholding” function.

 

DeutJ also incorporates post-thresholding improvements over traditional functions available in ImageJ. Notably, a size threshold has been implemented, which filters out small artifacts that arise from masking. The program achieves this by using a Sobel edge detection algorithm to find the gradient of image intensity at each pixel within an image (Garain et al., 2006), and the direction of maximal increase (Despotovi? et al., 2005) from light to dark of said pixel, to construct the edge lines (Vincent and Folorunso, 2009). Once edges have been detected, the program filters all edges out under a size of 25 pixels squared, which we have found to be most optimized for cells examined, but are adjustable in the software (Figure 4).

 

 

Where, Gx and Gy are gradient approximations in the approximations of x and y direction, respectively.

 

 

Another post-mask feature the program includes a median filter, which replaces each pixel with a median value in the array of its 3×3 neighbourhood (Chen et al., 1999). When the median filter is applied to a binary image, this produces rounded out cells that more accurately capture the natural shape of cells (Figure 5).

 

Finally, the DeutJ uses a triangle thresholding algorithm (Tajima and Kato, 2011). Popular algorithms like Otsu assume a foreground and background class, and seek to minimize the variance of each class in the image’s resulting bimodal histogram, which is later split for image segmentation (Zhu et al., 2009). Unfortunately, in the case of uneven shading, Otsu will “force” a bimodal distribution and inadvertently encapsulate objects into the background or capture background elements into the foreground.

 

To combat this, DeutJ uses the triangle algorithm, which thresholds by determining the line of maximal length perpendicular to the connection between the maximum and minimum points of the intensity histogram. This works well with the conjunction of linear normalization adjustments described previously (Figure 6).

 

 

Operation

 

DeutJ works on desktop computers running either OS or Windows operating systems. Computers must have Java 8 or later installed and requires no internet connection. The software is open source under Apache 2.0 license. To use, run the file from an IDE, and select the folder that contains the desired microscopy images. All images of interest must be in a single folder. Click “open folder”, and the program will automatically create a folder inside the previously selected folder with file names that end with “_mask #X”, where X corresponds to the n^th mask generated.

This paper aims to present the results of DeutJ, a software tool for segmenting and masking images from various types of fluorescent microscopy. Most notably confocal, DeutJ also works with widefield types. The software is designed to accurately identify and segment specific objects within an image, and then apply a mask to those objects in order to obscure or remove them given differing background intensity values. Our lab uses GFP, but any molecule-of-interest that emits fluorescent radiation will suffice. Within our laboratory, DeutJ performs with a higher degree of precision and accuracy compared to ImageJ’s native plugins. However, further research is needed in order to fully evaluate its performance in other areas of image segmentation. Furthermore, requiring ImageJ installed on the user’s computer when running the application is a limitation. Therefore, in future work we hope to turn DeutJ into a standalone program in order to increase portability and access. Though ImageJ-accompanying macros and plugins have been well documented, few have attempted to build upon the software’s existing segmentation tools (Guzmán et al., 2014; Pool et al., 2008; Poulet et al., 2014; Roels et al., 2020; Valente et al., 2017; Whitehead et al., 2019; Handala et al., 2019).

DeutJ is a robust, easy-to-use, free tool that automates the segmentation of confocal microscopy images under poor light conditions. It works particularly well to create masks from images that suffer from uneven light shading and significant speckle. In our laboratory it is used to segment kidney samples that have been shaded with GFP. Furthermore, the software may serve many researchers in applications related to counting labeled cells or even for test segmentation software in photos. Future work may build off of DeutJ’s open source to develop imaging software for non-confocal segmentation.

The author has not declared any conflict of interests.