Near-Infrared Fluorescent Digital Pathology for the Automation of Disease Diagnosis and Biomarker Assessment

Hematoxylin-eosin (H&E) staining of tissue has been the mainstay of pathology for more than a century. However, the learning curve for H&E tissue interpretation is long, whereas intra- and interobserver variability remain high. Computer-assisted image analysis of H&E sections holds promise for increased throughput and decreased variability but has yet to demonstrate significant improvement in diagnostic accuracy. Addition of biomarkers to H&E staining can improve diagnostic accuracy; however, coregistration of immunohistochemical staining with H&E is problematic as immunostaining is completed on slides that are at best 4 μm apart. Simultaneous H&E and immunostaining would alleviate coregistration problems; however, current opaque pigments used for immunostaining obscure H&E. In this study, we demonstrate that diagnostic information provided by two or more independent wavelengths of near-infrared (NIR) fluorescence leave the H&E stain unchanged while enabling computer-assisted diagnosis and assessment of human disease. Using prostate cancer as a model system, we introduce NIR digital pathology and demonstrate its utility along the spectrum from prostate biopsy to whole mount analysis of H&E-stained tissue.

A FTER HEMATOXYLIN 1 AND EOSIN 2,3 were intro- duced separately for tissue staining in 1865 and 1876, respectively, they were combined by Busch in 1878 4 to produce the current gold standard for pathology, the hematoxylin-eosin (H&E) stain. 5More than a century later, H&E remains the most widely used technique around the world as it provides rich information content about tissue disease status.
Nevertheless, several fundamental difficulties plague pathology.First, the learning curve to interpret complex benign and disease tissue patterns provided by H&E requires significant training time, limiting the number of available pathologists to examine patient tissues.Prostate cancer is a particularly difficult clinical example, where diagnosis and disease grading by a highly trained genitourinary (GU) pathologist on the sparse tissue sampling of prostate biopsies yields an interobserver agreement of only 70%. 6By comparison, when the same specimens are analyzed by generally trained pathologists, agreement falls to barely moderate. 7The complexity of accurate prostate cancer diagnosis and grading from H&E-stained prostate biopsies can be further demonstrated through the only 50% intraobserver repeatability by Dr. Gleason himself, who developed the World Health Organization accepted Gleason grade for prostate cancer. 8Second, the addition of immunohistochemical (IHC) staining to H&E can significantly improve diagnostic accuracy but typically requires consecutive sections, which may be hundreds of microns away from the original H&E-stained section within the tissue block.Thus, the features of interest for diagnostic and prognostic accuracy will at best have changed shape and size and in the worst case may not be present in both the H&E-and IHCstained slides depending on the distance between sections.This is especially problematic for sparse tissue samples, such as routine diagnostic biopsy, where little tissue is collected from which to recognize disease patterns using conventional H&E staining.Additionally, when colorimetric methods are used for IHC detection, only a single antigen at a time can be analyzed, requiring multiple sections for additional antigens.Costaining with H&E and colorimetric IHC is also not desirable as the precipitated colored pigments obscure the original H&E stain.Clearly, a possible method for simultaneous immuno-and H&E staining is through the use of immunofluorescence, which does not interfere with conventional H&E.However, it is not possible to use visible fluorophores in the presence of hematoxylin and eosin as both chromophores have strong absorbance in the visible region and eosin is highly fluorescent over much of the visible spectrum.0][11] Lastly, disease diagnosis through H&E staining, especially for prostate cancer, relies primarily on overall tissue pattern recognition, whereas modern biomarker studies, especially for cancer stem cells, stromal cells, and immune cells, require single-cell analysis.
Two previously published strategies to facilitate improved coregistration of immuno-and H&E staining include spectral unmixing of precipitated immunostains 12 and H&E destaining followed by virtual microscopy. 13lthough advantageous in their ability to add information to H&E-stained tissue sections, both of these techniques permanently alter the H&E-stained slide.Conventionally, H&E-stained slides are stored for future examination by the pathologist; thus, spectral unmixing of precipitated immunostains and H&E destaining techniques alter normal clinical workflow.
In this study, we report a new technology, near-infrared (NIR) digital pathology, which exploits invisible NIR fluorescent light to add multiple layers of phenotypic and/or genotypic information content to standard H&E-stained tissue sections on the same tissue section.Immunofluorescence staining using NIR fluorophores can alleviate the spectral overlap of H&E with visible fluorophores while maintaining the integrity of the original H&E-stained slide, enabling perfect coregistration of immunostaining and H&E without altering that ability for archiving and future examination.

Spectral Characterization
Absorbance and fluorescence spectra of hematoxylin (Fisherbrand Gill Method, Stain 3), eosin (Eosin Y, Fisher Scientific, Waltham, MA), rabbit antidonkey conjugated Alexa Fluor 680 (Life Technologies, Grand Island, NY), and mouse antigoat (Jackson Immunoresearch, West Grove, PA) conjugated ZW800-1 14,15 were collected using a Cary 50 Bio UV-Visible spectrophotometer and Cary Eclipse fluorescence spectrometer, respectively (Varian/Agilent, Mattapoisett, MA).Excitation wavelengths blue-shifted from the absorption maxima were chosen for each chromophore to enable collection of full fluorescence spectra.Spectra from hematoxylin were collected in deionized water.All absorbance and fluorescence spectra were normalized to accurately portray spectral shape and wavelength range among the chromophores.

Labeling of Antibody with ZW800-1
Mouse antigoat secondary antibody was conjugated to ZW800-1 N-Hydroxysuccinimide (NHS) ester (10 mM stock solution in anhydrous dimethyl sulfoxide) at a conjugation ratio of 2 to 3 fluorophore molecules per antibody using the following protocol.The antibody was buffer exchanged with phosphate-buffered saline (PBS) using a 10,000 Da molecular weight cutoff (10K MWCO) spin column (Vivaspin, Fisher Scientific) prior to resuspension in PBS.The pH of the antibody solution was raised to 8.0 using sodium phosphate dibasic (35 mM, Fisher Scientific).Twenty molar equivalents of ZW800-1 NHS ester was added to the antibody solution dropwise.The reaction was agitated gently at room temperature (RT) for 3 hours.
The conjugated antibody was purified from the free fluorophore using an Akta Prime (GE Healthcare Life Sciences, Pittsburgh, PA) gel filtration chromatography system and desalting columns (Bio-Scale Mini Bio-gel, Bio-Rad, Hercules, CA).The antibody solution was loaded onto the column where PBS (0.13) was run at 2 mL/min.After about 6 minutes, the conjugated antibody flowed from the column, which was collected in 1 mL fractions 5 to 7 minutes following column loading.The conjugated antibody was further purified and concentrated using a 10K MWCO spin column.The conjugation ratio and protein concentration for the conjugated antibody were calculated using absorbance measurements collected with the Cary 50 Bio UV-Visible spectrometer, where the extinction coefficient (e) was 210,000 M -1 cm -1 at 280 nm for the antibody and 249,000 M -1 cm -1 at 770 nm for ZW800-1. 15tigen Retrieval for Prostate Tissue Specimens Discarded anonymized human prostate tissue specimens were acquired under an approved Institutional Review Board protocol from the Department of Pathology at Beth Israel Deaconess Medical Center as formalin-fixed paraffin-embedded tissue blocks.The tissues were sectioned at 4 mm thickness and dried thoroughly.The paraffin was removed from the slides using an Autostainer XL (Leica Microsystems, Buffalo Grove, IL), where the slides were incubated as follows: xylenes (5 minutes), xylenes (5 minutes), 100% ethanol (1 minute), 100% ethanol (1 minute), 95% ethanol (1 minute), and water (3 minutes).To unmask antigens for antibody staining, antigen retrieval was completed in the following buffer: 10 mM Tris base (Fisher Scientific), 1 mM ethylenediaminetetraacetic acid (EDTA; Fisher Scientific), and 0.05% Tween 20 (Sigma-Aldrich, St. Louis, MO) in PBS, pH 8.The slides were boiled in the Tris-EDTA buffer for 20 minutes, after which the buffer and slides were removed from the pressure cooker and allowed to cool at RT for 20 minutes.

Simultaneous Same-Slide NIR Fluorescence and H&E Staining
The slides were washed three times with PBS-T (PBS with 0.1% Tween 20).All slides were blocked with 5% normal goat serum (Jackson Immunoresearch catalog #005-000-001, West Grove, PA) in PBS for 1 hour at RT. Monoclonal mouse antihuman high-molecular-weight cytokeratin primary antibody (Dako clone #34bE12, Dako, Carpinteria, CA) was diluted from the stock solution at 1:50 and mixed with monoclonal rabbit antihuman a-methylacyl-CoA racemase (AMACR, Dako clone #13H4) diluted from the stock solution at 1:100.The primary antibody solution was incubated on the tissue in a humidified chamber overnight at 4uC.Following the primary antibody removal, the slides were washed three times with PBS-T.Donkey antimouse conjugated to Alexa Fluor 680 (Life Technologies catalog #A10038, Grand Island, NY) was diluted 1:50 and mixed with unlabeled goat antimouse (Jackson Immunoresearch catalog #115-005-146) diluted 1:50 and incubated with the tissue for 1 hour at RT protected from light.The secondary antibody solution was removed, and the slides were washed three times with PBS-T for 5 minutes per wash.Mouse antigoat conjugated to ZW800-1 was diluted to 0.25 mM protein concentration and incubated with the tissue for 1 hour at RT protected from light.After washing of the tertiary antibodies, the tissue was fixed with 2% paraformaldehyde in PBS for 15 minutes at RT.The fixative was removed, and the tissue was washed once with PBS for 5 minutes.

Simultaneous Brightfield and NIR Fluorescence Microscopy
Simultaneous brightfield and NIR fluorescence microscopy images were collected on a modified Nikon 55i fluorescence microscope with a programmable, motorized, encoded stage (Prior Scientific, Rockland, MA).The stage was equipped with interchangeable adaptors to accommodate either a 1 3 3 inch (prostate needle biopsies) or 2 3 3 inch (prostate whole mount) slide.The microscope was equipped with a custom beam splitter on the camera port to accommodate a 12-bit color camera (Prosilica, Allied Vision Technologies, Burnaby, BC) for brightfield imaging and an Orca-ER 12-bit camera for fluorescence imaging (Hamamatsu, Bridgewater, NJ).All filters were from Chroma Technologies (Brattleboro, VT).For imaging of Alexa Fluor 680, the xenon lamp was passed through a 650 6 22.5 nm BP excitation filter, a 680 nm LP dichroic, and a 710 6 25 nm BP emission filter.For imaging of ZW800-1, the xenon lamp was passed through a 750 6 25 nm BP excitation filter, a 790 nm LP dichroic, and an 824 6 23.5 nm BP emission filter.The white light used for brightfield illumination was depleted of all NIR wavelengths using a 650 nm SP filter.
Using custom software, a calibration image was collected for flat field correction for the color, 700 nm, and 800 nm channels.The exposure times for the 700 nm and 800 nm channels were set based on the average fluorescence signal intensity seen in the sample data set to prevent image saturation.The slides were scanned with 10% overlap first for color, followed by 800 nm fluorescence emission, then for 700 nm fluorescence emission.A color data set using a 43 objective from the entire slide was collected and used for specimen detection.
All 2 3 3 inch slides were scanned at a maximum magnification of 103.The z-focus for the entire slide was determined using two focused points at one end of the slide and a third focused point at the opposite end of the slide to calculate the focal plane.The focal information was used to collect the color images directly and used with an empirically determined offset to collect the 700 nm and 800 nm fluorescence images.The 1 3 3 inch slides containing prostate needle biopsy specimens were scanned at a magnification of 403 to enable sufficient magnification to identify nucleoli for diagnostic purposes.Due to inhomogeneities in the glass height of the slides, a custom autofocusing algorithm was used to determine z-focus, which was autofocused using seven z-steps every five images.The focused z-coordinates were saved and used with the empirically derived offset factor to acquire 700 nm and 800 nm fluorescence images.Following completion of scanning, all data was flat field corrected using the acquired calibration images for color, 700 nm, and 800 nm fluorescence.Using custom software, the corrected data were registered and stitched into a single map, 16 which was digitally downsampled from the maximum resolution to 13 resolution in eight steps that were each saved as individual images.
NIR fluorescence images were acquired and saved as raw 12-bit data (pixel intensity values ranging from 0 to 4,095).Preliminary scanning permitted adjustment of camera exposure time to ensure that pixel saturation did not occur.Brightness, contrast, and gamma for the entire stitched image of a particular NIR wavelength could then be adjusted to maximize the dynamic range of the displayed data.

Spectral Characteristics of NIR Immunofluorescence, Hematoxylin, and Eosin
Due to the absorbance (Figure 1A) and emission (Figure 1B) of hematoxylin and eosin, visible fluorescence cannot be used with H&E staining.Hematoxylin has broad absorption between 400 and 700 nm, with virtually no fluorescence emission.Eosin has much narrower absorption (450-550 nm) but exhibits strong fluorescence emission ranging from 500 to 700 nm.NIR fluorophores, which emit between 700 and 900 nm and typically have a Stokes shift of $ 15 nm, are ideal for adding information content to H&E slides.As shown in Figure 1, emission wavelengths for the two most common classes of NIR fluorophores, that is, those with peak emission < 700 nm and < 800 nm, are nonoverlapping with H&E.

Simultaneous Two-Wavelength NIR Fluorescence and H&E for Disease Detection
Unlike many diseases, where the presence or absence of single cells can provide diagnostic and prognostic value, prostate cancer requires high-level analysis of glandular patterns.Analysis is further confounded by the fact that a 4 mm section of a three-dimensional object results in many distorted gland geometries that are not interpretable.NIR fluorescence, however, can render virtually any tissue section informative.The biomarker a-methylacyl-CoA racemase (AMACR) stains the spectrum of abnormal prostate tissue from premalignant prostatic intraepithelial neoplasia (PIN) through Gleason grade 5 prostate adenocarcinoma, [17][18][19][20] whereas high-molecular-weight cytokeratin (CK903) is a basal cell marker that stains normal glands and PIN. 21,22The combination of the two biomarkers can aid in tissue classification when added to H&E staining where benign (AMACR2, CK903+), PIN (AMACR+, CK903+), and malignant (AMACR+, CK9032) glands demonstrate distinct immunostaining patterns. 18,20,22As shown in Figure 2, NIR fluorescence permits independent assessment of biomarkers in the context of the same tissue specimen, without alteration of the conventional H&E stain.It also permits overlay of the two NIR fluorescence channels with the H&E, thus providing single cell-level coregistration of biomarkers and histology.Of note, 800 nm fluorophores were purposely used for nuclear antigens because of hematoxylin's strong absorption into the red wavelengths.Additionally, tertiary antibody staining was used to amplify the signal from weak antigens.

Digitized Biomarker Maps of Large Specimens Using NIR Digital Pathology
As our understanding of diseases such as prostate cancer evolves, it has become clear that tumor heterogeneity can be extreme. 23,24Whole mounts, where the entire resected prostate gland is sectioned and stained on 2 3 3 inch slides, provide a map of disease heterogeneity, which can be difficult to fully catalog and analyze manually.As shown in Figure 3A, NIR digital pathology permits biomarker detection and quantitation over an entire whole mount section while preserving the H&E staining that is required by pathologists for disease confirmation.Areas suspicious for malignancy can be rapidly detected using the NIR fluorescence channels and confirmed at higher resolution through both immunofluorescence and H&E (see Figure 3A).

Semiautomated and Automated Analysis of Sparse Biopsy Samples
The most profound problem in prostate cancer care is making significant clinical decisions based on the sparse tissue obtained from core needle biopsy.Even if 20 or more needle cores are analyzed from a single patient, this represents only a small percentage of the prostate volume, and heterogeneity can be substantial (see Figure 3A, for example).Because prostate cancer requires high-level analysis of glandular patterns and intra-and interobserver repeatability can be relatively low, accurate diagnosis and grading of biopsies are particularly challenging due to the sparse tissue sampling.As shown in Figure 3B, NIR digital pathology permits prostate needle biopsies to be analyzed at near single-cell resolution, with areas suspicious for PIN and adenocarcinoma initially identified without pathologist intervention.Moreover, using simple image analysis, the fraction of the biopsy occupied by malignancy or premalignancy can be quantified with high precision compared to the conventional clinical method of qualitatively assigning fractional involvement.
The potential of NIR fluorescence-derived biomarker information is highlighted in the representative example shown in Figure 4, where a trained GU pathologist was asked to identify prostate cancer on a needle biopsy specimen using only the digital H&E image at two different magnifications.Although the large areas of Gleason score 6 (3+3) adenocarcinoma were readily identified, there were also areas representing partial glands that were not definitively identified by H&E alone.However, by biomarker assessment, these glandular structures demonstrated AMACR staining and were further examined for malignancy.Because many needle biopsies have indistinguishable glands present, 25 independent biomarker assessment could significantly improve pathologic diagnosis as well as intraand interobserver repeatability.Prostate specimens were stained with an AMACR-specific primary antibody and visualized using Dk-Ms AF680 (700 nm NIR fluorescence channel), as well as a CK903-specific primary antibody and visualized using Ms-Gt ZW800-1 (800 nm NIR fluorescence channel).In the H&E/NIR Merge image, the 700 nm channel is pseudocolored in yellow and the 800 nm channel is pseudo colored in lime green.Representative images of benign, malignant, and mixed benign and malignant prostate glands are shown.Scale bars 5 200 mm. Figure 3. Biomarker analysis of whole mount radical prostatectomy specimens and sparse prostate needle biopsy specimens.A, Two 3 3 inch whole mount sections of prostatectomies were stained as described for Figure 2 and scanned at 103 magnification with 10% overlap followed by stitching of the 2,850 individual images into a single map of the prostatectomy section for the hematoxylin-eosin (H&E), 700 nm near-infrared (NIR) fluorescence, and 800 nm NIR fluorescence channels.The images were then digitally downsampled to visualize the entire specimen.Scale bar 5 5 mm.Insets on bottom two rows correspond to malignant (solid box and arrow) and benign (dashed box and arrow) areas.Inset scale bars 5 200 mm.B, A representative prostate needle core biopsy was stained and displayed as described for Figure 2 except scanning was performed at 403 magnification.The images were digitally downsampled to visualize the entire specimen.Scale bar 5 500 mm.

Discussion
As our understanding of cancer evolves, the literature strongly suggests that enhanced information content will improve diagnosis, treatment selection, and patient prognosis.With more than a century of diagnostic and prognostic experience using H&E tissue staining, a technology that enhances its information content would significantly improve patient care.In addition, the advances in genomics and proteomics as well as the novel phenotypic and genotypic biomarkers for diseases are rapidly developing; however, the technology to validate these against gold standard H&E has not been previously available.In the work presented herein, we demonstrate a facilitating fluorescencebiomarker assessment technology termed NIR digital pathology for single-cell coregistration with H&E staining.
The enabling technology behind our NIR digital pathology technique is the spectral separation between the absorption and emission spectra of NIR fluorophores and the H&E chromophores.In the current work, we have demonstrated proof of principle studies on prostate whole mounts and biopsy specimens using two-color antibodybased staining.High-molecular-weight cytokeratin (CK903) was used as a biomarker to delineate benign glands, whereas AMACR, an antigen present in the spectrum of premalignant and malignant prostate glands, was used to mark suspicious glandular tissues.Whole mount prostatectomy specimens were imaged for CK903, AMACR, and H&E through NIR digital pathology, demonstrating the significant disease heterogeneity present in the representative example of prostate adenocarcinoma.Additionally, prostate biopsy specimens were also imaged using NIR digital pathology, where disease diagnosis generally shows increased variability due to the sparse tissue sampling.Although proof of principle was established in this study using antibodies, NIR fluorophores are routinely conjugated to nucleic acids, small molecules, and other protein ligands, permitting virtually any combination of phenotypic and/or genotypic markers to be analyzed in future work.In addition, NIR digital pathology should perform similarly well with pathology stains other than H&E.The only two requirements are that they do not have spectral overlap with NIR fluorophores and that processing conditions are compatible with polymethine indocyanines, for example, avoidance of highly basic conditions.
NIR digital pathology provides high versatility in quantifying and interpreting NIR fluorescence signals.
Because scanning is typically performed with a 403 objective (< 1-2 mm resolution), individual cells can be readily identified from the H&E image based on simple segmentation for chromatin.Once identified, NIR fluorescence signals can be assigned to individual cells or collections of cells (e.g., glands).Because of the high dynamic range (12 bit) of the NIR data sets, the distinction between NIR signals deemed positive or negative is straightforward and can even be done on the fly by adjusting segmentation windows and thresholds.Using these methods, the presence or absence of biomarkers can be quantified at the single-cell level, glandular level, or section level as needed.
As NIR digital pathology is further developed and validated, we envision that this technology will be useful in a number of clinical scenarios.First, it could be used to automatically prescreen biopsy specimens for the presence of disease so that pathologists would spend their time on informative areas of the slide, spending less time ''finding'' suspicious areas with the potential for decreased incidence of missing small lesions.The combination of computer-aided diagnosis via H&E staining as well as the addition of biomarker information through NIR immunofluorescence could enable prescreening for disease.Second, NIR digital pathology can be used to perform immunostaining on the same slide.This would eliminate the current laborious technique of consecutive sectioning and the associated coregistration problems.Third, for validation of new biomarkers, where comparison with gold standard H&E is essential, NIR digital pathology permits two or more biomarkers to be simultaneously identified on a single slide.Finally, NIR digital pathology could enable effective telemedicine.With a worldwide shortage of specialty pathologists, especially in GU pathology, digitization and prescreening by NIR digital pathology would permit transmission of relatively small amounts of digital data as opposed to terabytes of benign data present on each slide.It should be stated clearly that the goal of this technology is not to replace pathologists but rather to increase diagnostic efficiency and provide tools for validation of next-generation biomarkers with conventional H&E staining with which the medical community has more than 130 years of experience.
The work presented herein serves to demonstrate the basics of the technology and its potential utility in prostate cancer diagnosis.Future work will be focused on validation of the NIR digital pathology for prostate cancer diagnosis as well as digital image analysis and computer-aided tissue pattern recognition.As is evidenced from the representative examples of prostate cancer, segmentation could be used to identify areas of benign, premalignant, and malignant tissues to quantify disease.7][28] Through the combined use of H&E pattern recognition and NIR digital pathology, it could be possible to standardize the assessment of cancer versus benign tissue areas, something that is now completed in coarse, qualitative terms but that is one of many factors used by clinicians to make therapeutic decisions.

Figure 2 .
Figure 2. Disease detection using simultaneous near-infrared (NIR) fluorescence and hematoxylin-eosin (H&E).Prostate specimens were stained with an AMACR-specific primary antibody and visualized using Dk-Ms AF680 (700 nm NIR fluorescence channel), as well as a CK903-specific primary antibody and visualized using Ms-Gt ZW800-1 (800 nm NIR fluorescence channel).In the H&E/NIR Merge image, the 700 nm channel is pseudocolored in yellow and the 800 nm channel is pseudo colored in lime green.Representative images of benign, malignant, and mixed benign and malignant prostate glands are shown.Scale bars 5 200 mm.

Figure 4 .
Figure 4. Annotated core needle prostate biopsy.A, A representative core needle prostate biopsy stained and displayed as described for Figure3Awas annotated by an experienced genitourinary (GU) pathologist.Malignant regions are circled in black and were found to contain prostatic adenocarcinoma with a Gleason score of 6 (primary Gleason grade 3 and secondary Gleason grade 3; 3+3).Scale bar 5 200 mm.B, Zoomed images of the middle portion of the core needle prostate biopsy demonstrate that near-infrared (NIR) fluorescence identifies five otherwise indeterminant glandular structures that are positive for malignancy but were not annotated by the GU pathologist.Arrows point to these three AMACRpositive glands in the hematoxylin-eosin (H&E) image.Scale bar 5 50 mm.