(T1330-01-03) Stain-Free Approach to Determine and Monitor Cell Heath Using Supervised and Unsupervised Image-Based Deep Learning

Purpose: The demand for cell-based medicinal products (CBMPs) is increasing since they provide a promising outlook for treating complex and rare diseases that do not have readily available protein-based therapeutic cures. Unlike protein-based biologics, the final product for CBMPs are live cells that are far more intrinsically fragile and susceptible to unfavorable conditions.^1–5 Existing techniques to evaluate and monitor cell health characteristics typically constitute labor-intensive, expensive and highly-specific staining assays and biomarkers. ^2,4,6–8 Consequently, there is a need for sensitive, orthogonal, and complementary analytical techniques that allow extensive process and product characterization of cell health characteristics.^7,9–12 While the definition of a cell’s “state” during the process of cell death is not straightforward,^13,14 there is consensus that the status of cell’s health manifests into macroscopic morphological changes which can be observed via light microscopy images of cells.^15–19 Our group, as well as many others, have used a combination of flow imaging microscopy (FIM) and convolutional neural network (CNN) based supervised (i.e., algorithm trained with human-generated labels for images) machine learning techniques to show that the morphological features extracted from images of subvisible particles found in protein formulations can be interpreted as “fingerprints” that carry insight into the particle formation mechanisms.^20–24 In this work, we leverage this CNN based supervised fingerprint algorithm in conjunction with FIM to predict cell health metrics using cellular morphology extracted from images of unstained Jurkat cells (immortalized human T-lymphocyte cells). We then use an unsupervised (i.e., algorithm trained without human-generated labels for images) variational autoencoder (VAE) algorithm to demonstrate the utility of VAEs as a powerful exploratory and process monitoring analytical tool.

Methods: We first task our supervised CNN-based “fingerprint” algorithm to differentiate between FIM images of unstained “live” cells (i.e., Jurkat cells that were left untreated) and unstained “dead” cells (i.e., cells incubated in heated water bath; a stress known to induce accidental cell death by the process of protein unfolding and disruption of lipid bilayers in cell membrane^25–27). We use the resulting fingerprints to estimate percent viability and further compare these estimations with results from a conventional flow cytometry assay using a permeable stain dye label. By using external environmental stimulations (e.g., heat shock, incubation in hydrogen peroxide) and chemotherapeutic drug treatments (e.g., incubation in Camptothecin and Staurosporine), we induce accidental (necrosis) or programmed cell death (apoptosis) in Jurkat cells and task a new supervised fingerprinting network to differentiate between human generated labels for images of viable, dead and apoptotic cells. We then supply the same FIM images to an unsupervised VAE algorithm and use Unified Manifold Approximation and Projection (UMAP) to generate “feature maps” of images of live, dead and apoptotic cells.

Results: The bi-modal distribution of our supervised fingerprint algorithm (Figure 1a and 1b) trained on FIM images of viable cells and dead cells shows that the features extracted are reflective of cell health. Indeed, representative images (Figure 1d) of cells corresponding to the dead region (Figure 1b) were irregular, had disrupted membranes and were more translucent compared to the representative live cells (Figure 1c) that were circular, darker and the outline of the cell was in high contrast with the image background indicating intact cell membranes. These observations can be explained as visual indications of increased permeability and/or rupture of cell membrane, a well-known marker for accidental cell death.^13,16,28 A linear fit comparing our supervised machine learning fingerprint approach to traditional fluorescence based FACs analysis showed an r-squared value of 0.9. Our supervised fingerprint algorithm trained with human-generated labels for images of unstained viable, dead, and apoptotic cells was able to generate three distinct fingerprints (Figures 2b, 2c and 2d), reflecting the three distinct populations found in the Annexin V/PI flow cytometry assay (i.e., Q4, Q3 and Q2 in Figure 2a for viable, apoptotic and dead cells, respectively). When supplied with images of live, dead, and apoptotic cells along with images of debris and larger particles (i.e., images that were not size filtered) in an unlabeled fashion, the unsupervised VAE generated feature distributions were distinct and made up of clusters that corresponded to different types of easily distinguishable cell/cellular particle morphologies (Figure 3).

Conclusion: In this work, we first present a stain-free, rapid, and straightforward fingerprinting-based analytical method that uses the inherent cell morphological features extracted from a combination of FIM and supervised CNN techniques to differentiate between live or dead Jurkat cell (immortalized human T-lymphocyte cells) phenotypes. We then show that an unsupervised VAE algorithm that is trained on the images of live, dead and apoptotic cells but without human-generated labels is a powerful exploratory technique that can be used as a process monitoring analytical tool.

References: Baboo J, Kilbride P, Delahaye M, et al. The Impact of Varying Cooling and Thawing Rates on the Quality of Cryopreserved Human Peripheral Blood T Cells. Sci Rep. 2019;9(1):3417. doi:10.1038/s41598-019-39957-x
2. Hoogendoorn KH, Crommelin DJA, Jiskoot W. Formulation of Cell-Based Medicinal Products: A Question of Life or Death? JPharmSci. 2021;110(5):1885-1894. doi:10.1016/j.xphs.2020.07.002
3. Weng L. Cell Therapy Drug Product Development: Technical Considerations and Challenges. JPharmSci. 2023;0(0). doi:10.1016/j.xphs.2023.08.001
4. Jiang J, Ahuja S. Addressing Patient to Patient Variability for Autologous CAR T Therapies. Journal of Pharmaceutical Sciences. 2021;110(5):1871-1876. doi:10.1016/j.xphs.2020.12.015
5. Clarke D. Addressing Particulates, Extractables & Leachables & the Quality of Single-Use Systems for Cell & Gene Therapy Manufacturing. Cell Gene Therapy Insights. 2018;4(3):187-196. doi:10.18609/cgti.2018.020
6. Jere D, Sediq AS, Huwyler J, Vollrath I, Kardorff M, Mahler HC. Challenges for Cell-Based Medicinal Products From a Pharmaceutical Product Perspective. Journal of Pharmaceutical Sciences. 2021;110(5):1900-1908. doi:10.1016/j.xphs.2020.11.040
7. Guideline on quality, non-clinical and clinical aspects of medicinal products containing genetically modified cells.
8. Campbell A, Brieva T, Raviv L, et al. Concise Review: Process Development Considerations for Cell Therapy. Stem Cells Translational Medicine. 2015;4(10):1155-1163. doi:10.5966/sctm.2014-0294
9. Bak A, Friis KP, Wu Y, Ho RJY. Translating Cell and Gene Biopharmaceutical Products for Health and Market Impact. Product Scaling From Clinical to Marketplace: Lessons Learned and Future Outlook. Journal of Pharmaceutical Sciences. 2019;108(10):3169-3175. doi:10.1016/j.xphs.2019.05.027
10. Sharpe M, Barry J, Kefalas P. Clinical Adoption of Advanced Therapies: Challenges and Opportunities. Journal of Pharmaceutical Sciences. 2021;110(5):1877-1884. doi:10.1016/j.xphs.2020.08.027
11. De Grandis G, Brass I, Farid SS. Is regulatory innovation fit for purpose? A case study of adaptive regulation for advanced biotherapeutics. Regulation & Governance. 2023;17(3):810-832. doi:10.1111/rego.12496
12. Eyles JE, Vessillier S, Jones A, Stacey G, Schneider CK, Price J. Cell therapy products: focus on issues with manufacturing and quality control of chimeric antigen receptor T‐cell therapies. J Chem Technol Biotechnol. 2019;94(4):1008-1016. doi:10.1002/jctb.5829
13. Fink SL, Cookson BT. Apoptosis, Pyroptosis, and Necrosis: Mechanistic Description of Dead and Dying Eukaryotic Cells. Infect Immun. 2005;73(4):1907-1916. doi:10.1128/IAI.73.4.1907-1916.2005
14. Galluzzi L, Vitale I, Aaronson SA, et al. Molecular mechanisms of cell death: recommendations of the Nomenclature Committee on Cell Death 2018. Cell Death Differ. 2018;25(3):486-541. doi:10.1038/s41418-017-0012-4
15. Ziegler U, Groscurth P. Morphological Features of Cell Death. Physiology. 2004;19(3):124-128. doi:10.1152/nips.01519.2004
16. Leist M, Nicotera P. The Shape of Cell Death. Biochemical and Biophysical Research Communications. 1997;236(1):1-9. doi:10.1006/bbrc.1997.6890
17. Chen CS, Mrksich M, Huang S, Whitesides GM, Ingber DE. Geometric control of cell life and death. Science. 1997;276(5317):1425-1428. doi:10.1126/science.276.5317.1425
18. Selig M, Azizi S, Walz K, Lauer JC, Rolauffs B, Hart ML. Cell morphology as a biological fingerprint of chondrocyte phenotype in control and inflammatory conditions. Frontiers in Immunology. 2023;14. Accessed November 20, 2023. https://www.frontiersin.org/articles/10.3389/fimmu.2023.1102912
19. Morphological and Biochemical Aspects of Apoptosis, Oncosis and Necrosis - Van Cruchten - 2002 - Anatomia, Histologia, Embryologia - Wiley Online Library. Accessed October 17, 2023. https://onlinelibrary.wiley.com/doi/full/10.1046/j.1439-0264.2002.00398.x
20. Thite NG, Ghazvini S, Wallace N, Feldman N, Calderon CP, Randolph TW. Machine learning analysis provides insight into mechanisms of protein particle formation inside containers during mechanical agitation. Journal of Pharmaceutical Sciences. Published online July 11, 2022. doi:10.1016/j.xphs.2022.06.017
21. Greenblott DN, Zhang J, Calderon CP, Randolph TW. Machine learning approaches to root cause analysis, characterization, and monitoring of subvisible particles in monoclonal antibody formulations. Biotechnology and Bioengineering. n/a(n/a). doi:https://doi.org/10.1002/bit.28239
22. Calderon CP, Ripple DC, Srinivasan C, et al. Testing Precision Limits of Neural Network-Based Quality Control Metrics in High-Throughput Digital Microscopy. Pharm Res. 2022;39(2):263-279. doi:10.1007/s11095-021-03130-9
23. Daniels AL, Calderon CP, Randolph TW. Machine learning and statistical analyses for extracting and characterizing “fingerprints” of antibody aggregation at container interfaces from flow microscopy images. Biotechnology and Bioengineering. 2020;117(11):3322-3335. doi:10.1002/bit.27501
24. Grabarek AD, Senel E, Menzen T, et al. Particulate impurities in cell-based medicinal products traced by flow imaging microscopy combined with deep learning for image analysis. Cytotherapy. 2021;23(4):339-347. doi:10.1016/j.jcyt.2020.04.093
25. Adkins I, Sadilkova L, Hradilova N, Tomala J, Kovar M, Spisek R. Severe, but not mild heat-shock treatment induces immunogenic cell death in cancer cells. OncoImmunology. 2017;6(5):e1311433. doi:10.1080/2162402X.2017.1311433
26. Richter K, Haslbeck M, Buchner J. The Heat Shock Response: Life on the Verge of Death. Molecular Cell. 2010;40(2):253-266. doi:10.1016/j.molcel.2010.10.006
27. Velichko AK, Markova EN, Petrova NV, Razin SV, Kantidze OL. Mechanisms of heat shock response in mammals. Cell Mol Life Sci. 2013;70(22):4229-4241. doi:10.1007/s00018-013-1348-7
28. Van Cruchten S, Van den Broeck W. Morphological and Biochemical Aspects of Apoptosis, Oncosis and Necrosis. Anatomia, Histologia, Embryologia. 2002;31(4):214-223. doi:10.1046/j.1439-0264.2002.00398.x

Figure 1: Contour plots showing “fingerprints” or the estimated probability density functions of CNN embeddings for FIM images of a) unstressed Jurkat cells (i.e., control/viable cells) and b) Jurkat cells imaged after incubation in a water bath controlled at 47ºC for 30 minutes (i.e., dead cells). Representative FIM images sampled from c) viable and d) dead cell populations corresponding to the black arrow are shown. The two coordinates of the fingerprint representation are data-driven dimensions returned by the trained CNN-based algorithm and do not have a readily discernable physical interpretation.

Figure 2: a) Overlay of flow cytometry density scatter plots of Annexin V – Alexa 488 versus PI for untreated cells denoted as viable cells (grey), cells heat-shocked for 30 minutes denoted as dead cells (red), and cells treated with 10μM CPT for 5 hours at 37ºC and 5% CO2 denoted as apoptotic cells (green). Contour plots showing “fingerprints” for the CNN-based fingerprinting trained on FIM images of b) untreated Jurkat cells denoted as viable cells, c) cells heat-shocked for 30 minutes denoted as dead cells, and d) cells treated with 10μM CPT for 5 hours at 37ºC and 5% CO2 denoted as apoptotic cells. The 40 closest (in terms of Euclidean distance) FIM images to the region indicated by the black arrow corresponding to e) viable, f) dead and g) apoptotic cell populations.

Figure 3: Contour plots showing the feature distribution obtained using UMAP for the VAE model trained on FIM images of a) untreated cells denoted as viable cells, b) cells incubated in a heated water bath for 30 minutes denoted as dead cells, and cells treated with 10μM Camptothecin for 5 hours at 37ºC and 5% CO2 denoted as apoptotic cells.