Advertisement

Artificial Intelligence in Pathology

  • Stanley Cohen
    Correspondence
    Address correspondence to Stanley Cohen, M.D., Perelman School of Medicine, University of Pennsylvania, 295a John Morgan Bldg., Philadelphia, PA 19104-6112.
    Affiliations
    Department of Pathology, Rutgers New Jersey Medical School, Newark, New Jersey

    Department of Pathology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania

    Department of Pathology, Kimmel Medical College, Jefferson University, Philadelphia, Pennsylvania
    Search for articles by this author
  • Richard Levenson
    Affiliations
    Department of Pathology and Laboratory Medicine, UC Davis Health, Sacramento, California
    Search for articles by this author
  • Liron Pantanowitz
    Affiliations
    Department of Pathology, University of Michigan, Ann Arbor, Michigan
    Search for articles by this author
      About two and a half years ago, The American Journal of Pathology (AJP) introduced a new publication topic category, “Machine Learning, Computational Pathology, and Biophysical Imaging.”
      • Cohen S.
      • Furie M.B.
      Artificial intelligence and pathobiology join forces in The American Journal of Pathology.
      Since then, over a dozen articles have been published in this topic category in the AJP and there has been a steady influx of submissions in this topic category.
      Machine learning and, more recently, deep learning [in its new guise as artificial intelligence (AI)] have come to play increasingly large roles in all aspects of life, including education, finance, law, engineering, science, the humanities, and even biblical textual analysis, and this multi-billion–dollar industry continues to grow. In pathology, AI promises a paradigm shift at least as consequential as was the introduction of molecular biology (and before that, fluorescent antibodies). It is important to remember that the AJP was one of the first pathology journals to welcome molecular-based articles in its pages, and the free-standing American Society for Investigative Pathology (ASIP) publication, The Journal of Molecular Diagnostics, evolved from this starting point. It is this analogy that prompted the introduction of the new machine learning–oriented section for AJP.
      Although the AJP has always been focused on mechanistic studies in pathobiology, the nature of the research enterprise has evolved. Rather than requiring a reductionist, purely hypothesis-driven approach, it is now possible to generate large amounts of multiplexed data in a single experiment, deploying any of the various omics-based strategies under active development. In addition, clinical practice and the need for data archiving have resulted in vast data banks that have become increasingly difficult for unaided humans to explore; these almost certainly contain correlations (both valuable and spurious), insights, and predictions. Therefore, rather than taking a back seat to hypothesis-testing research, observational studies, when properly performed and interpreted, can now be defined as hypothesis-generating research. Machine learning allows us to take full advantage of this reciprocal interaction between observation and hypothesis to the same extent as is possible in the so-called hard sciences such as astronomy, physics, chemistry, cell biology, and so on.
      Over and above the research environment, the utility of AI has become obvious in diagnostic settings, most clearly in radiology, oncology, and, more recently, infectious disease. Pathology has been slow to embrace this field in comparison to radiology, and so we are still in the early stages of this revolution. It is important to recognize that AI can now address all aspects of pathology, from clinical diagnosis to translational research to basic research, and the arrows linking them go in both directions. Currently, however, AI-enhanced pathology is being developed and implemented at major academic centers and has not reached the stage of deployment across the pathology community at large. This is due to the relative paucity of pathologists with the appropriate background; the lack of didactic training programs for house staff, fellows, and junior faculty, the large investments required for a digital infrastructure; the relatively small-scale proprietary ventures that exist in silos; lack of guidelines for best practice; and, last but not the least, inertia. In hindsight, this is not only similar to the early development of molecular pathology as described above, but also to the adoption of digital pathology, and use of fluorescent antibodies decades before that. Fortunately, in the case of AI there are already many examples showing how to overcome these obstacles. This theme issue, through a series of Mini-Reviews and a Review, is designed to highlight progress ongoing in this field.
      In order to approach this topic successfully, it is helpful to provide a brief introduction to the terms and concepts involved. A good starting point is to recognize that the term artificial intelligence is an oxymoron. As Edsgar Dijkstra has pointed out informally, the question of whether a computer can think is no more interesting than the question of whether a submarine can swim. General intelligence appears to be an emergent property of approximately three pounds of a semi-solid viscid mass containing many billions of neurons interconnected via trillions of connections. Even the deepest of the deep learning algorithms we employ do not approach that level of complexity and are unlikely to do so without radical new strategies embracing both neuromorphic and quantum computing, especially under the additional constraints of operating at room temperature and requiring only about 20 W (as compared to the kilowatt and mega-kilowatt requirement of current computing platforms).
      Although machines at current levels of technology cannot actually think, they can still learn. Intelligence involves understanding, reasoning, and planning. In contrast, learning is simply about improving performance through experience. In all forms of machine learning the basic building block is classification; from the ability to identify and distinguish members of one class from another, we can derive predictions about outcomes, associations, and correlations, and are well on our way to develop models for causal inference. For many years, classic machine learning has been based on three strategies: geometric (how close classes are in a multidimensional attribute space), probabilistic (Bayesian models), and stratification models (decision trees and random forests). However, almost from the beginning, semi-realistic analogs of brain-like behavior were proposed, such as the perceptron model. The perceptron has its input and output linked through a layer of interconnected software representation of neurons. Unlike biological neurons, signal strength is determined by summing signal amplitudes rather than spike frequency of biological neurons. This makes it possible to perform calculations using linear algebra rather than higher level calculus. This perceptron model evolved into current deep learning strategies that are the basis of most of what we have come to think of as AI. In these, an artificial network is constructed of artificial neurons arranged in layers. Each neuron in a layer connects to most or all of the neurons in the next layer. However, there are no connections between neurons of the same level. As inputs are processed up through the various layers, higher levels of abstraction are created until the final output is an accurate representation of an input class, based upon training with a large number of similar but not identical members of each class. The techniques that are used to implement neural network function are back propagation for error correction and convolution to efficiently process large, interconnected arrays such as image pixels. Error minimization, reinforcement learning, and even game theoretic approaches are used to train such networks, but all require daunting volumes of annotated data. It takes a toddler only a few examples to distinguish between a banana and an apple, but it would take hundreds to thousands of examples for a computer to accomplish such a simple task. Much research is being done to try to overcome, or at least minimize, this constraint.
      The Review and Mini-Reviews that comprise this special theme issue are designed to focus on these various aspects of AI in pathology, beginning with an overview of the practical challenges facing implementation,
      • Cheng J.Y.
      • Abel J.T.
      • Balis U.G.J.
      • McClintock D.S.
      • Pantanowitz L.
      Challenges in the development, deployment, and regulation of artificial intelligence in anatomic pathology.
      and including a discussion of ethical considerations that arise in this field.
      • Chauhan C.
      • Gullapali R.
      Ethics of AI in pathology: current paradigms and emerging issues.
      The evolution of AI in pathology is illuminated by earlier but analogous trends in radiology.
      • Durkee M.S.
      • Abraham R.
      • Clark M.R.
      • Giger M.L.
      Artificial intelligence and cellular segmentation in tissue microscopy images.
      Analogies with the use of AI in reduction of interobserver variability are considered,
      • Tizhoosh H.R.
      • Diamandis P.
      • Campbell C.J.V.
      • Safarpoor A.
      • Kalra S.
      • Maleki S.
      • Riasatian A.
      • Babaie M.
      Searching images for consensus: can AI remove observer variability in pathology?.
      as are current strategies for reducing the need for massive annotation in machine learning through either the use of existing supervised frameworks or by exploiting hybrid models using unsupervised learning, generative models, and/or synthetic data.
      • Mitchell B.R.
      • Cohen M.C.
      • Cohen S.
      Dealing with multi-dimensional data and the burden of annotation.
      Because of the rapid emergence of generative algorithms, a separate Review is included on generative deep learning in the pathology environment.
      • Morrison D.
      • Harris-Birtill D.
      • Caie P.D.
      Generative deep learning in digital pathology workflows.
      Special emphasis on segmentation strategies is included, because much of pathology data is in the form of image-based content that is spatially heterogenous, as well as ways of deducing the relevance of cellular interactions in this context. This is discussed using tumor-stromal interfaces and tumor heterogeneity as examples.
      • Laurinavisius A.
      • Rasmusson A.
      • Plancoulaine B.
      • Shirbak M.
      • Levenson R.
      Machine-learning–based evaluation of intratumoral heterogeneity and tumor-stroma interface for clinical guidance.
      While there is some overlap, the authors bring their collective insights and perspectives to the overall goal of optimizing digital and computational pathology by partnering with AI tools. Hopefully, the central message that emerges is that AI is neither master nor servant, but, rather, can be a dedicated and tireless pathologist's assistant. Perhaps, in the future, the computer will evolve into a true colleague, but only after General AI has emerged. The Turing Test for this event may well be the achievement of artificial tenure.

      References

        • Cohen S.
        • Furie M.B.
        Artificial intelligence and pathobiology join forces in The American Journal of Pathology.
        Am J Pathol. 2019; 189: 4-5
        • Cheng J.Y.
        • Abel J.T.
        • Balis U.G.J.
        • McClintock D.S.
        • Pantanowitz L.
        Challenges in the development, deployment, and regulation of artificial intelligence in anatomic pathology.
        Am J Pathol. 2021; 191: 1684-1692
        • Chauhan C.
        • Gullapali R.
        Ethics of AI in pathology: current paradigms and emerging issues.
        Am J Pathol. 2021; 191: 1673-1683
        • Durkee M.S.
        • Abraham R.
        • Clark M.R.
        • Giger M.L.
        Artificial intelligence and cellular segmentation in tissue microscopy images.
        Am J Pathol. 2021; 191: 1693-1701
        • Tizhoosh H.R.
        • Diamandis P.
        • Campbell C.J.V.
        • Safarpoor A.
        • Kalra S.
        • Maleki S.
        • Riasatian A.
        • Babaie M.
        Searching images for consensus: can AI remove observer variability in pathology?.
        Am J Pathol. 2021; 191: 1702-1708
        • Mitchell B.R.
        • Cohen M.C.
        • Cohen S.
        Dealing with multi-dimensional data and the burden of annotation.
        Am J Pathol. 2021; 191: 1709-1716
        • Morrison D.
        • Harris-Birtill D.
        • Caie P.D.
        Generative deep learning in digital pathology workflows.
        Am J Pathol. 2021; 191: 1717-1723
        • Laurinavisius A.
        • Rasmusson A.
        • Plancoulaine B.
        • Shirbak M.
        • Levenson R.
        Machine-learning–based evaluation of intratumoral heterogeneity and tumor-stroma interface for clinical guidance.
        Am J Pathol. 2021; 191: 1724-1731