Advertisement

Deep Learning Analysis of Histologic Images from Intestinal Specimen Reveals Adipocyte Shrinkage and Mast Cell Infiltration to Predict Postoperative Crohn Disease

      Most patients with Crohn disease (CD), a chronic inflammatory gastrointestinal disease, experience recurrence despite treatment, including surgical resection. However, methods for predicting recurrence remain unclear. This study aimed to predict postoperative recurrence of CD by computational analysis of histopathologic images and to extract histologic characteristics associated with recurrence. A total of 68 patients who underwent surgical resection of the intestine were included in this study and were categorized into two groups according to the presence or absence of postoperative disease recurrence within 2 years after surgery. Recurrence was defined using the CD Activity Index and the Rutgeerts score. Whole-slide images of surgical specimens were analyzed using deep learning model EfficientNet-b5, which achieved a highly accurate prediction of recurrence (area under the curve, 0.995). Moreover, subserosal tissue images with adipose cells enabled highly accurate prediction. Adipose cell morphology showed significant between-group differences in adipose cell size, cell-to-cell distance, and cell flattening values. These findings suggest that adipocyte shrinkage is an important histologic characteristic associated with recurrence. Moreover, there was a significant between-group difference in the degree of mast cell infiltration in the subserosa. These findings show the importance of mesenteric adipose tissue in patient prognosis and CD pathophysiology. These findings also suggest that deep learning–based artificial intelligence enables the extraction of meaningful histologic features.
      Disease prognostication is an important aspect of clinical practice. Clinicians have attempted to integrate different types of clinical information to predict disease outcomes, and various prediction models have been designed for diverse disease conditions. In malignant neoplasms, for example, TNM histopathologic classification is an efficient prognostic model.
      • O'Sullivan B.
      • Brierley J.
      • Byrd D.
      • Bosman F.
      • Kehoe S.
      • Kossary C.
      • Piñeros M.
      • Van Eycken E.
      • Weir H.K.
      • Gospodarowicz M.
      The TNM classification of malignant tumours—towards common understanding and reasonable expectations.
      In chronic diseases, such as immune-related conditions, prediction of symptom transition and long-term outcomes is important; these aspects influence the quality of life of affected patients. It is important to use information yielded by new modalities to predict disease prognosis.
      Crohn disease (CD) is a chronic inflammatory disease that affects the gastrointestinal tract.
      • Mills S.C.
      • von Roon A.C.
      • Tekkis P.P.
      • Orchard T.R.
      Crohn's disease.
      ,
      • Ogino T.
      • Mizushima T.
      • Matsuda C.
      • Mori M.
      • Doki Y.
      Essential updates 2018/2019: colorectal (benign): recent updates (2018-2019) in the surgical treatment of benign colorectal diseases.
      While inflammatory changes may occur at any segment of the gastrointestinal tract, from the mouth to the anus; they typically involve the lower gastrointestinal tract.
      • Veauthier B.
      • Hornecker J.R.
      Crohn's disease: diagnosis and management.
      The disease may manifest from early childhood to late adulthood, but >80% of patients with CD are diagnosed before the age of 40.
      • Mills S.C.
      • von Roon A.C.
      • Tekkis P.P.
      • Orchard T.R.
      Crohn's disease.
      Treatment for CD is primarily based on combinations of nutritional and drug therapies, such as corticosteroids, immunosuppressants, and biologicals.
      • Veauthier B.
      • Hornecker J.R.
      Crohn's disease: diagnosis and management.
      Remission can be induced by anti–tumor necrosis factor biological agents, such as infliximab and adalimumab, which are available in many countries, including Japan.
      • Targan S.R.
      • Hanauer S.B.
      • van Deventer S.J.H.
      • Mayer L.
      • Present D.H.
      • Braakman T.
      • DeWoody K.L.
      • Schaible T.F.
      • Rutgeerts P.J.
      A short-term study of chimeric monoclonal antibody cA2 to tumor necrosis factor α for Crohn's disease.
      ,
      • Colombel J.F.
      • Sandborn W.J.
      • Rutgeerts P.
      • Enns R.
      • Hanauer S.B.
      • Panaccione R.
      • Schreiber S.
      • Byczkowski D.
      • Li J.
      • Kent J.D.
      • Pollack P.F.
      Adalimumab for maintenance of clinical response and remission in patients with Crohn's disease: the CHARM trial.
      Novel drugs, including anti–IL-12/23p40 and anti–integrin α4β7, have recently become available.
      • Sabino J.
      • Verstockt B.
      • Vermeire S.
      • Ferrante M.
      New biologics and small molecules in inflammatory bowel disease: an update.
      However, CD is a progressive condition that leads to the formation of strictures, fistulas, and abscesses; approximately 71% of patients require surgical resection of the intestine within 10 years after diagnosis.
      • Bernell O.
      • Lapidus A.
      • Hellers G.
      Risk factors for surgery and postoperative recurrence in Crohn's disease.
      The 10-year rate of postoperative symptomatic recurrence has been estimated at 40%.
      • Bernell O.
      • Lapidus A.
      • Hellers G.
      Risk factors for surgery and recurrence in 907 patients with primary ileocaecal Crohn's disease.
      The rate of postoperative endoscopic recurrence was 85% within 3 years.
      • Burr N.E.
      • Hall B.
      • Hamlin P.J.
      • Selinger C.P.
      • Ford A.C.
      • O'Connor A.
      Systematic review and network meta-analysis of medical therapies to prevent recurrence of post-operative Crohn's disease.
      Numerous scoring systems have been proposed to evaluate CD activity. The CD Activity Index (CDAI) is used to evaluate disease activity based on clinical symptoms and physical findings.
      • Best W.R.
      • Becktel J.M.
      • Singleton J.W.
      • Kern Jr., F.
      • Kern F.
      Development of a Crohn's disease activity index: National Cooperative Crohn's Disease Study.
      CDAI is calculated on the basis of a combination of eight variables, such as number of liquid stools in 7 days, presence of extraintestinal complications, presence of an abdominal mass, need for use of antidiarrheal drugs, body weight, hematocrit levels, general well-being, and presence of abdominal pain in the previous week. CDAI ranges from 0 to 600 points, with ≤150 points defined as remission, 150 to 219 points defined as mild activity, 220 to 450 points defined as moderate activity, and ≥450 points defined as severe activity. The Rutgeerts score
      • Rutgeerts P.
      • Geboes K.
      • Vantrappen G.
      • Beyls J.
      • Kerremans R.
      • Hiele M.
      Predictability of the postoperative course of Crohn's disease.
      is a common endoscopic scoring system and useful surrogate marker of postoperative recurrence, and it is based on the number of aphthous lesions at the anastomosis site, as well as the presence of diffuse inflammation or ulcers observed by endoscopy. The score ranges from i0 to i4, depending on endoscopic findings after surgery, and i2 to i4 are defined as postoperative recurrence. In contrast, no scoring system can predict the recurrence of CD.
      • Novak G.
      • Parker C.E.
      • Pai R.K.
      • Macdonald J.K.
      • Feagan B.G.
      • Sandborn W.J.
      • D'Haens G.
      • Jairath V.
      • Khanna R.
      Histologic scoring indices for evaluation of disease activity in Crohn's disease.
      Several risk factors (eg, smoking,
      • Reese G.E.
      • Nanidis T.
      • Borysiewicz C.
      • Yamamoto T.
      • Orchard T.
      • Tekkis P.P.
      The effect of smoking after surgery for Crohn's disease: a meta-analysis of observational studies.
      extent of disease,
      • Fornaro R.
      • Caratto E.
      • Caratto M.
      • Fornaro F.
      • Caristo G.
      • Frascio M.
      • Sticchi C.
      Post-operative recurrence in Crohn's disease: critical analysis of potential risk factors: an update.
      and visceral adiposity
      • Holt D.Q.
      • Moore G.T.
      • Strauss B.J.G.
      • Hamilton A.L.
      • De Cruz P.
      • Kamm M.A.
      Visceral adiposity predicts post-operative Crohn's disease recurrence.
      ) for postoperative recurrence of CD and fecal calprotectin findings have been reported to sensitively predict endoscopic recurrence.
      • Wright E.K.
      • Kamm M.A.
      • De Cruz P.
      • Hamilton A.L.
      • Ritchie K.J.
      • Krejany E.O.
      • Leach S.
      • Gorelik A.
      • Liew D.
      • Prideaux L.
      • Lawrance I.C.
      • Andrews J.M.
      • Bampton P.A.
      • Jakobovits S.L.
      • Florin T.H.
      • Gibson P.R.
      • Debinski H.
      • Macrae F.A.
      • Samuel D.
      • Kronborg I.
      • Radford-Smith G.
      • Selby W.
      • Johnston M.J.
      • Woods R.
      • Elliott P.R.
      • Bell S.J.
      • Brown S.J.
      • Connell W.R.
      • Day A.S.
      • Desmond P.V.
      • Gearry R.B.
      Measurement of fecal calprotectin improves monitoring and detection of recurrence of Crohn's disease after surgery.
      Histopathologically, CD is characterized by transmural inflammation with lymphoid aggregation, submucosal thickening, fissure formation, and sarcoid granuloma formation.
      • Stange E.F.
      • Travis S.P.L.
      • Vermeire S.
      • Beglinger C.
      • Kupcinskas L.
      • Geboes K.
      • Barakauskiene A.
      • Villanacci V.
      • Von Herbay A.
      • Warren B.F.
      • Gasche C.
      • Tilg H.
      • Schreiber S.W.
      • Schölmerich J.
      • Reinisch W.
      European evidence based consensus on the diagnosis and management of Crohn's disease: definitions and diagnosis.
      Pathologists typically report the severity, pattern, and characteristic features of inflammation when diagnosing biopsy samples and surgical specimens from patients with CD. However, no studies have established histologic criteria to evaluate disease activity or predict postoperative recurrence of CD,
      • Novak G.
      • Parker C.E.
      • Pai R.K.
      • Macdonald J.K.
      • Feagan B.G.
      • Sandborn W.J.
      • D'Haens G.
      • Jairath V.
      • Khanna R.
      Histologic scoring indices for evaluation of disease activity in Crohn's disease.
      with the exception of a few pioneering studies based on mesenteric granulomas
      • Unger L.W.
      • Argeny S.
      • Stift A.
      • Yang Y.
      • Karall A.
      • Freilinger T.
      • Müller C.
      • Bergmann M.
      • Stift J.
      • Riss S.
      Mesenteric granulomas independently predict long-term risk of surgical recurrence in Crohn's disease.
      and plexitis.
      • Nakao S.
      • Itabashi M.
      • Yamamoto T.
      • Okamoto T.
      Predictive value of myenteric and submucosal plexitis for postoperative Crohn's disease recurrence.
      • Ferrante M.
      • de Hertogh G.
      • Hlavaty T.
      • D'Haens G.
      • Penninckx F.
      • D'Hoore A.
      • Vermeire S.
      • Rutgeerts P.
      • Geboes K.
      • van Assche G.
      The value of myenteric plexitis to predict early postoperative Crohn's disease recurrence.
      • Lemmens B.
      • de Buck van Overstraeten A.
      • Arijs I.
      • Sagaert X.
      • Van Assche G.
      • Vermeire S.
      • Tertychnyy A.
      • Geboes K.
      • Wolthuis A.
      • D'Hoore A.
      • De Hertogh G.
      • Ferrante M.
      Submucosal plexitis as a predictive factor for postoperative endoscopic recurrence in patients with Crohn's disease undergoing a resection with ileocolonic anastomosis: results from a prospective single-centre study.
      Histologic findings of CD may enable the prediction of recurrence, and histologic analysis of specimens from patients with CD is important for prognostication and management.
      The application of artificial intelligence to histopathology data has progressed markedly and has entered both research and clinical practice.
      • Acs B.
      • Rantalainen M.
      • Hartman J.
      Artificial intelligence as the next step towards precision pathology.
      In inflammatory bowel disease as well, past studies have applied artificial intelligence to diagnose the disease and to evaluate the risk and therapeutic response based on clinical information, genetics, and endoscopic findings.
      • Gubatan J.
      • Levitte S.
      • Patel A.
      • Balabanis T.
      • Wei M.T.
      • Sinha S.R.
      Artificial intelligence applications in inflammatory bowel disease: emerging technologies and future directions.
      In the field of histopathology, convolutional neural network algorithms are used in image recognition and can accurately classify histologic images.
      • Wang S.
      • Yang D.M.
      • Rong R.
      • Zhan X.
      • Xiao G.
      Pathology image analysis using segmentation deep learning algorithms.
      The current study aimed to perform histopathologic analysis of surgical specimens from patients with CD using a deep learning–based artificial intelligence algorithm and attempted to predict postoperative recurrence. This study also aimed to identify histologic characteristics that enable accurate prediction of recurrence.

      Materials and Methods

      Patients

      This retrospective review involved patients with CD who underwent bowel resection between January 2007 and July 2018 at Osaka University Hospital (Osaka, Japan). Of 117 patients who underwent bowel resection, patients with malignant complications, patients lacking sufficient clinical follow-up information, and patients lacking surgical specimens available at the institution were excluded. The study eligibility criteria and patient flow are presented in Figure 1. Data from a total of 68 patients were included in the analysis. Postoperative recurrence was defined as a CDAI of ≥220 points and/or a Rutgeerts score of ≥i2. The patients were divided into two groups according to the presence or absence of postoperative recurrence within 2 years after surgery. All experiments were performed in accordance with the relevant institutional guidelines and regulations. The Osaka University Graduate School of Medicine Institutional Review Board approved the study protocol on January 6, 2020 (number 19325).
      Figure thumbnail gr1
      Figure 1Study eligibility criteria and patient flow. A total of 68 patients were included in the analysis.

      Histologic Images

      All the glass slides of surgical specimens prepared by pathologists at the time of surgery were collected and stained with hematoxylin and eosin (n = 550). The specimens were then scanned using a whole-slide imaging scanner (Hamamatsu NanoZoomer 2.0HT; Hamamatsu Photonics K.K., Hamamatsu, Japan) with a 20× objective lens. The resolution of whole-slide images was 0.46 μm per pixel. The acquired images were then stored on a computer.
      The whole-slide images were manipulated for analysis using OpenSlide Python 1.1.2,
      • Goode A.
      • Gilbert B.
      • Harkes J.
      • Jukic D.
      • Satyanarayanan M.
      OpenSlide: a vendor-neutral software foundation for digital pathology.
      and analyzed using Python 3.7.10, Pytorch 1.7.0,
      • Paszke A.
      • Gross S.
      • Chintala S.
      • Chanan G.
      • Yang E.
      • DeVito Z.
      • Lin Z.
      • Desmaison A.
      • Antiga L.
      • Lerer A.
      Automatic differentiation in PyTorch.
      and OpenCV-Python 4.2.0. All software used for this analysis and its versions are listed in Supplemental Table S1. After obtaining whole-slide images of all samples, the images with 1/8 resolution of the original images were used for computational analysis. The images were cropped into 256 × 256-pixel tile images without overlapping. In this cropping process, we excluded empty images that did not contain any tissue area by examining the hue saturation value color space of each image. Specifically, the saturation value of each image was extracted, background noise was removed by using the closing function of OpenCV [cv2.morphologyEx (img, cv2.MORPH_CLOSE, kernel = 7)], and it was binarized with a threshold of 10. The area obtained by this process was considered the tissue area; nonoverlapping 256 × 256-pixel tile images were obtained, so that the tissue area occupied ≥35% of each tile. Finally, each 256 × 256-pixel image was divided into four nonoverlapping 128 × 128-pixel tiles for computational analysis (Figure 2A). In total, 619,464 tile images were obtained for the analysis. The tile images from the nonrecurrence and recurrence groups were labeled as 0 and 1, respectively.
      Figure thumbnail gr2
      Figure 2Overview of image-data collection and the computational analysis framework. A: Data collection. Whole-slide images were obtained from surgical intestine specimens, empty images were excluded, and the resulting images were cropped into 128 × 128 tiles. Tile images were augmented for the training data set. B: Patients were randomly divided into training/validation and test groups, such that the number of tile images was similar in both groups. The test/validation group was subdivided into five subgroups. C: In the training/validation phase, image data used were from four of the five training subgroups. The remaining image data were used for validation. Each group was used for validation, and five machine learning models were generated. D: In the test phase, five models were used to calculate the predictive values five times for each tile image in the test data sets. Finally, mean of the five predictive values obtained from each model was used.
      Python programs that implemented a stratified group k-fold model to subgroup cases were used for machine learning. This program is almost the same algorithm as StratifiedGroupKFold function currently implemented in the scikit-learn module (https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedGroupKFold.html, last accessed January 2022). The patients in each group were automatically sorted into two subgroups, training/validation data and test data, to ensure that the patients did not cross subgroups; the two subgroups contained approximately the same numbers of tile images. Training/validation data were divided into five subgroups, such that data from each patient did not cross subgroups, and cross-validation was performed using each subgroup (Figure 2B).
      The EfficientNet-b5
      • Tan M.
      • Le Q.V.
      EfficientNet: rethinking model scaling for convolutional neural networks.
      model was pretrained by noisy student Training
      • Xie Q.
      • Luong M.T.
      • Hovy E.
      • Le Q.V.
      Self-training with noisy student improves imagenet classification.
      using the ImageNet data set. Augmix
      • Hendrycks D.
      • Mu N.
      • Cubuk E.D.
      • Zoph B.
      • Gilmer J.
      • Lakshminarayanan B.
      AugMix: a simple data processing method to improve robustness and uncertainty.
      (a method that mixes multiple augmentation methods) and the GeM pooling
      • Radenovic F.
      • Tolias G.
      • Chum O.
      Fine-tuning CNN image retrieval with no human annotation.
      methods were added to the model. The model was designed to output values between 0 (nonrecurrence) and 1 (recurrence). For machine learning, the Adam optimizer was used at the learning rate to 0.001. CosineAnnealingLR was used for the scheduler, and eta_min was set to 1 × 10−5. The distribution of predictive values in each slide image was visualized by reconstructing a heat map. In addition, the cutoff value was set so that the Youden index (sensitivity + specificity – 1) was maximized, and a confusion matrix was generated based on its value.

      Image Analysis of Adipose Cell Morphology

      From the tile images in the test data set, 4000 correctly sorted tile images with prediction values close to 0 (nonrecurrence) and 4000 tile images with prediction values close to 1 (recurrence) were extracted. As this study involved an analysis based on adipose cell morphology, tiles were manually selected in which tissue occupied >75% of the image area, and adipose cells occupied >50% of the image area. Image analysis was performed using OpenCV-Python and segmented fat droplets of adipose cells using the cv2.watershed function. Specifically, a grayscale image of each selected tile image was obtained and binarized with the average value of each pixel value as the threshold. Then, the binarized image was dilated with a 2 × 2 kernel using the cv2.dilate function to obtain the region most likely to be background. In addition, the cv2.distanceTransform function was used to calculate the distance of each pixel from the certain background area; based on these distance values, the region most likely to be a fat droplet was extracted. Finally, the regions that could not be classified as either the background or fat droplet area were segmented using the cv2.watershed function. Of the fat droplet regions obtained by segmentation, the regions of <500 pixels were considered noise, and those that occupied >3% of each tile were considered blank areas other than fat droplets; these regions were excluded from the analysis. Because fat droplets occupy most of the adipose cell area, the morphology of adipose cells was substituted with that of fat droplets. Next, the area and flattening of adipose cells, distance between cell centers, and stromal area values in each image were calculated. Flattening (f) was defined using the lengths of the major axis (a) and minor axis (b) as follows: f = (a − b) ÷ a.
      Stromal area was defined by tissue area, excluding adipocytes and background. Subsequently, the tile images used in the analysis were divided into cases, and the median value of each morphologic feature of the adipocytes was obtained for each sample. Cases in which ≥10 tiles were included in the image data set were used for the correlation analysis. The correlation between the median morphologic feature value and the Rutgeerts score of each case was analyzed.

      Histologic Quantitative Analysis of Inflammatory Cells in Subserosal Tissue

      Of the 68 cases analyzed using artificial intelligence, 67 were included in the analysis of inflammatory cells, because tissue blocks of one case in the nonrecurrence group were not available. One formalin-fixed, paraffin-embedded tissue block was selected for each patient in the analysis, and immunostaining was performed for CD3 (Dako, Agilent Technologies, Santa Clara, CA; monoclonal mouse anti-human; clone F7.2.38; code number M7254; 1:400) and CD68 (Dako, Agilent Technologies; monoclonal mouse anti-human; clone PG-M1; code number M0876; 1:100) for T-cell and macrophage counting, as well as toluidine blue staining for mast cell counting on each tissue block. For T cells and macrophages, three regions were selected with high positive cell density in a ×400 field of view from subserosal tissue, and the number of positive cells was counted in these regions with the assistance of the analysis software Winroof 2015 (MITANI Corp., Fukui, Japan). For mast cells, three regions with high positive cell density in a ×200 field of view were selected from subserosal tissue, and the number of cells was counted with metachromatic staining. For all stained slides, the total number of inflammatory cells was used for analysis of the three regions.

      Statistical Analysis

      For clinicopathologic information of patients and quantitative analysis of inflammatory cell infiltration, data were presented as means ± SD. The Welch t-test was used to draw statistical inferences when comparing the mean of variables. P < 0.05 was considered statistically significant. For morphologic analysis of adipose cells, data were presented as median ± median absolute deviation. The U-test was used to draw statistical inferences when comparing the median of variables. The κ coefficient was calculated to assess the accuracy of image classification by the experienced pathologists. The Spearman rank correlation coefficient was used to perform a correlation analysis of the Rutgeerts score and adipocyte morphology.

      Data/Code Availability

      The code developed for tile image cropping, model training and testing, and morphologic analysis of adipose cells can be accessed at https://github.com/abebe9849/Crohn_wsi (last accessed January 2022).

      Results

      Deep Learning Model for Prediction of Postoperative Recurrence

      Twenty-five patients were assigned to the recurrence group, whereas 43 patients were assigned to the nonrecurrence group (Figure 1). There were no significant differences in the clinicopathologic characteristics between the two groups (Table 1 and Supplemental Table S2). EfficientNet was trained and validated using 619,464 tile images obtained from whole-slide images (Figure 2A). The training/validation data were divided into five subgroups such that data from each patient did not cross groups, and five trained models were generated using each subgroup for cross-validation; finally, each model was trained for five epochs (Figure 2, A–C). The model was evaluated using the test data set, which consisted of 308,705 tile images. Predictive values were the mean of five values obtained from each trained model (Figure 2D). The receiver operating characteristic curve is shown in Figure 3A, and the area under the curve is 0.995. The confusion matrix is illustrated in Figure 3B. The accuracy, precision, recall, and F1 score values were 0.969, 0.964, 0.965, and 0.964, respectively. These results indicate that the deep learning model accurately classified tile images according to the presence or absence of disease recurrence.
      Table 1Clinicopathologic Characteristics of Patients Included in the Analysis
      CharacteristicNonrecurrence groupRecurrence groupP value
      Total, n4325
      Glass slides per patient, n7.98 ± 3.768.28 ± 3.47
      Sex, n0.61
       Male3319
       Female106
      Age at diagnosis, years28.77 ± 10.9728.68 ± 7.470.53
      Age at operation, years40.56 ± 13.6139.44 ± 7.060.66
      Body mass index, kg/m219.23 ± 2.4219.07 ± 3.170.84
      Smoking status, n
      Information on smoking status was not available for one patient in the nonrecurrence group.
      0.63
       No history2616
       Past/current history169
      Location, n0.89
       Ileal2514
       Colonic42
       Ileocolonic149
      Behavior, n0.27
       Stricturing219
       Penetrating2216
      Preoperative use of biological agent, n
       No2060.06
       Yes2319
      Preoperative serum albumin, g/dL3.52 ± 0.593.28 ± 0.550.43
      Preoperative CRP, mg/dL0.99 ± 2.251.61 ± 2.700.25
      Preoperative CDAI195.62 ± 105.01200.62 ± 115.00.71
      Surgical site, n0.58
       Ileal159
       Colonic93
       Ileocolonic1913
      Numerical values are presented as means ± SD.
      CDAI, Crohn Disease Activity Index; CRP, C-reactive protein.
      Information on smoking status was not available for one patient in the nonrecurrence group.
      Figure thumbnail gr3
      Figure 3High-accuracy prediction of disease recurrence by the learning model. Receiver operating characteristic curve (A) and confusion matrix (C) of the learning model; the area under the curve (AUC) was 0.995. Representative histologic images and heat maps from the nonrecurrence (B) and recurrence (D) groups. Each image was reconstructed by spatial tiling of the test results (left) and the original cropped images (right). Each color in the left image indicates the predictive value of the corresponding cropped image (right); the reference value is indicated at the bottom right. Black areas were excluded computationally as the background. In both groups, the machine learning model generally made accurate predictions using images of subserosal adipose tissue.
      Next, predictive value heat maps were generated to identify areas and histologic features from which the machine learning model could predict recurrence with high accuracy (Figure 3, C and D). These heat maps showed that the machine learning model yielded correct predictions, specifically, using subserosal adipose tissue images, in both recurrence and nonrecurrence patients. The model was less accurate in other areas, including in the mucosal and proper muscular layers. Therefore, images with the most accurate predictions were extracted from the test data sets of the nonrecurrence and recurrence groups. The 20 (Figure 4) and 50 (Supplemental Figure S1) images with the best predictive results in both groups all contained adipose tissue. Conversely, two experienced pathologists (M.K. and S.T.) attempted to correctly classify 100 tile images of adipose tissue, which had been correctly classified using the machine learning model. However, the classification results were extremely inaccurate compared with the machine learning model (Supplementary Table S3). The results showed that histologic images of subserosal adipose tissue can be accurately classified using the present machine learning model.
      Figure thumbnail gr4
      Figure 4Top 20 tile images of high-accuracy predictions. Twenty images with the most accurate predictions from the test data set of the nonrecurrence group (images with predictive values closest to 0; A) and the recurrence group (images with predictive values closest to 1; B). All 40 images contained adipose tissue.

      Morphology of Intestinal Subserosal Adipose Cells

      Because the present machine learning model achieved accurate predictions from images of subserosal tissue, we hypothesized that subserosal adipose cell morphologies differed between the recurrence and nonrecurrence groups. Therefore, adipocyte morphology was analyzed using tile images with predictive values close to 0 or 1. From the test data set, 4000 correctly sorted tile images with prediction values close to 0, and 4000 tile images with prediction values close to 1, were extracted. A total of 6257 tile images, which mainly contained adipose tissue, were selected for analysis (Figure 5A), and fat droplets of adipose cells were automatically segmented in each selected tile image (Figure 5B). Because a fat droplet occupies most of the adipose cell area, the morphology of adipose cells was substituted with that of fat droplets. Adipose cells in the recurrence group had a significantly smaller cell size, higher flattening, and smaller center-to-center cell distance values than adipose cells in the nonrecurrence group (Figure 5, B–E, and Table 2). The stromal area also significantly differed between the two groups (Figure 5F and Table 2). These extracted images were used to further analyze the correlation between the values of the adipocyte morphologic features and the Rutgeerts score for 18 patients. The area of the adipocytes and the center-to-center distances were significantly inversely correlated with the Rutgeerts score (Figure 5, G–J). These results suggested that adipocyte shrinkage and subsequent morphologic changes are important histologic characteristics associated with disease recurrence.
      Figure thumbnail gr5
      Figure 5Morphologic analysis of adipose cells from the top predictive images. A: Inclusion criteria for the tile images for adipose cell morphology analysis. B: Representative image of adipocyte segmentation using OpenCV watershed function. Areas of different colors (right panel) were recognized as different adipocytes. Left panel: Original cropped image of hematoxylin and eosin staining. CF: Statistical analysis of segmented adipose cell morphology yielded graphs that show center-to-center cell distances (C), cell areas (D), cell flattening (E), and stromal areas of each tile image (F). Blue histograms, nonrecurrence group; red histograms, recurrence group. Statistically significant differences between the nonrecurrence and recurrence groups were observed for all parameters. GJ: The correlation analysis between the Rutgeerts score and the center-to-center distance (G), cell areas (H), cell flattening (I), and stromal areas (J). ρ Indicates the Spearman rank correlation coefficient. n = 18 (GJ).
      Table 2Morphologic Analysis of Adipose Cells in the Top Predictive Images
      VariableNonrecurrence groupRecurrence groupP value
      nMedian ± MADnMedian ± MAD
      Cell distance22,254,624379.95 ± 190.3119,399,809316.4 ± 179.80<0.0001
      Cell area343,4793414 ± 3527.84315,5372654 ± 4143.48<0.0001
      Flattening343,4790.3726 ± 0.1433315,5370.3990 ± 0.1468<0.0001
      Stromal area2942266,715 ± 74,759.362980275,766 ± 74,846.350.00018
      MAD, median absolute deviation.

      Histologic Quantitative Analysis of Inflammatory Cells in Subserosal Tissue

      Subsequently, we hypothesized that the differences in adipocyte morphology between the two groups were associated with some degree or type of inflammatory condition in the subserosal adipose tissue. Previous reports have indicated that T cells and macrophages in adipose tissue could affect pathologic situations, such as obesity, by altering adipocyte function.
      • Nishimura S.
      • Manabe I.
      • Nagasaki M.
      • Eto K.
      • Yamashita H.
      • Ohsugi M.
      • Otsu M.
      • Hara K.
      • Ueki K.
      • Sugiura S.
      • Yoshimura K.
      • Kadowaki T.
      • Nagai R.
      CD8+ effector T cells contribute to macrophage recruitment and adipose tissue inflammation in obesity.
      • Lumeng C.N.
      • DeYoung S.M.
      • Bodzin J.L.
      • Saltiel A.R.
      Increased inflammatory properties of adipose tissue macrophages recruited during diet-induced obesity.
      • Porsche C.E.
      • Delproposto J.B.
      • Geletka L.
      • O'Rourke R.
      • Lumeng C.N.
      Obesity results in adipose tissue T cell exhaustion.
      Therefore, the degrees of T-cell and macrophage infiltration in the subserosal adipose tissue were analyzed. However, there was no significant between-group difference in the number of T cells or macrophages infiltrating the subserosal tissue (Figure 6, A–H, O, and P , and Table 3). Mast cell infiltration in the subserosal tissue was examined next. Because mast cells are difficult to recognize morphologically by hematoxylin and eosin staining, toluidine blue staining was used. The recurrence group had a significantly higher number of mast cells infiltrating the subserosal adipose tissue (Figure 6, I–N and Q, and Table 3). This finding indicated that mast cells in subserosal adipose tissue are associated with the recurrence of CD and the adipocyte shrinkage phenomenon.
      Figure thumbnail gr6
      Figure 6Histologic quantitative analysis of inflammatory cell infiltration in subserosal tissue. AH: Representative images of tissue staining of immune cells. For immunostaining of CD3 (AD) and CD68 (EH), three 278-μm square regions with high positive cell densities (B, D, F, and H) were selected and the number of positive cells were counted. IN: For toluidine blue staining, three 557-μm square regions with high cell density (J and L) showing metachromasia (M and N) were selected and the number of mast cells were counted. Each red circle indicates a mast cell. B, D, F, H, J, and L: Magnified images of the red square area in A, C, E, G, I, and K, respectively. O and P: Statistical analysis of inflammatory cell numbers (counted as described) revealed no significant difference between the nonrecurrence and recurrence groups for T cells (O) and macrophages (P). Q: In contrast, significantly more mast cell infiltration was observed in the subserosal tissue in the recurrence group. ∗∗P < 0.01. Scale bar = 2 mm (A, C, E, G, I, and K).
      Table 3Quantitative Analysis of Inflammatory Cell Infiltration Markers in the Subserosal Tissue
      VariableNonrecurrence

      group
      Recurrence

      group
      P

      value
      n4225
      CD3+ T cell1168.0 ± 632.21220.5 ± 712.40.7627
      CD68+ macrophage412.7 ± 168.3398.7 ± 206.10.7743
      Mast cell66.7 ± 31.192.1 ± 33.60.0035
      Numerical values are presented as means ± SD. For all stainings, the total numbers of inflammatory cells in the three fields were analyzed.

      Discussion

      The present model involving deep learning–based computational analysis of histologic images predicts postoperative recurrence of CD with high accuracy. The advances in digital pathology technology have led to an increased interest in the adaptation of deep learning–based artificial intelligence. In tumor pathology, machine learning technologies are applied for routine diagnosis
      • Acs B.
      • Rantalainen M.
      • Hartman J.
      Artificial intelligence as the next step towards precision pathology.
      and identification of genetic mutations.
      • Coudray N.
      • Ocampo P.S.
      • Sakellaropoulos T.
      • Narula N.
      • Snuderl M.
      • Fenyö D.
      • Moreira A.L.
      • Razavian N.
      • Tsirigos A.
      Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning.
      In contrast, such technologies are less frequently applied to nonneoplastic diseases, including inflammatory bowel disease. The pathologic diagnostic criteria for inflammatory diseases are typically descriptive rather than quantitative, thus hampering the appropriate labeling of histopathologic images. This study used artificial intelligence analysis by adopting clinical information called the presence or absence of recurrence, instead of histopathologic diagnosis itself, as the label. The application of artificial intelligence to pathologic evaluation of inflammatory diseases may yield clinically useful information by using characteristic clinical information as well as histopathologic diagnosis.
      In this study, detailed postoperative clinical information regarding patients with CD was used to label the training and test data. The clinical CDAI score and the endoscopic Rutgeerts score were used to define disease recurrence and to objectively classify patients into two groups based on recurrence within 2 years after surgery. Clinical information, including disease prognosis, is often integrated into image analysis.
      • Yu K.H.
      • Zhang C.
      • Berry G.J.
      • Altman R.B.
      • Ré C.
      • Rubin D.L.
      • Snyder M.
      Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features.
      ,
      • Yamamoto Y.
      • Tsuzuki T.
      • Akatsuka J.
      • Ueki M.
      • Morikawa H.
      • Numata Y.
      • Takahara T.
      • Tsuyuki T.
      • Tsutsumi K.
      • Nakazawa R.
      • Shimizu A.
      • Maeda I.
      • Tsuchiya S.
      • Kanno H.
      • Kondo Y.
      • Fukumoto M.
      • Tamiya G.
      • Ueda N.
      • Kimura G.
      Automated acquisition of explainable knowledge from unannotated histopathology images.
      However, in nontumor pathology, there has been minimal integration of image analysis and clinical information. Inflammatory diseases, including CD, are often chronic. Biological agents, such as infliximab and adalimumab, are useful for controlling the postoperative recurrence of CD,
      • Regueiro M.
      • Schraut W.
      • Baidoo L.
      • Kip K.E.
      • Sepulveda A.R.
      • Pesci M.
      • Harrison J.
      • Plevy S.E.
      Infliximab prevents Crohn's disease recurrence after ileal resection.
      ,
      • De Cruz P.
      • Kamm M.A.
      • Hamilton A.L.
      • Ritchie K.J.
      • Krejany E.O.
      • Gorelik A.
      • Liew D.
      • Prideaux L.
      • Lawrance I.C.
      • Andrews J.M.
      • Bampton P.A.
      • Gibson P.R.
      • Sparrow M.
      • Leong R.W.
      • Florin T.H.
      • Gearry R.B.
      • Radford-Smith G.
      • Macrae F.A.
      • Debinski H.
      • Selby W.
      • Kronborg I.
      • Johnston M.J.
      • Woods R.
      • Elliott P.R.
      • Bell S.J.
      • Brown S.J.
      • Connell W.R.
      • Desmond P.V.
      Crohn's disease management after intestinal resection: a randomised trial.
      but they are costly. There is a need for clinicians to clearly establish drug indications. The present findings indicate that deep learning–based histologic analysis is useful for classifying the clinical course of nonmalignant diseases and determining therapeutic indications.
      This study has shown that the subserosal layer of the intestinal tract is an important research area in the recurrence of CD. Most histopathologic studies based on artificial intelligence have targeted selected images with uniform histologic properties (eg, tumor area) or small tissues (eg, biopsy specimens). This study incorporated a computational analysis using nonannotated cropped images that depicted whole-slide images of surgical intestinal specimens. The images were histologically heterologous and included all layers of the intestinal wall (from the mucosal layer to the serosa). The present analysis using artificial intelligence showed that the accuracy of recurrence prediction differs depending on the layer of the intestinal tract. These findings suggest that deep learning–based image analysis of histologically heterologous samples may enable identification of tissue areas or histologic features that should be the focus of attention.
      In the present study, subserosal adipose tissue had the greatest number of predictive histologic features detectable by artificial intelligence. To our knowledge, this is the first report of the relationship between postoperative recurrence of CD and the histology of subserosal adipose cells. Thickening of mesenteric fat, known as creeping fat, is a typical macroscopic feature of CD.
      • Crohn B.B.
      • Ginzburg L.
      • Oppenheimer G.D.
      Regional ileitis: a pathologic and clinical entity.
      In recent studies, the role of mesenteric fat has been emphasized in CD, and its hypertrophy on abdominal computed tomography images is associated with early postoperative recurrence.
      • Li Y.
      • Zhu W.
      • Gong J.
      • Zhang W.
      • Gu L.
      • Guo Z.
      • Cao L.
      • Shen B.
      • Li N.
      • Li J.
      Visceral fat area is associated with a high risk for early postoperative recurrence in Crohn's disease.
      Furthermore, mesenteric granulomas are indicators of the long-term risk of postoperative disease recurrence.
      • Unger L.W.
      • Argeny S.
      • Stift A.
      • Yang Y.
      • Karall A.
      • Freilinger T.
      • Müller C.
      • Bergmann M.
      • Stift J.
      • Riss S.
      Mesenteric granulomas independently predict long-term risk of surgical recurrence in Crohn's disease.
      The present findings are consistent with those of these previous reports and are the first to support them from the viewpoint of adipocyte histology. The present findings offer new insights into the importance of adipose tissue in CD.
      Histopathologic analysis indicated that the degree of mast cell infiltration into the subserosal tissue was associated with postoperative recurrence. Previous studies suggest that mast cell activation and hyperplasia are involved in the pathophysiology of CD.
      • He S.H.
      Key role of mast cells and their major secretory products in inflammatory bowel disease.
      ,
      • Bischoff S.C.
      Role of mast cells in allergic and non-allergic immune responses: comparison of human and murine data.
      One previous study revealed an association between mast cell accumulation and stricture in CD.
      • Gelbmann C.M.
      • Mestermann S.
      • Gross V.
      • Köllinger M.
      • Schölmerich J.
      • Falk W.
      Strictures in Crohn's disease are characterised by an accumulation of mast cells colocalised with laminin but not with fibronectin or vitronectin.
      However, there have been no previous reports showing an association between mast cells and subserosal adipose tissue in CD or the risk of postoperative recurrence. To our knowledge, this report is the first to show a link between subserosal mast cell infiltration and postoperative recurrence of CD. Mast cells are difficult to morphologically recognize using hematoxylin and eosin staining. In addition, previous studies have reported the interaction between adipocytes and mast cells under pathophysiological conditions.
      • Elieh Ali Komi D.
      • Shafaghat F.
      • Christian M.
      Crosstalk between mast cells and adipocytes in physiologic and pathologic conditions.
      ,
      • Zatterale F.
      • Longo M.
      • Naderi J.
      • Raciti G.A.
      • Desiderio A.
      • Miele C.
      • Beguinot F.
      Chronic adipose tissue inflammation linking obesity to insulin resistance and type 2 diabetes.
      Mast cells may accumulate in adipose tissue in some pathologic situations, including obesity; they may also promote fibrosis.
      • Gurung P.
      • Moussa K.
      • Adams-Huet B.
      • Devaraj S.
      • Jialal I.
      Increased mast cell abundance in adipose tissue of metabolic syndrome: relevance to the proinflammatory state and increased adipose tissue fibrosis.
      Overall, this evidence suggests that subserosal mast cell activity affected and altered adipose cell morphology, which differentiated the recurrence and nonrecurrence groups. Further research is required to elucidate the molecular basis of the interaction between mast cells and subserosal adipocytes.
      This study used a deep learning model to accurately distinguish images and predict disease recurrence. Because of the nature of the analysis using deep learning, it is, in principle, difficult to extract the morphologic features that the machine learning model finds useful in image classification. However, the morphologic analysis of adipose cells from correctly sorted histologic images revealed a significant difference in adipose cell morphology between the recurrence and nonrecurrence groups. Adipose tissue in the recurrence group had a significantly smaller size, shorter cell-to-cell distances, and higher flattening values. These histologic features were defined as adipocyte shrinkage; these features were associated with postoperative recurrence of CD. In general, creeping fat is histologically characterized by inflammation and fibrosis, and an increased number and reduced size of mesenteric adipocytes,
      • Gonçalves P.
      • Magro F.
      • Martel F.
      Metabolic inflammation in inflammatory bowel disease: crosstalk between adipose tissue and bowel.
      and it is possible that the recurrence group showed more histologic changes comprising creeping fat. Although it was difficult for pathologists to classify the images into two groups without computer assistance, the machine learning model classified the same images with high accuracy. This result implies the importance of capturing subtle morphologic changes in adipocytes using artificial intelligence and indicates that the clinical application of artificial intelligence may provide more useful information from histologic findings.
      This study had some limitations. First, in this analysis, there was a significant difference in adipocyte morphology and degree of inflammatory cell infiltration between the two groups. However, it was not possible to set cutoff values for the morphologic characteristic values of adipocytes and the degree of inflammatory cell infiltration that clinicians can use on a daily basis. It may be speculated that this is because the machine learning model does not recognize adipocyte morphology and the degree of inflammatory cell infiltration individually, but comprehensively recognizes many morphologic features. Second, only basic analysis was performed of the degree of inflammatory cell infiltration by CD3 and CD68 immunostaining and toluidine blue staining, despite the fact that there are many different subsets of T cells (type 1, 2, and 17 helper T cells and anti-inflammatory regulatory T cells) and macrophages (M1 and M2 subsets). Although there were no statistically significant differences in the degree of infiltration of T cells or macrophages in this analysis, the possibility that there might be some differences between the recurrence and nonrecurrence groups with a more detailed subset analysis of T cells and macrophages could not be completely ruled out. Future studies using a wider variety of immune markers, along with codetection by indexing methods,
      • Goltsev Y.
      • Samusik N.
      • Kennedy-Darling J.
      • Bhate S.
      • Hale M.
      • Vazquez G.
      • Black S.
      • Nolan G.P.
      Deep profiling of mouse splenic architecture with CODEX multiplexed imaging.
      to analyze the relationship between immune cells in the subserosal adipose tissue and CD recurrence are desirable. Third, there was a significant association of CD recurrence with adipocyte morphology and mast cell infiltration in the subserosa, but the biological mechanism of association between adipocyte morphology and inflammatory cell infiltration has not been elucidated. Future mechanistic studies are needed to further evaluate this relationship. Finally, this analysis was based on a retrospective study design and the use of data from a single facility. Because this study required detailed preoperative and postoperative clinical data along with surgical samples, it was difficult to obtain data with a larger sample size. Prospective multicenter studies with larger number of cases are required to validate the present findings and generate a universally useful machine learning model that can be applied clinically. In addition, the clinical data used in this study did not include records of the presence or degree of creeping fat at the time of surgery. For future analysis, it will be necessary to collect detailed data with macroscopic features, including creeping fat.
      In summary, this computational analysis using convolutional neural network algorithms accurately classified histologic images of the intestine according to the postoperative recurrence of CD. The present findings may aid in integrating the histology of mesenteric adipose tissue into the prediction of postoperative disease recurrence and the pathophysiology of the disease.

      Acknowledgments

      We thank Masaharu Kohara, Megumi Nihei, and Takako Sawamura for kind technical assistance.

      Author Contributions

      H.K. and M.A. contributed equally to this work; H.K., M.A., and T.Ma. conceived and designed the study; H.K. collected and reviewed all the slides, analyzed the data, and wrote the article; M.A. performed computational analysis; T.Ma., M.K., K.O., S.T., S.N., and E.M. reviewed histopathologic diagnoses; T.O., Y.S., and T.Mi. collected and analyzed clinical data. All authors reviewed the article and approved the submitted version.

      Supplemental Data

      Figure thumbnail figs1
      Supplemental Figure S1Top 50 tile images of high-accuracy predictions. Fifty images with the most accurate predictions from the test data set of the nonrecurrence group (images with predictive values closest to 0; A) and the recurrence group (images with predictive values closest to 1; B). All 100 images contained adipose tissue.

      References

        • O'Sullivan B.
        • Brierley J.
        • Byrd D.
        • Bosman F.
        • Kehoe S.
        • Kossary C.
        • Piñeros M.
        • Van Eycken E.
        • Weir H.K.
        • Gospodarowicz M.
        The TNM classification of malignant tumours—towards common understanding and reasonable expectations.
        Lancet Oncol. 2017; 18: 849-851
        • Mills S.C.
        • von Roon A.C.
        • Tekkis P.P.
        • Orchard T.R.
        Crohn's disease.
        BMJ Clin Evid. 2010; 2010: 0416
        • Ogino T.
        • Mizushima T.
        • Matsuda C.
        • Mori M.
        • Doki Y.
        Essential updates 2018/2019: colorectal (benign): recent updates (2018-2019) in the surgical treatment of benign colorectal diseases.
        Ann Gastroenterol Surg. 2020; 4: 30-38
        • Veauthier B.
        • Hornecker J.R.
        Crohn's disease: diagnosis and management.
        Am Fam Physician. 2018; 98: 661-669
        • Targan S.R.
        • Hanauer S.B.
        • van Deventer S.J.H.
        • Mayer L.
        • Present D.H.
        • Braakman T.
        • DeWoody K.L.
        • Schaible T.F.
        • Rutgeerts P.J.
        A short-term study of chimeric monoclonal antibody cA2 to tumor necrosis factor α for Crohn's disease.
        N Engl J Med. 1997; 337: 1029-1036
        • Colombel J.F.
        • Sandborn W.J.
        • Rutgeerts P.
        • Enns R.
        • Hanauer S.B.
        • Panaccione R.
        • Schreiber S.
        • Byczkowski D.
        • Li J.
        • Kent J.D.
        • Pollack P.F.
        Adalimumab for maintenance of clinical response and remission in patients with Crohn's disease: the CHARM trial.
        Gastroenterology. 2007; 132: 52-65
        • Sabino J.
        • Verstockt B.
        • Vermeire S.
        • Ferrante M.
        New biologics and small molecules in inflammatory bowel disease: an update.
        Therap Adv Gastroenterol. 2019; 12 (1756284819853208)
        • Bernell O.
        • Lapidus A.
        • Hellers G.
        Risk factors for surgery and postoperative recurrence in Crohn's disease.
        Ann Surg. 2000; 231: 38-45
        • Bernell O.
        • Lapidus A.
        • Hellers G.
        Risk factors for surgery and recurrence in 907 patients with primary ileocaecal Crohn's disease.
        Br J Surg. 2000; 87: 1697-1701
        • Burr N.E.
        • Hall B.
        • Hamlin P.J.
        • Selinger C.P.
        • Ford A.C.
        • O'Connor A.
        Systematic review and network meta-analysis of medical therapies to prevent recurrence of post-operative Crohn's disease.
        J Crohns Colitis. 2019; 13: 693-701
        • Best W.R.
        • Becktel J.M.
        • Singleton J.W.
        • Kern Jr., F.
        • Kern F.
        Development of a Crohn's disease activity index: National Cooperative Crohn's Disease Study.
        Gastroenterology. 1976; 70: 439-444
        • Rutgeerts P.
        • Geboes K.
        • Vantrappen G.
        • Beyls J.
        • Kerremans R.
        • Hiele M.
        Predictability of the postoperative course of Crohn's disease.
        Gastroenterology. 1990; 99: 956-963
        • Novak G.
        • Parker C.E.
        • Pai R.K.
        • Macdonald J.K.
        • Feagan B.G.
        • Sandborn W.J.
        • D'Haens G.
        • Jairath V.
        • Khanna R.
        Histologic scoring indices for evaluation of disease activity in Crohn's disease.
        Cochrane Database Syst Rev. 2017; 7: CD012351
        • Reese G.E.
        • Nanidis T.
        • Borysiewicz C.
        • Yamamoto T.
        • Orchard T.
        • Tekkis P.P.
        The effect of smoking after surgery for Crohn's disease: a meta-analysis of observational studies.
        Int J Colorectal Dis. 2008; 23: 1213-1221
        • Fornaro R.
        • Caratto E.
        • Caratto M.
        • Fornaro F.
        • Caristo G.
        • Frascio M.
        • Sticchi C.
        Post-operative recurrence in Crohn's disease: critical analysis of potential risk factors: an update.
        Surgeon. 2015; 13: 330-347
        • Holt D.Q.
        • Moore G.T.
        • Strauss B.J.G.
        • Hamilton A.L.
        • De Cruz P.
        • Kamm M.A.
        Visceral adiposity predicts post-operative Crohn's disease recurrence.
        Aliment Pharmacol Ther. 2017; 45: 1255-1264
        • Wright E.K.
        • Kamm M.A.
        • De Cruz P.
        • Hamilton A.L.
        • Ritchie K.J.
        • Krejany E.O.
        • Leach S.
        • Gorelik A.
        • Liew D.
        • Prideaux L.
        • Lawrance I.C.
        • Andrews J.M.
        • Bampton P.A.
        • Jakobovits S.L.
        • Florin T.H.
        • Gibson P.R.
        • Debinski H.
        • Macrae F.A.
        • Samuel D.
        • Kronborg I.
        • Radford-Smith G.
        • Selby W.
        • Johnston M.J.
        • Woods R.
        • Elliott P.R.
        • Bell S.J.
        • Brown S.J.
        • Connell W.R.
        • Day A.S.
        • Desmond P.V.
        • Gearry R.B.
        Measurement of fecal calprotectin improves monitoring and detection of recurrence of Crohn's disease after surgery.
        Gastroenterology. 2015; 148: 938-947.e1
        • Stange E.F.
        • Travis S.P.L.
        • Vermeire S.
        • Beglinger C.
        • Kupcinskas L.
        • Geboes K.
        • Barakauskiene A.
        • Villanacci V.
        • Von Herbay A.
        • Warren B.F.
        • Gasche C.
        • Tilg H.
        • Schreiber S.W.
        • Schölmerich J.
        • Reinisch W.
        European evidence based consensus on the diagnosis and management of Crohn's disease: definitions and diagnosis.
        Gut. 2006; 55: i1-i15
        • Unger L.W.
        • Argeny S.
        • Stift A.
        • Yang Y.
        • Karall A.
        • Freilinger T.
        • Müller C.
        • Bergmann M.
        • Stift J.
        • Riss S.
        Mesenteric granulomas independently predict long-term risk of surgical recurrence in Crohn's disease.
        Color Dis. 2020; 22: 170-177
        • Nakao S.
        • Itabashi M.
        • Yamamoto T.
        • Okamoto T.
        Predictive value of myenteric and submucosal plexitis for postoperative Crohn's disease recurrence.
        J Anus Rectum Colon. 2017; 1: 56-64
        • Ferrante M.
        • de Hertogh G.
        • Hlavaty T.
        • D'Haens G.
        • Penninckx F.
        • D'Hoore A.
        • Vermeire S.
        • Rutgeerts P.
        • Geboes K.
        • van Assche G.
        The value of myenteric plexitis to predict early postoperative Crohn's disease recurrence.
        Gastroenterology. 2006; 130: 1595-1606
        • Lemmens B.
        • de Buck van Overstraeten A.
        • Arijs I.
        • Sagaert X.
        • Van Assche G.
        • Vermeire S.
        • Tertychnyy A.
        • Geboes K.
        • Wolthuis A.
        • D'Hoore A.
        • De Hertogh G.
        • Ferrante M.
        Submucosal plexitis as a predictive factor for postoperative endoscopic recurrence in patients with Crohn's disease undergoing a resection with ileocolonic anastomosis: results from a prospective single-centre study.
        J Crohns Colitis. 2017; 11: 212-220
        • Acs B.
        • Rantalainen M.
        • Hartman J.
        Artificial intelligence as the next step towards precision pathology.
        J Intern Med. 2020; 288: 62-81
        • Gubatan J.
        • Levitte S.
        • Patel A.
        • Balabanis T.
        • Wei M.T.
        • Sinha S.R.
        Artificial intelligence applications in inflammatory bowel disease: emerging technologies and future directions.
        World J Gastroenterol. 2021; 27: 1920-1935
        • Wang S.
        • Yang D.M.
        • Rong R.
        • Zhan X.
        • Xiao G.
        Pathology image analysis using segmentation deep learning algorithms.
        Am J Pathol. 2019; 189: 1686-1698
        • Goode A.
        • Gilbert B.
        • Harkes J.
        • Jukic D.
        • Satyanarayanan M.
        OpenSlide: a vendor-neutral software foundation for digital pathology.
        J Pathol Inform. 2013; 4: 27
        • Paszke A.
        • Gross S.
        • Chintala S.
        • Chanan G.
        • Yang E.
        • DeVito Z.
        • Lin Z.
        • Desmaison A.
        • Antiga L.
        • Lerer A.
        Automatic differentiation in PyTorch.
        (31st Conference on Neural Information Processing Systems)2017
        • Tan M.
        • Le Q.V.
        EfficientNet: rethinking model scaling for convolutional neural networks.
        Proceedings of the 36th International Conference on Machine Learning. 97. PMLR, 2019: 6105-6114 (97. PMLR, 2019, pp. 6105–6114)
        • Xie Q.
        • Luong M.T.
        • Hovy E.
        • Le Q.V.
        Self-training with noisy student improves imagenet classification.
        Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2020; : 10684-10695
        • Hendrycks D.
        • Mu N.
        • Cubuk E.D.
        • Zoph B.
        • Gilmer J.
        • Lakshminarayanan B.
        AugMix: a simple data processing method to improve robustness and uncertainty.
        ICLR 2020: Eighth International Conference on Learning Representations. ICLR, 2020
        • Radenovic F.
        • Tolias G.
        • Chum O.
        Fine-tuning CNN image retrieval with no human annotation.
        IEEE Trans Pattern Anal Mach Intell IEEE Computer Soc. 2019; 41: 1655-1668
        • Nishimura S.
        • Manabe I.
        • Nagasaki M.
        • Eto K.
        • Yamashita H.
        • Ohsugi M.
        • Otsu M.
        • Hara K.
        • Ueki K.
        • Sugiura S.
        • Yoshimura K.
        • Kadowaki T.
        • Nagai R.
        CD8+ effector T cells contribute to macrophage recruitment and adipose tissue inflammation in obesity.
        Nat Med. 2009; 15: 914-920
        • Lumeng C.N.
        • DeYoung S.M.
        • Bodzin J.L.
        • Saltiel A.R.
        Increased inflammatory properties of adipose tissue macrophages recruited during diet-induced obesity.
        Diabetes. 2007; 56: 16-23
        • Porsche C.E.
        • Delproposto J.B.
        • Geletka L.
        • O'Rourke R.
        • Lumeng C.N.
        Obesity results in adipose tissue T cell exhaustion.
        JCI Insight. 2021; 6: e139793
        • Coudray N.
        • Ocampo P.S.
        • Sakellaropoulos T.
        • Narula N.
        • Snuderl M.
        • Fenyö D.
        • Moreira A.L.
        • Razavian N.
        • Tsirigos A.
        Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning.
        Nat Med. 2018; 24: 1559-1567
        • Yu K.H.
        • Zhang C.
        • Berry G.J.
        • Altman R.B.
        • Ré C.
        • Rubin D.L.
        • Snyder M.
        Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features.
        Nat Commun. 2016; 7: 1-10
        • Yamamoto Y.
        • Tsuzuki T.
        • Akatsuka J.
        • Ueki M.
        • Morikawa H.
        • Numata Y.
        • Takahara T.
        • Tsuyuki T.
        • Tsutsumi K.
        • Nakazawa R.
        • Shimizu A.
        • Maeda I.
        • Tsuchiya S.
        • Kanno H.
        • Kondo Y.
        • Fukumoto M.
        • Tamiya G.
        • Ueda N.
        • Kimura G.
        Automated acquisition of explainable knowledge from unannotated histopathology images.
        Nat Commun. 2019; 10: 1-9
        • Regueiro M.
        • Schraut W.
        • Baidoo L.
        • Kip K.E.
        • Sepulveda A.R.
        • Pesci M.
        • Harrison J.
        • Plevy S.E.
        Infliximab prevents Crohn's disease recurrence after ileal resection.
        Gastroenterology. 2009; 136 (quiz 716): 441-450.e1
        • De Cruz P.
        • Kamm M.A.
        • Hamilton A.L.
        • Ritchie K.J.
        • Krejany E.O.
        • Gorelik A.
        • Liew D.
        • Prideaux L.
        • Lawrance I.C.
        • Andrews J.M.
        • Bampton P.A.
        • Gibson P.R.
        • Sparrow M.
        • Leong R.W.
        • Florin T.H.
        • Gearry R.B.
        • Radford-Smith G.
        • Macrae F.A.
        • Debinski H.
        • Selby W.
        • Kronborg I.
        • Johnston M.J.
        • Woods R.
        • Elliott P.R.
        • Bell S.J.
        • Brown S.J.
        • Connell W.R.
        • Desmond P.V.
        Crohn's disease management after intestinal resection: a randomised trial.
        Lancet. 2015; 385: 1406-1417
        • Crohn B.B.
        • Ginzburg L.
        • Oppenheimer G.D.
        Regional ileitis: a pathologic and clinical entity.
        Am J Med. 1952; 13: 583-590
        • Li Y.
        • Zhu W.
        • Gong J.
        • Zhang W.
        • Gu L.
        • Guo Z.
        • Cao L.
        • Shen B.
        • Li N.
        • Li J.
        Visceral fat area is associated with a high risk for early postoperative recurrence in Crohn's disease.
        Color Dis. 2015; 17: 225-234
        • He S.H.
        Key role of mast cells and their major secretory products in inflammatory bowel disease.
        World J Gastroenterol. 2004; 10: 309-318
        • Bischoff S.C.
        Role of mast cells in allergic and non-allergic immune responses: comparison of human and murine data.
        Nat Rev Immunol. 2007; 7: 93-104
        • Gelbmann C.M.
        • Mestermann S.
        • Gross V.
        • Köllinger M.
        • Schölmerich J.
        • Falk W.
        Strictures in Crohn's disease are characterised by an accumulation of mast cells colocalised with laminin but not with fibronectin or vitronectin.
        Gut. 1999; 45: 210-217
        • Elieh Ali Komi D.
        • Shafaghat F.
        • Christian M.
        Crosstalk between mast cells and adipocytes in physiologic and pathologic conditions.
        Clin Rev Allergy Immunol. 2020; 58: 388-400
        • Zatterale F.
        • Longo M.
        • Naderi J.
        • Raciti G.A.
        • Desiderio A.
        • Miele C.
        • Beguinot F.
        Chronic adipose tissue inflammation linking obesity to insulin resistance and type 2 diabetes.
        Front Physiol. 2020; 10: 1607
        • Gurung P.
        • Moussa K.
        • Adams-Huet B.
        • Devaraj S.
        • Jialal I.
        Increased mast cell abundance in adipose tissue of metabolic syndrome: relevance to the proinflammatory state and increased adipose tissue fibrosis.
        Am J Physiol Endocrinol Metab. 2019; 316: E504-E509
        • Gonçalves P.
        • Magro F.
        • Martel F.
        Metabolic inflammation in inflammatory bowel disease: crosstalk between adipose tissue and bowel.
        Inflamm Bowel Dis. 2015; 21: 453-467
        • Goltsev Y.
        • Samusik N.
        • Kennedy-Darling J.
        • Bhate S.
        • Hale M.
        • Vazquez G.
        • Black S.
        • Nolan G.P.
        Deep profiling of mouse splenic architecture with CODEX multiplexed imaging.
        Cell. 2018; 174: 968-981.e15