Toward predictive models for drug-induced liver injury in humans: are we there yet?

Drug-induced liver injury (DILI) is a clinical event that can be associated with severe outcomes such as acute liver failure [1]. Approximately 1000 drugs, herbal products, vitamins and illicit compounds are associated with liver injury [2], albeit generally at a low frequency. These DILI events are mostly of an idiosyncratic nature and are host dependent [3,4]. DILI is also a major cause for the termination of drug development programs [5] and frequently results in regulatory actions including denied approval and black box warnings [6]. Notably, more than 50 approved drugs have been withdrawn from domestic markets in different countries worldwide owing to DILI [7,8].

Currently employed preclinical testing strategies do not predict DILI in patients reliably [9], with less than 55 and 25% of DILI drugs being predicted based on the regulatory animal toxicity studies and simple in vitro tests, respectively [10,11]. Major investments have therefore been made to reduce risk for DILI, including the search and validation of predictive biomarkers [12]. Diverse liver-related biological and chemical data have been deposited in the public domain to bridge the gap between DILI prediction and clinical outcome, enabling independent research teams to analyze the data. Turning data into knowledge remains a challenge, particularly with microarray-based toxicogenomic data sets. Likewise, very large relational databases can be generated by high-content screening (HCS) assays that simultaneously monitor multiple biological end points that become enormously valuable when combined with whole-genome toxicogenomics. However, these tests mostly rely on a limited number of drugs, which limits statistical power for developing robust predictive models. This review will focus on the current state of efforts to develop predictive models that can be used in preclinical studies for detecting DILI risk in humans.

Classification of drugs for DILI risk in humans

An improved understanding of DILI is vital for the development of accurate and useful predictive models [13]. DILI is alleged to be either a drug’s inherent property or an idiosyncratic outcome peculiar to a case or individual; however, the value and validity of such a classification is questionable and no conclusive evidence supports that intrinsic and idiosyncratic DILIs arise by different modes of action [14,15]. Here, we focused on the classification of DILI risk in humans instead.

Annotating a drug’s potential to cause DILI in humans is a huge undertaking in a clinical setting [16]. At least three attributes, including severity, causality and incidence, need to be considered when a drug’s DILI risk in humans is assessed [8]. The difficulty of accurate annotations can be attributed to factors such as the diverse and complicated clinical manifestation of DILI, the lack of sensitive and specific biomarkers for DILI diagnosis, the difficulties in determining causality and the serious under-reporting of DILI cases.

Some prominent data sources for the classification of human DILI risk are listed in Table 1, and each are either case report-based approaches, drug compendium-based approaches or a combination of these. Clinical investigators always prefer the case report-based approach. Suzuki et al. surveyed multiple well-documented and adjudicated DILI cases and developed a repository including 319 drugs causing overall liver injury, 107 drugs causing acute liver failure and 47 drugs suspended or withdrawn from the market [17]. Likewise, drug compendia such as the Physicians’ Desk Reference are widely used as resources. Drug compendia are continually updated and provide judgmental information based on a body of comprehensive evidence and expert opinions. Chen et al. employed US FDA-approved drug labels to assess DILI risk in humans, categorizing drugs as of most-DILI-concern, less-DILI-concern or no-DILI-concern [8]. The DILI concern classification is in a straightforward manner based on the drug label section and severity of DILI. Despite imperfections and limitations, drug labels reflect the serious thoughts and consensus of numerous experts and constitute one major public data source that comprehensively integrates case reports and compendia [8].

Most published DILI classifications are of case reports and compendia. Xu et al. defined a drug to be DILI positive, if it was either withdrawn, not approved in the USA, issued with a boxed warning, marketed with warnings related to DILI or associated with a significant number (>10) of case reports of serious DILIs that met the criteria of the Hy’s law, which estimates risk of fatal DILI [18]. Sakatis et al. annotated a drug as DILI when at least 50 reports of clinical hepatotoxicity or three reports of life-threatening events were recorded in a bibliographic database of drug-related liver injuries and the Physicians’ Desk Reference [19].

In an effort for assessing the consistency of DILI classifications, four data sets were selected (the Suzuki et al.[17], Chen et al.[8], Xu et al.[18], and Sakatis et al.[19] data sets) as indicated in Supplementary Table 1. The drugs qualified for analyses were only those that were classified in two or more data sets as well as belonging to the group of FDA-approved drugs. A drug was defined as equivocal classification when there was disagreement between any two data sets. Among the qualified drugs, 181 drugs (74%) have consistent DILI classification, and 64 drugs were equivocal. The equivocal drugs were further analyzed to identify in which label sections their DILI risk were disclosed. Some 15 drugs were in ‘Warning & Precautions’ (19% of the 77 ‘Warning & Precautions’ drugs), 39 were in ‘Adverse Reactions’ (63% of the 62 ‘Adverse Reactions’ drugs), and ten were in ‘No Match’ (24% of the 41 ‘No Match’ drugs). Notably, the DILI lexeme ‘Adverse Reactions’ is quite debatable and accounts over 60% of the total 64 equivocal drugs.

Predictive models from homogenous data

The primary impediment to improving DILI prediction models and screening classifiers is the poor understanding of DILI mechanisms and pathogenesis. Models so far fall into three categories as determined by type of data used: first, chemical structure based; second, in vitro assay-based; and third, toxicogenomics based. The prominent predictive models selected from the literature are summarized in Table 2. Animal DILI models are not included since most are drug-specific studies (for a comprehensive review see [20]).

▪ Chemical structure-based in silico models

Once the model has been developed, the chemical structure-based in silico model has the advantage of rapidly screening an unlimited number of chemicals at minimal cost. The basic assumption is that similar chemical structures have similar properties and toxicity profiles [21]. Use of structure-based models to screen for DILI has been limited, but efforts have nonetheless progressed [22] using both knowledge-based expert systems and quantitative structure–activity relationship (QSAR)-based approaches [23].

Knowledge-based models

The knowledge-based or expert system models are usually developed with commercial software such as Derek for Windows (Lhasa Limited) from which structure alerts are derived. The structure alerts are chemical substructures or motifs the expert system statistically associates with the biological end point of interest. The validity of the structural alerts needs to be assessed by experts in order to minimize and balance false-positive and false-negative predictions. Groups of chemicals with similar toxicity profiles are identified in order to ascertain the shared structural motifs associated with toxicity. Robust structure alerts with mechanistic implications have been used as screening tools to identify potential toxins and to mitigate the risk of idiosyncratic drug toxicity [24].

Egan et al. reported 74 structure alerts for predicting liver toxicity using a collection of 244 molecules [22]. Most of the structure alerts were functional groups related to reactive or toxic metabolites. The authors did not evaluate the performance but stated that prediction accuracy of alerts was limited. Later, Marchant et al. derived structure alerts from both propriety and literature data [25]. Their alerts were able to identify 40% of 137 chemicals in one published data set, but the sensitivity dropped to 23% when challenged by another set of 300 hepatotoxins. The authors claimed that the performance could be further improved by using applicability domains defined by lipophilicity and molecular weight. Hewitt et al. utilized a published data set comprising of 951 compounds having annotated DILI risk to develop structural alerts for DILI prediction [21]. Mechanistic rationales for the alerts were proposed, but the predictive performance was not adequately assessed. Greene et al. developed 23 DILI structure alerts based on the literature knowledge and assessed its performance using 623 compounds from a Pfizer-developed data set [26]. An overall accuracy of 56% was reported with 46% sensitivity and 73% specificity.

QSAR-based models

QSAR models have been developed using supervised machine learning methods where chemical structure is encoded by calculated molecular descriptors. Given adequate chemicals both active and inactive, QSAR approaches can be faster since the need for expert opinion is averted.

Cheng and Dixon were among the first to report a QSAR model for DILI prediction [27]. Some 149 hepatotoxins with dose–response data and 233 nonhepatotoxins comprised the training set for a classifier developed with recursive partitioning trees. Leave-10%-out cross-validation yielded 76% accuracy and the external validation on a hold-out data set of 54 drugs was of similar order, that is, 81% accuracy. Cruz-Monteagudo et al. employed several different classifiers to develop a predictive model based on 74 selected drugs [28]. Internal cross-validation yielded accuracies of 78–86% for classifiers, and 82% accuracy for an external data set of 13 hepatotoxins. Matthews et al. developed models for five types of DILI using QSAR programs based on approximately 1600 chemical structures [29]. Their best model exhibited 39% sensitivity and 87% specificity estimated from internal cross-validation and 89% sensitivity on an external validation set of 18 hepatotoxic drugs. Roger et al. reported a QSAR model based on approximately 200 selected drugs from the FDA’s human liver adverse event database [30]. The model yielded 84% accuracy, 74% sensitivity and 94% specificity for predicting composite liver enzyme score assessed on an external validation set of 37 drugs. Fourches et al. developed QSAR classification models with a training set of 248 hepatotoxins and 283 nonhepatotoxins, and reported cross-validation accuracies of 62–68%, and 78% accuracy for an external validation set with 18 compounds [31].

Despite the fact that some of the previous reports presented moderately high predictive accuracy, their external validation sets are often quite small, consisting of only 20–50 drugs. Among the published models with larger external validation sets, the predictive performance seems to be less favorable. For example, Liew et al. reported a QSAR model for DILI prediction [32]. When applied to an external validation of 120 compounds, the model achieved 75% accuracy, 82% sensitivity and 65% specificity. Ekins et al. developed a predictive ligand-based Bayesian QSAR model for assessing human DILI using a training set of 295 compounds [33]. It achieved 60% accuracy with 56% sensitivity and 67% specificity on an external validation set with 237 drugs. Liu et al. developed a DILI prediction system based on a group of in silico predictive models using 13 selected DILI-relevant end points [34]. Accuracies of 60–70% were achieved when assessed by three external validation sets with a total of >200 drugs. Chen et al. developed a QSAR model using Mold2 molecular descriptors, a decision forest algorithm and a training set with 197 drugs [35]. The model achieved 69, 62 and 63% accuracies when assessed by three separate external validation sets having a total of approximately 500 drugs. Notably, the three studies (i.e., Liew et al.[32], Ekins et al.[33] and Chen et al.[35]) and except for Liu et al.[34] employed a relative balance of hepatotoxic-positive and -negative drugs in the validation sets.

Considering the applicability domain of a model, as determined by the chemical space of the training set, is a powerful way to ascribe confidence to predictions [36]. If a chemical’s structure is within the applicability domain of the model, there will be greater confidence in a given prediction of an untested chemical. By contrast, if a chemical’s structure is outside the domain, prediction confidence will be low. An applicability domain has been successfully applied to the prediction of end points with well-established mechanisms by attaching statistical confidence to individual predictions [37,38]; however, some studies reported no such benefit when used for DILI prediction [32,33,35]. For example, Liew et al. reported that chemical structure-based applicability domain did not delineate separate categories of high-confidence and low-confidence predictions, with corresponding higher and lower prediction accuracy, respectively [32], and these observations were confirmed independently [33,35]. Instead, Chen et al. proposed a therapeutic categories-based applicability domain approach and demonstrated its superiority in the prediction of DILI when assessed by three independent external validation sets [35].

▪ In vitro assay-based models

Due to its requirement for a low quantity of drug substances and the possibility of high-throughput screening, in vitro assays have been widely used in preclinical testing. However, conventional cytotoxicity assays employing a single end point have poorly correlated (mostly <25% sensitivity) with human DILI outcomes [11], an unremarkable results given that the assay has little to do with putative mechanisms and pathogenesis of DILI. Simultaneous use of multiple in vitro models could improve results as a broader spectrum of molecular mechanisms would be encompassed, some of which might be associated with DILI. HCS assays that simultaneously measure multiple end points and associated cellular reactions offer an opportunity to detect changes of DILI-related molecular signals in live cells with higher sensitivity [39,40].

One of the first reports using HCS for hepatotoxicity employed the HepG2 (human hepatoma) cell line [41]. The HCS assay tested 243 drugs/chemicals that were a mixture of hepatotoxins, nontoxins and those known to be toxic in organs other than liver. Five mechanistic end points (i.e., intracellular calcium level, mitochondrial membrane potential [MMP], cell number, nuclear area and plasma membrane permeability) were measured. A predictive model based on a simple logistical combination of the measured end points reported 80% sensitivity and 90% specificity. Later, Xu et al. developed a primary human hepatocyte-based HCS assay and tested over 300 drugs/chemicals at a single concentration of 100-times human the therapeutic plasma concentration C_max[18]. Four related assays (nuclear and lipid content, intracellular glutathione, mitochondrial damage and oxidative stress) were measured after a 24-h cell incubation. A predictive model based on a random forest algorithm yielded sensitivities of 50–60% and specificities of 95–100%. Considering the rapid deterioration of primary hepatocytes cultured under conventional conditions, Khetani et al. developed a micropatterned co-culture liver model that enabled long-term dosing in functionally viable hepatocytes [42]. They tested 45 drugs along with four standard cytotoxic end points (i.e., glutathione, ATP level, albumin and urea secretion). The human primary hepatocytes model yielded 66% sensitivity and 90% specificity. More recently, Persson et al. reported a HepG2 assay based on six parameters (nuclei counts, nuclear area, plasma membrane integrity, lysosomal activity, MMP and mitochondrial area) [43]. They developed a zone classifier based on nuclei count, MMP and human C_max, and reported 45% sensitivity and 100% specificity when tested on 102 selected drugs.

Predictive models derived from other in vitro assays that improve prediction of DILI have also appeared. For example, covalent binding is a measure of chemically reactive metabolites, but alone is insufficient to discriminate positive from negative hepatotoxic compounds [24,44]. Nakayama et al. developed a zone classifier using an ordinal logistic regression analysis based on daily dose and covalent binding [45]. It correctly identified 17 out of 22 hepatotoxic drugs (77% sensitivity) and 12 out of 14 safe drugs (86% specificity). More recently, Sakatis et al. developed a testing strategy using a decision tree algorithm based on assays for covalent binding, daily dose, CYP450 metabolism inhibition and glutathione adducts [19]. The model was assessed using 113 hepatotoxic-positive and 110 -negative marketed drugs and reported 45% sensitivity and 90% specificity. Additionally, the in vitro models like those for screening intrahepatic cholestasis allow insight into the underlying mechanisms of DILI [46].

Notably, most of the in vitro-based predictive models required drug exposure data (i.e., human C_max). These data are commonly determined in human safety and efficacy studies and may possibly be predicted at the preclinical stage. Jones et al. predicted C_max with reasonable accuracy using in silico and in vitro data [47]. However, the predictions are far from perfect and tend to generally underestimate C_max[48].

▪ Toxicogenomics-based models

The HCS assays alone can only offer limited sensitivities to identify liver liabilities of drugs mostly due to the mechanisms evolved in the available end points studied. In this regard, microarray-based technology enables unprecedented opportunities to comprehensively assess alterations in gene-expression profiles induced by hepatoxins. Consequently, the microarray technology was considered to be a groundbreaking addition to conventional toxicology assays at its inception [49,50]. However, the progress in developing toxicogenomic predictive models has not been as effective as envisioned, partly due to their high cost and the lack of advanced knowledge discovery and data mining tools [51]. Additionally, the transcriptomic approaches have been for end points not necessarily relevant to DILI, that is, the measured altered gene expression might or might not be associated with mechanisms involving hepatocellular injury. Two large-scale toxicogenomics data sets published recently, for example the Japanese toxicogenomics database [52] and DrugMatrix [53], might enable real progress in developing DILI prediction models [51].

Several toxicogenomic studies have been performed for characterizing DILI; however, only very few models have been developed for predicting human DILI based on toxicogenomics data. Cha et al. developed a toxicogenomic model based on HepG2 cells to predict human DILI risk of NSAIDs [54]. The model identified 77 discriminatory genes derived from measurements for eight hepatotoxic and eight nonhepatotoxic drugs, and successfully classified four additional NSAIDs. Zhang et al. claimed that genomic biomarkers might not be more sensitive than the well-established markers of human DILI used in preclinical studies [55]. They demonstrated that the gene signatures derived from rats with abnormal ALT or bilirubin could predict human DILI, but the gene signatures derived from rats without ALT or bilirubin abnormal elevation cannot. Low et al. reported a toxicogenomic model derived from microarray data obtained from rat livers at 24 h after a single dose treatment, and the model was used to predict rat hepatotoxicity observed after 28 days of continued exposure [56]. The predictive model achieved a 76% correct classification rate (calculated as [sensitivity + specificity]/2)when assessed by fivefold cross-validation.

Metabolomics is yet another promising approach for discovering hepatotoxicity biomarkers. The approach has the major advantages of providing noninvasive sample collection and easy sample preparation and analysis [57,58]. A large-scale metabolomics study was reported that utilized the proton nuclear magnetic resonance spectra of rodent urine and serum to develop a predictive model for liver toxicity with 67% sensitivity and 77% specificity [59]. Circulating miRNAs also show potential to provide biomarkers of human DILI [60,61]. The miRNAs (miR-122 and miR-192) were observed to be significantly increased in the acetaminophen-induced liver injury patients compared with the healthy controls [62].

Predictive models from heterogeneous data

Most published models were developed from a homogenous data source, that is, chemical structure-based, in vitro assay-based or toxicogenomics-based data. Heterogeneous models employ one or more data types in an integrative approach. Rusyn et al. reviewed the efforts to predict in vivo toxicity by integrating chemical structure descriptors with in vitro assay parameters to attain more predictive accuracy [63]. The heterogeneous models can be categorized into data integration or model integration modes in accordance with the objectives laid out.

▪ Data integration

The data integration approach pools data from different sources prior to model development. Data integration from multiple sources can be challenging but provides opportunities to improve a model’s predictive power. For example, Chen et al. analyzed 164 FDA-approved oral medications and showed an association of high daily doses (≥100 mg/day) and lipophilicity (partition coefficient [logP] ≥3) with significant risk for DILI, thus defining a ‘rule-of-two’ [64]. This principle was further applied and validated using an independent set of 179 oral medications, drug pairs with similar chemical structures and molecular targets, but different DILI potential, and to clinical case studies with complex comedication regimes. The authors demonstrated that the ‘rule-of-two’ caused fewer false positives than daily dose alone as a means of assessing risk for DILI in humans. Zhu et al. combined molecular descriptors from chemical structure and biological descriptors from in vitro assays to develop a hybrid model [65]. A fivefold cross-validation based on 156 DILI-positive and 136 DILI-negative compounds indicated that the hybrid model performed slightly better than the QSAR model alone. Moreover, Low et al. utilized chemical descriptors and toxicogenomic profiles to develop a hybrid model for 127 drugs to predict rat hepatotoxicity [56]. Assessed by a fivefold cross-validation, the hybrid model reported a correct classification of 68–77%, while the QSAR and toxicogemonic models alone reported a correct classification of 61 and 76%, respectively. The authors emphasized that the use of chemical and biological descriptors enriched mechanistic understanding and thus the model development.

Directly pooling data does not always work well for data integration, especially when the data structures of multiple data sources are different. Hence, an adaptor that does not compromise data integrity needs to be developed. Another type of indirect integration approach was developed by largely relying on advanced algorithms instead of a simple combination that incorporates different data sources together as a joint model. Consequently, it has the advantage of retaining the data structure of different data sources and mostly offers a better prediction [66]. Zitnik and Zupan developed a matrix factorization-based data integration approach to predict DILI potential of drugs from a gene-expression data set [101]. The model was derived from data of 29 different sources and achieved better accuracy than the conventional methods when assessed by cross-validation.

▪ Model integration

Alternatively, models can be integrated after being independently developed from multiple data sources. The model integration approaches, including model consensus, have the capacity to improve predictive power [63,67]. Chen et al. reported a testing strategy by combining the ‘rule-of-two’ model with data from an in vitro HCS assay [Chen M, Tung C, Shi Q et al. Improve the prediction of in vitro assays for drug-induced liver injury by the integration of the ‘rule-of-two’. (2013), Submitted]. Specifically, a primary rat hepatocyte-based HCS assay was used to further investigate the negative predictions from the ‘rule-of-two’ model. Assessed by a set of 73 drugs with clear evidence for clinically relevant DILI, the integrated model improved the accuracies by approximately 10% and reduced the number of drugs requiring in vitro test by approximately 20%, compared with the corresponding in vitro model alone.

Conclusion

Recent advances in preclinical testing strategies have improved our ability to identify drugs with risk for DILI. The construction of predictive models benefits from an integration of chemical structure, cellular end points, toxicogenomics data and data from multiple sources. These models can generate a high specificity between 90 and 95%, but perform less well for sensitivity (˜50%). The limited power of DILI prediction is mostly attributed to the complex nature of DILI, a poor understanding of mechanisms, a scarcity of human hepatotoxicity data and insufficient bioinformatics capabilities. Altogether, the currently reported models have not demonstrated superiority of the predictive power over the regulatory-required animal toxicity studies, but the evolution of DILI study will continue to refine testing strategies.

Future perspective

The lack of an in-depth understanding of DILI mechanisms limits the development of improved DILI models. However, the continuing research efforts will result in knowledge gain that can be translated into novel testing strategies, thus leading to better predictive models. For example, an in vitro approach that assays for the effects of cytokines was proposed to screen drugs with inflammation-associated idiosyncratic hepatotoxicity [68]. The assay was conceived from the observation that idiosyncratic hepatotoxicity for certain drugs has been reproduced in animals with mild inflammatory stress induced by bacterial LPS [9,20].

Despite the difficulty of performing mechanistic studies for idiosyncratic DILI, significant progress has been realized in identifying an association between carriers of specific HLA haplotypes and DILI events for some specific drugs [61]. For drugs such as flucloxacillin, abacavir, lapatinib, lumiracoxib, ticlopidine and ximelagatran that do not cause hypersensitivity, as seen for an immune-related response, an association between the HLA phenotype and risk for DILI was determined [12,69]. A large-scale genome-wide association study suggested a more significant role of common variants of immune-coding genes than those involved in drug metabolism or transport in the predisposition to DILI [70,71]. Similarly, the application of next-generation sequencing technologies (i.e., RNA-Seq) may help to identify rare functional variants underlying disease that are potentially associated with DILI events [71]. The clinical use of genetic association data in identifying DILI-susceptible patients is hurdled by the infrequent allele frequencies and a low incidence of drug-specific DILI events [71]. However, an understanding of the role of HLA in DILI events will stimulate the generation of more powerful predictive models for DILI [72].

Furthermore, enabling technologies based on pluripotent stem cells [73], organ-on-chips [74], zebrafish [75] and humanized mouse models [76] will provide powerful tools to investigate DILI mechanisms. Notably, the rapidly advancing research in stem cells raises possibility of using human pluripotent stem cells to provide more standardized hepatocytes for in vitro testing strategies [73]. The stem cell-derived hepatocytes represent a new cell source for the existing human primary hepatocyte model compared with thawed hepatocytes after cryopreservation and freshly-isolated hepatocytes. Until now, the stem cell-derived hepatotocyte systems are not fully validated and most of applications are still in research. Additionally, it is also promising to combine organotypic liver culture models based on hepatocytes and immune cells, the latter of which could be derived from patients who have experienced DILI [77].

The scarcity of human hepatotoxicity data in the public domain is another hindrance to developing predictive models. Due to the lack of sensitive and specific biomarkers for DILI diagnosis in clinical practice, DILI assessment is still largely a diagnosis of exclusion. The causality assessment of a DILI case is a laborious task and requires expert knowledge [78]. The need to comprehensively collect human hepatotoxicity data remains and the drug labeling-based approach seems to be more reliable, mostly owing to the intensive involvement of experts’ knowledge and the reliance on comprehensive data collection [8]. Meanwhile, many efforts to collect the human hepatotoxicity data have been explored, including the NIH’s LiverTox database [79,102] and the FDA’s Liver Toxicity Knowledge Base (LTKB) [80,103].

Advanced computational techniques are also needed to mine the data and develop models with improved predictive power. Given the complexity of DILI mechanisms, a systematic approach is necessary to make full use of a wide range of information (experimental, clinical and computational) for the development of robust predictive models. Integrative models are promising, which can entrain specific algorithms to interrogate heterogeneous data and fuse them together in a joint model without distorting the modular structure of data from different sources. Network modeling is another promising approach to extract significant relationships among the features from heterogeneous data that benefit DILI mechanism studies [80]. Meanwhile, topic modeling has been demonstrated as a powerful text mining technique and demonstrated its utility in mining the FDA drug labels by extracting/discovering the relationships of different concepts (e.g., drugs, diseases and mechanisms) presented in the documents [81]. Furthermore, a unifying ontology to integrate histological and clinical observations for DILI lexeme was recently reported to improve the development of advanced knowledge discovery and data mining tools across different public repositories [82]. Overall, the advancement of bioinformatics tools will boost the development of DILI predictive models and help improve the understanding of its mechanisms.

For a complex end point such as DILI, a ‘one-fits-all’ model might fail to generate sufficient predictive performances. The success of applicability domains in the predictive model suggests that a proper stratification of drug space is essential. The therapeutic categories defined in the ontology of the WHO’s Anatomical Therapeutic Chemical Classification System were demonstrated to be a potential option [35]. Lastly, the predictive models should be validated, and their strengths and weaknesses should be fully assessed [83], which will be critical for their applications in drug development.

DILI studies are a concerted effort and involve multiple disciplines such as clinical hepatology, preclinical toxicology, molecular biology, and bioinformatics; an integration of different expert knowledge is the foundation for success. Several national and international programs have been initiated, including but not limited to: the AASLD-FDA-NIH-PhRMA – Hepatotoxicity Special Interest Group Conference [104], the DILI network project of the NIH [105], the DILIsym project of the Hamner Institutes for Health Sciences [106], the Virtual Liver Project (v-Liver™) of the US Environmental Protection Agency [107] and the Virtual Liver Network project funded by the German government [108]. All these efforts will improve our understanding of the complicated mechanisms of DILI to create better testing strategies that will reduce the time and cost of drug development and ultimately benefit patient safety.