US FDA public meeting: identification of concepts and terminology for multicomponent biomarkers

Biomarkers play a pivotal role in medical practice and product development. Increasing complexity in the field of biomarker development has revealed important differences in biomarker terminology frameworks. Definitions of different biomarkers, and subsequent implications for regulatory decisions, necessitate continued dialogue among many stakeholders, including research scientists, clinicians, industry and regulatory agencies. Biomarkers composed of multiple components are widely utilized across disciplines, creating a need for standardized descriptive language, both at a conceptual and a granular level. Standardized descriptive language for a multicomponent biomarker (MCB) would allow for harmonized discussions and applications in decision-making and support for clinical and regulatory use. To address this need, the US FDA held a virtual public workshop [1] with two central goals related to MCBs: 1) obtain feedback from the scientific community and identify MCB concepts for which terminology development or refinement is needed for effective communication, and 2) discuss the diverse use of biomarkers across the FDA and identify common issues. Attendees discussed biomarker output interpretation across regulatory settings. This included case studies highlighting FDA-wide and external experience regarding the use of MCBs. The following topics were emphasized: context of use, measurements, outputs, transformation/modeling of outputs and application in the decision-making process.

The workshop comprised four prerecorded background presentations (Supplementary Table 1) six case study presentations (Supplementary Appendix 1), and live virtual sessions. Three live-panel discussions were conducted around the example case studies, with each panel covering two cases (Supplementary Table 2). The final panel session recapitulated common themes and concepts of MCB usage from the presentations and panel discussions. Key MCB usage themes were identified during the workshop: the rigor and appropriateness of the biomarker development processes; challenges related to terminology for MCB construction, identification and scientific description; and how hierarchical categorization of MCBs may facilitate research, clinical application and validation.

This manuscript summarizes background presentations (Supplementary Table 1) and highlights themes and ideas raised during each panel discussion (Supplementary Table 2) from the meeting. Finally, some concluding remarks and future considerations are noted. In addition, summaries of each case study discussed are included in Supplementary Appendix 1.

Background presentations

Representatives from industry, government and academia provided several prerecorded presentations to introduce biomarker and MCB terminology. Dr Lisa McShane presented an overview of the publicly available lexicon resource ‘Biomarkers, EndpointS and other Tools’ (BEST) [2]. BEST describes a biomarker as “a defined characteristic that is measured as an indicator of normal biological processes, pathogenic processes, or biological responses to an exposure or intervention, including therapeutic interventions.” A biomarker may be comprised of multiple components of the same type or different types of independent measurements appropriate for a range of biomarker types, which may include molecular, histologic, radiographic or physiologic characteristics. MCBs can include multiple types of biomarker measurements, combined through algorithms, to describe defined characteristics indicative of biological processes, pathogenic processes or responses to an exposure or intervention.

In the keynote address, Dr John Wagner proposed the following MCB definition: “multiple biomarkers used individually for pattern recognition, or a single calculated value derived from a defined set of biomarkers and a known algorithm, required when a ‘characteristic’ [is] not adequately captured by [a] single measurement.” Further, two subtypes were defined: integrative biomarkers (“a single calculated value derived from a defined set of biomarkers and a known algorithm”) and multiplex biomarkers (“multiple single biomarkers used individually or for pattern recognition”). Dr Wagner proposed a possible third subtype, classification biomarkers (“using multiple biomarkers and a defined algorithm to create an interpretation or categorization”) but suggested it may also be considered as a subtype of integrative or multiplex biomarkers. He concluded by noting four potential MCB-associated issues: (1) multiple measures may challenge their analytical validation, (2) interpretation of one biomarker may have to be made in a particular category or context of another measure, (3) difficulty in determining the MCB measure's sufficiency, and (4) complications posed by a measure with characteristics of both biomarkers and clinical outcomes.

Dr Patrick Bossuyt discussed applications and challenges posed by biomarkers and MCBs related to validation and acceptance for clinical use. Biomarker evaluation has traditionally been largely dependent on analytical validation (e.g., sensitivity, specificity, accuracy and precision), clinical validation (e.g., clinical experience demonstrating acceptable identification, measurement and/or prediction of outcomes) and clinical utility or benefit (i.e., empirical evidence demonstrating that a biomarker tool/test can lead to a net improvement in health outcomes or provide useful information about diagnosis, treatment, management or prevention of disease). Dr Bossuyt pointed out that the classical essentialist view of a biomarker's value, which is based on truth, results, validity and accuracy, contrasts with the consequentialist view that is based on consequences, outcomes, utility and effects. Consequentialist views on biomarker development focus on clinical benefit to guide use recommendations, not on the validity of the multimarker score construction. However, the evaluation of clinical benefit is rarely contextualized and comparative. Dr Bossuyt concluded that evaluating complex MCB scores may be challenging because scientific validity is sometimes limited. In addition, clinical performance may not provide sufficient evidence of biological plausibility, and a lack of contextualized analytical and clinical validation hinders the determination of true clinical benefit.

Dr Jeffrey Siegel provided a regulatory perspective on biomarkers as drug development tools. Data supporting biomarkers in regulatory drug applications to the FDA's Center for Drug Evaluation and Research can come from three sources: regulatory submissions such as an investigational new drug or new drug application, the Biomarker Qualification Program and, finally, scientific and community consensus and acceptance (Supplementary Appendix 2). The three are not mutually exclusive and evidence generated within any program can contribute to the body of evidence to support the proposed use for a biomarker and its context of use. The three-stage process [3] to qualify a biomarker through the Center for Drug Evaluation and Research Biomarker Qualification Program was described and consists of a Letter of Intent, Qualification Plan, and Full Qualification Package. An example of a successful qualification by the Biomarker Qualification Program was provided for the total kidney volume biomarker, which was qualified [4] for use as a prognostic biomarker. After its qualification, the use of this biomarker in regulatory submissions has increased, generating additional data to support its use as a reasonably likely surrogate endpoint for accelerated approval. MCBs undergo largely the same development processes as single-component biomarkers, with additional considerations made regarding their composite nature. The benefits of using an MCB compared with a single biomarker should be reasoned, and the importance of each component of the MCB to the whole should be investigated – a process including a systematic approach to selection or deselection of individual biomarkers comprising the MCB. Finally, causality will likely be more difficult to determine when studying an MCB. MCB development poses unique challenges, and the workshop proposed several approaches to address differences in biomarker terminology frameworks to improve the MCB development process.

Summary of panel discussions

Panel session 1

Understanding the significance of single biomarkers to a disease state in the context of drug development and outcome of interest is often difficult. Combining multiple biomarkers into an MCB compounds these complexities. Understanding biomarker significance is further confounded by factors such as uncharacterized diseases and drug mechanisms of action, the independent and multidirectional change of biomarker components, the arbitrary nature of the units used and an absence of well-defined scale cutoffs. These confounding factors make MCBs difficult to assess and may mask the individual contributions of each biomarker component. Further challenges are posed by the necessity of harmonizing units, weighting components, setting baseline values for assessments of MCBs and understanding how the values move along a continuous scale. In some instances, varying perspectives on identification of biomarkers add to the challenge. For example, perspectives may differ on whether each MCB subcomponent functions as its own biomarker or whether all the features together are identified as a single MCB. Ambiguity arises in complex biomarkers such as microRNAs or other such markers that include subcomponents.

Typically, a disease- and drug-specific, or context-specific, investigation is conducted on a given biomarker. The role of the biomarker, such as a predictive or response biomarker, should be considered in relation to the specific disease in the context of treatment. This includes evaluation of the onset and progression of the disease and the drug-specific mechanism of action and effects. Exploring subcomponents of the potential biomarker is essential when developing an MCB. Assessment of assay performance of MCBs with subcomponents integrated within a single panel or index value with arbitrary units remains challenging. Selection and exclusion of the potential candidate components play a crucial role in the biomarker development process and contribute to disparity in the ultimate MCB features and weighting. The challenges associated with MCB component selection and algorithm tuning highlight the importance of stability in biomarker component selection and algorithm design during MCB validation to ensure interpretable results.

To overcome the challenges posed by characterization and validation of complex MCBs, organizations have formed to investigate the comparability of different multicomponent tests designed to measure or evaluate a given pathophysiologic characteristic. These groups attempt to establish standardized definitions and terminology in a disease-specific context. One such group is the Friends of Cancer Research, which works in the space of homologous recombination deficiency. The Friends of Cancer Research has worked to establish standardization in the face of diverse test devices, performance metrics and biologic signatures for homologous recombination deficiency MCBs. Differences in terminology and testing methods can impact comparability in data, potentially cause discordant test results and may play a role in the interpretation of metric sensitivity and specificity. However, head-to-head comparisons between biomarkers and outcome measures are often not readily available. Furthermore, the need for practical clinical availability may outweigh the community's need to harmonize definitions and context.

Panel session 2

In this panel, the development of fluid MCBs related to the progression of nonalcoholic steatohepatitis (NASH) and MCBs associated with tobacco use were discussed.

Nonalcoholic fatty liver disease is the most common hepatic pathology, affects about 25% of the general population and includes several pathologic conditions, ranging from simple steatosis to NASH [5]. Because patients living with NASH have a higher probability of developing cirrhosis and dying from cardiovascular or other liver-related causes, it is important to identify noninvasive biomarkers to differentiate NASH and fibrosis patients early among patients living with nonalcoholic fatty liver disease. Signaling pathways related to NASH development include apoptosis, oxidative stress, inflammation networks and adiponectin-mediated signals. Fibrosis biomarkers include molecules directly involved in fibrogenesis and/or fibrinolysis.

The Enhanced Liver Fibrosis (ELF) [5–7] can be considered an MCB. The ELF output is a composite score of three biomarkers believed to be related to fibrosis in human serum: hyaluronic acid, amino-terminal pro-peptide of type III procollagen (PIINP) and tissue inhibitor of matrix metalloproteinase (TIMP) 1. The three component tests measure molecules that are involved in the synthesis and degradation of the extracellular matrix during liver fibrogenesis [5]. The clinical data provided by the device's sponsor demonstrated that the ELF was correlated with progression of the condition in two populations. The ELF is a prognostic biomarker that can be used in conjunction with other laboratory findings and clinical assessments for progression to cirrhosis in NASH patients who had advanced fibrosis but were not yet cirrhotic at the time of the measurement of the ELF. Additionally, it can be used to track progression to specific clinical outcomes in NASH patients who were already cirrhotic at the time of the measurement of the ELF.

Regarding biomarkers associated with the use of tobacco products (e.g., combusted cigarettes, smokeless tobacco, electronic nicotine delivery systems), it is well established that tobacco products expose their users to a wide range of toxic compounds. In addition, individuals who inhale secondhand smoke from combusted tobacco products are also exposed to some of these toxic compounds. Entering the human body, most tobacco product constituents undergo metabolic changes, and the components can be quantified in bioliquids as parent compounds or metabolites and serve as biomarkers of exposure. Exposure to tobacco product constituents further creates a subsequent cascade of biological processes resulting in the production of multiple responses, including early biological effects; alterations in cell morphology, structure or function; and clinical symptoms. To measure biological effects consistent with harm due to tobacco product exposure, multiple biomarkers of potential harm are utilized [2,8]. Therefore, in tobacco products research, it is feasible to evaluate the relationship of not only a single biomarker but also sets of MCBs to the development of diseases associated with tobacco use. Examples of MCBs for tobacco products include sets of biomarkers of exposure such as metabolites of polycyclic aromatic hydrocarbons, tobacco-specific nitrosamines, volatile organic compounds or a combination of some or all associated biomarkers. In addition, the various sets of multicomponent biomarkers of exposure may be used to distinguish among users of combusted cigarettes, smokeless tobacco and electronic nicotine delivery systems. The sets of multicomponent biomarkers of potential harm can reveal associations between tobacco product use and oxidative stress, inflammation and other potential health risks, for which there may be a lack of longitudinal epidemiologic substantiation. High-sensitivity C-reactive protein (CRP), interleukin-6 (IL-6), fibrinogen, soluble intercellular adhesion molecule-1 (sICAM-1) and an oxidative stress biomarker (F2-isoprostane) are biomarkers of potential harm used as pharmacodynamic biomarkers to evaluate inflammation and related downstream biological effects associated with former or current use of tobacco products [8]. The multiple mechanisms by which tobacco causes various diseases may be more strongly associated with MCBs than with single biomarkers [9].

Panel session 3

The panel highlighted the need to separate the concept of the biomarker from its measurement modality. Biomarker measurement is an important aspect of biomarker use but does not define the biomarker's context of use, which is comprised of the BEST biomarker category and the drug development use. Nevertheless, the modality used to measure a biomarker influences various aspects of its development and use. This is especially true for MCBs because each component may be measured by different modalities.

The panel discussed considerations for MCBs that combine distinct measurement modalities, such as the iBox Scoring System, which combines molecular assays and histology. In such cases, the panel noted the importance of linking each measurement individually to the clinical outcome of interest and establishing biological plausibility. For example, the iBox Scoring System measurement reflects both kidney health and allograft function. However, combining continuous measurements (e.g., estimated glomerular filtration rate and proteinuria), ordinal measurements (e.g., histopathology using Banff lesions) and dichotomous measurements (e.g., donor specific antibody mean fluorescence intensity, which is a qualitative binary measurement) into a single MCB end point could make establishing these links a challenge. It is best to determine the reasonable description for each component of the MCB early during development. For example, inflammation can be defined either radiographically or histologically, so it is important to establish and state which definition will be used for the interpretation, alongside the analytical methods, and reasons leading to this choice. Finally, there is a need to establish the best modality to assess the intended biomarkers. For example, in the case of the prognostic biomarker being developed for the enrichment/identification of subjects with knee osteoarthritis, cartilage and bone morphology may be better assessed through imaging methods, whereas cartilage degradation may be better measured by a biochemical assay.

For MCBs, an integrative approach may be taken for biomarker development depending on the biomarker use and category. A good example of this is the development of a prognostic MCB score that has a continuous relationship with an outcome used for stratification or enrichment to create categories in clinical trials. For such a continuous MCB to be used as a drug development tool, a robust model can be built when relevant datasets are available. Drug developers can then use this model to run simulations at multiple time points to help define trial-specific and drug-specific enrichment categories without the need to lock in a threshold from the beginning.

The panel noted several considerations when dealing with the complexities of developing MCBs using artificial intelligence (AI) and machine learning (ML). The panel pointed out that when discussing the role of AI/ML in imaging biomarkers versus drug development tools, we may want to consider that in imaging biomarkers, the acquired data are processed to form a digital image that is interpretable and linked to patient-level information, while for a drug development biomarker, the focus is at the population level in the context of clinical trials, for either patient selection, enrichment, stratification or other considerations of the longitudinal aspect of the trial. The panel noted that during the process of optimizing patient selection for clinical trials through biomarkers of multiple types, careful consideration be taken when using AI-based analysis to support the proposed context of use. Though an AI approach can mimic established methods, making these methods more efficient and less human-intensive may not necessarily provide much evidence about the biomarker itself.

The panel expressed concerns about the potential black-box nature of the AI models and the fact that for some of these models, even if the outcomes appeared useful, it was difficult to understand which components contributed to the outcome of interest and what these components meant in terms of biology, physiology or pathology. The panel highlighted the joint work by the FDA, Health Canada and the UK's Medicines and Healthcare products Regulatory Agency in identifying ten guiding principles that are intended to lay the foundation for developing Good Machine Learning Practice [10]. They specifically highlighted the importance of clinical study participants and datasets being representative of the intended patient population, that training and test datasets are selected and maintained to be appropriately independent of one another and that selected reference datasets are based upon best available methods. From the perspective of devices, the panel also pointed out that how a threshold is chosen and validated is just as important as the threshold itself. Once determined, the threshold should remain constant during clinical and nonclinical validation to ensure confidence in the validation. In addition to the intended use of the product, the panel indicated that it would be useful to get a description of the algorithm and of the datasets that were being used for the training, testing and validation of the algorithm; having the validation dataset be independent of the training and testing datasets was also considered to be critical in understanding the performance range of the algorithm.

Panel session 4

The moderator began the final panel session by noting that the terms for an MCB used in the case studies during the 2-day workshop were ‘panel’, ‘signature’, ‘score’, ‘multivariate model’ and ‘index’. As background, the moderator listed the following aspects used to describe how an MCB was developed: nature of the multiple inputs, combining the multiple inputs, parallel versus sequential and methods for combining the parallel inputs versus developing the model (e.g., ML algorithm from many components and regression modeling with a small number of components). The panelists were then invited to opine on three prepared questions related to concerns raised during the background presentation and the previous panel discussions (Supplementary Table 3).

The first question posed to the panelists related to determining whether an MCB was properly developed. One panelist noted that an MCB is often perceived to be any signature or any profile that shows some evidence of clinical performance. However, providing multiple genes, for example, is not sufficient to adequately define an MCB. A critical issue is reproducibility, as different bioinformatic pipelines can result in different outputs of an MCB. Three key issues to properly develop an MCB were further elaborated: difficulty in adequately defining an MCB; when an MCB is adequately defined, properly developing and validating the MCB; and during development or post development, needing to move one or more assays for the MCB to a different assay platform.

Another panelist observed that there was an enormous range of strategies presented at the workshop. There were also classical combinations of single biomarkers for which we understood the biological variability and had proper assays to measure individually. When these single biomarkers are combined into a score, the issue is how the score identifies patients, because a score can be derived from a handful of components built up by very different measurements on single individual components. Usually, the justification of a score comes from its clinical performance. However, the biomarker assay is intimately tied to biomarker use. Developing an MCB assay can become much more challenging than doing so for a single biomarker assay when evidence is needed for clinical use or clinical trials. An MCB often may not be fully defined because insufficient details were provided about the assay and algorithm.

The second set of questions posed to the panelists related to challenges in properly developing an MCB (Supplementary Table 4).

Two major challenges of arriving at a properly developed MCB were mentioned by multiple panelists. One challenge is knowing how credible or scientifically valid the MCB inputs to the algorithm and readout are for performance. The robustness in the methodology for developing an MCB is not sufficient to demonstrate the MCB's scientific validity or the consequences of using it. Scientific validity includes an understanding about the biological variability of the components in the MCB, the multivariate combination of the components of the MCB and the measurement properties of the components in the MCB. Well-defined MCBs are important to identify the appropriate evidence relevant to support the use of an MCB for drug development or for clinical use. The classical issues that have been addressed with single biomarkers in terms of analytical validation are equally important and applicable to the components of an MCB. The second challenge is not being able to reproduce model performance, including AI/ML, because the set of samples or the group of patients included in the study is not clearly defined clinically or there are built-in biases, such as convenience samples. Consequently, in addition to the construction of the MCB, well-defined terminology describes the underlying scientific evidence about the MCB. Two panelists noted that what we decide on as far as the terminology is not as important as making sure that we clearly describe the MCB and the terms we are using.

A panelist noted that an MCB in genomics studies may run into an overfitting problem. To understand the analytical performance of an MCB with several components (e.g., an MCB described as a biomarker score), it is important to know the limit of detection, the limit of quantitation, linearity and precision. For precision, suppose immunohistochemistry is a method for measuring HER2. A subject is considered HER2 positive if the immunohistochemistry value is more than two plus, and HER2 negative otherwise. There are differences in the classification of HER2 positivity among the laboratories and countries; this panelist considered this phenomenon as an example of variability in biomarker measurement interfering with the determination of biological variability. It was noted that for every random measurement, care needs to be taken to prevent precision from being so high as to obscure the true biological variability. Analytical performance can be as impactful as clinical performance. For example, poor analytical performance can adversely affect clinical performance because it adds noise to weaken association with clinical outcome.

Panel 4 finished the session addressing the question of terminology for MCBs. The discussion focused on the proposed definition of an MCB given at the keynote session. It was advised that more clarity around MCB terminology is needed. To that end, the keynote speech elaborated on three types or groups of MCBs. For the category of integrative MCBs, a single summary comes out of multiple channels (similar to a composite endpoint), such as a score (e.g., iBox) and an index. Multiplex MCBs, a second category, function differently by focusing on pattern recognition or individual use for multiple biomarkers, such as homologous recombination deficiency. It was noted that further subcategorization could be proposed for different signatures, but whether these are important distinctions still needs to be determined. The third category, classification MCBs, is defined by its use or purpose, which, as the name suggests, is to classify, diagnose or categorize individuals based on the biomarker. Further discussion as to whether classification MCB represents a third type of MCB or a subcategory of either integrative or multiplex biomarkers was initially left as an open question.

In discussing the terms more deeply, it was noted that the terms ‘integrative’ and ‘multiplex’ describe or summarize the underlying algorithms – in essence, the terms inform how the MCB is assembled. The various inputs are combined in a particular way, and an output is produced. Each output is then used for a particular objective, such as classification by diagnosing a disease or condition. Based on the dichotomy between algorithm-based definitions and the purpose-based definition, the classification MCB could fall under a subcategory of either the integrative or multiplex composite biomarker, or both. A hierarchical method of defining MCBs was suggested by a panelist and was considered as preferred by another panelist; however, time did not permit elaboration on the form or structure that a hierarchical approach would entail.

The panel concluded by reiterating that as part of the clinical validation of a proposed MCB, both the exploratory and confirmatory validation data are important to the biomarker's correlation and utility for its proposed use. Although time was lacking for further discussion on terminology, the following recommendations provide additional terms and hierarchical structures that could aid in the discussion and research of MCBs.

The terms used in the case examples were varied but could fit under integrative and multiplex MCBs to further define the type of algorithm used to determine the MCB output. One possible hierarchical structure would be the following:

• score, ratio and index as types of integrative MCB;

• panel, signature and pattern as types of multiplex MCB.

These terms need definitions. Some might be more difficult to define precisely or may represent the same concept, such as ‘signature’ or ‘pattern’. Overall, these terms should be interpreted in view of each MCB's composition and intended clinical utility.

The Biomarker Working Group initially proposed to define MCBs as biomarkers that:

• include two or more features potentially including clinical characteristics such as patient demographics;

• are used independently or in combination through an algorithm;

• represent one or more defined characteristics indicating normal biological processes, pathogenic processes or responses to an exposure or intervention, including therapeutic interventions and environmental exposures.

The Biomarker Working Group used the term ‘features’ to represent ‘data sources’ because an MCB may include items that are not ‘defined characteristics’. The Biomarker Working Group has subsequently decided to exclude clinical characteristics or outcomes from the definition. Reflecting the comments received during the workshop, the Biomarker Working Group now proposes that an MCB be defined as a characteristic or set of characteristics derived from two or more data sources evaluated through an algorithm as an indicator of normal biological processes, pathogenic processes or responses to an exposure or intervention, including therapeutic interventions and environmental exposures.

Conclusion

Several proposed terms regarding the terminology of MCBs were presented during the workshop. The prepared case studies highlighted important applications of MCBs and generated discussion about the proper development of MCBs and the relationship between analytical performance and clinical performance. Although further work is needed to develop terminology related to MCBs, the workshop helped advance the field of biomarkers in this regard. One concrete step that would assist this process would be to pursue the inclusion of MCB terminology in the BEST glossary.

In addition to terminology, several challenges and concerns were raised regarding the development of MCBs. Although commonalities with single biomarker development were noted, MCB development presents increased challenges due to their multicomponent and often diverse nature and, in some instances, the lack of clear definition or characterization of critical aspects of the MCB. Understanding the contribution of an individual component to the MCB may help interpretation of the MCB; however, the relationship to validation expectations remains uncertain. Particularly, it may not be obvious whether it is more important to emphasize the assessment of the individual components of an MCB or the overall combined output of components for variability in their measurements, such as limit of detection, linearity and other analytical parameters. Future work may help further elucidate the challenges related to MCB development and provide helpful insights.