Research Article

Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement

Patricia A Deverka

* Author for correspondence

Center for Medical Technology Policy, 401 East Pratt Street, Suite 631, Baltimore, MD 21202-3117, USA.

Search for more papers by this author

Email the corresponding author at pat.deverka@cmtpnet.org

Danielle C Lavallee

Center for Medical Technology Policy, 401 East Pratt Street, Suite 631, Baltimore, MD 21202-3117, USA.

Search for more papers by this author

Priyanka J Desai

Center for Medical Technology Policy, 401 East Pratt Street, Suite 631, Baltimore, MD 21202-3117, USA.

Search for more papers by this author

Laura C Esmail

Center for Medical Technology Policy, 401 East Pratt Street, Suite 631, Baltimore, MD 21202-3117, USA.

Search for more papers by this author

Scott D Ramsey

Fred Hutchinson Cancer Research Center, University of Washington, 1100 Fairview Avenue, North Seattle, WA 98109-4433, USA

Search for more papers by this author

David L Veenstra

University of Washington, School of Pharmacy, 1959 NE Pacific Street, H362 Health Sciences Building, Seattle, WA 98195-7631, USA

Search for more papers by this author

Sean R Tunis

Center for Medical Technology Policy, 401 East Pratt Street, Suite 631, Baltimore, MD 21202-3117, USA.

Search for more papers by this author

Published Online:5 Mar 2012https://doi.org/10.2217/cer.12.7

View article

Abstract

Aims: Stakeholder engagement is fundamental to comparative effectiveness research (CER), but lacks consistent terminology. This paper aims to define stakeholder engagement and present a conceptual model for involving stakeholders in CER. Materials & methods: The definitions and model were developed from a literature search, expert input and experience with the Center for Comparative Effectiveness Research in Cancer Genomics, a proof-of-concept platform for stakeholder involvement in priority setting and CER study design. Results: Definitions for stakeholder and stakeholder engagement reflect the target constituencies and their role in CER. The ‘analytic-deliberative’ conceptual model for stakeholder engagement illustrates the inputs, methods and outputs relevant to CER. The model differentiates methods at each stage of the project; depicts the relationship between components; and identifies outcome measures for evaluation of the process. Conclusion: While the definitions and model require testing before being broadly adopted, they are an important foundational step and will be useful for investigators, funders and stakeholder groups interested in contributing to CER.

Keywords:

Papers of special note have been highlighted as: ▪ of interest ▪▪ of considerable interest

References

1 Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann. Intern. Med.151(3),203–205 (2009).Crossref, Medline, Google Scholar
2 Saunders C, Crossing S, Girgis A, Butow P, Penman A. Operationalising a model framework for consumer and community participation in health and medical research. Aust. New Zealand Health Policy4(1),13 (2007).Crossref, Medline, Google Scholar
3 Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda. Health Policy61(2),213–236 (2002).▪▪ Presents discussion of terminology and rationale relating to ‘stakeholder’ and ‘stakeholder involvement’ with an emphasis on patient and consumer involvement.Crossref, Medline, Google Scholar
4 Conway PH, Clancy C. Comparative-effectiveness research – implications of the Federal Coordinating Council’s report. N. Engl. J. Med.361(4),328–330 (2009).Crossref, Medline, CAS, Google Scholar
5 Tunis SR, Benner J, McClellan M. Comparative effectiveness research: policy context, methods development and research infrastructure. Stat. Med.29(19),1963–1976 (2010).Crossref, Medline, Google Scholar
6 Hoffman A, Montgomery R, Aubry W, Tunis SR. How best to engage patients, doctors, and other stakeholders in designing comparative effectiveness studies. Health Aff.29(10),1834–1841 (2010).▪▪ Presents case studies of stakeholder engagement in comparative effectiveness research (CER) identifying principles of successful practice.Crossref, Medline, Google Scholar
7 Keown K, Van Eerd D, Irvin E. Stakeholder engagement opportunities in systematic reviews: knowledge transfer for policy and practice. J. Contin. Educ. Health Prof.28(2),67–72 (2008).Crossref, Medline, Google Scholar
8 Mitton C, Smith N, Peacock S, Evoy B, Abelson J. Public participation in health care priority setting: a scoping review. Health Policy91(3),219–228 (2009).Crossref, Medline, Google Scholar
9 Noorani HZ, Husereau DR, Boudreau R, Skidmore B. Priority setting for health technology assessments: a systematic review of current practical approaches. Int. J. Technol. Assess. Health Care23(3),310–315 (2007).▪ Reviews practices in priority setting that are adaptable to CER.Crossref, Medline, Google Scholar
10 Ryan M, Scott DA, Reeves C et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol. Assess.5(5),1–186 (2001).▪ Discusses several methodologies key to involving stakeholders in CER.Crossref, Medline, CAS, Google Scholar
11 Boote J, Baird W, Sutton A. Public involvement in the design and conduct of clinical trials: a narrative review of case examples. Trials12(Suppl. 1),A82 (2011).Crossref, Medline, Google Scholar
12 Boote J, Baird W, Sutton A. Public involvement in the systematic review process in health and social care: a narrative review of case examples. Health Policy102(2),105–116 (2011).Crossref, Medline, Google Scholar
13 Abelson J, Forest P-G, Eyles J, Smith P, Martin E, Gauvin F-P. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Soc. Sci. Med.57(2),239–251 (2003).Crossref, Medline, Google Scholar
14 Abelson J, Giacomini M, Lehoux P, Gauvin F-P. Bringing ‘the public’ into health technology assessment and coverage policy decisions: from principles to practice. Health Policy82(1),37–50 (2007).Crossref, Medline, Google Scholar
15 Hailey D, Nordwall M. Survey on the involvement of consumers in health technology assessment programs. Int. J. Technol. Assess. Health Care22(4),497–499 (2006).Crossref, Medline, Google Scholar
16 Oels A. Evaluating Stakeholder Dialogs: Stakeholder Dialogs in Natural Resources Management. Stollkleemann S, Welp M (Eds). Springer, Berlin Heidelberg, Germany, 117–151 (2006).Google Scholar
17 Pedersen ER. Making corporate social responsibility (CSR) operable: how companies translate stakeholder dialog into practice. Bus. Society Rev.111(2),137–163 (2006).Crossref, Google Scholar
18 Reed MS. Stakeholder participation for environmental management: a literature review. Biol. Conserv.141(10),2417–2431 (2008).Crossref, Google Scholar
19 Tharani R, Wong W, Carlson J et al. Prioritization in comparative effectiveness research: the CANCERGEN experience in cancer genomics. Med. Care doi:10.1097/MLR.0b013e3182422a3b (2012) (Epub ahead of print).Google Scholar
20 Burton H, Adams M, Bunton R, Schroder-Back P. Developing stakeholder involvement for introducing public health genomics into public policy. Public Health Genomics12,11–9 (2008).Crossref, Medline, Google Scholar
21 Brugha R, Varvasovszky Z. Stakeholder analysis: a review. Health Policy Plan.15(3),239–246 (2000).Crossref, Medline, CAS, Google Scholar
22 Elwyn G, Crowe S, Fenton M et al. Identifying and prioritizing uncertainties: patient and clinician engagement in the identification of research questions. J. Eval. Clin. Prac.16(3),627–631 (2010).Medline, Google Scholar
23 Williamson C. What does involving consumers in research mean? QJM94(12),661–664 (2001).Crossref, Medline, CAS, Google Scholar
24 Oliver SR, Rees RW, Clarke-Jones L et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect.11(1),72–84 (2008).Crossref, Medline, Google Scholar
25 Boote J, Barber R, Cooper C. Principles and indicators of successful consumer involvement in NHS research: results of a Delphi study and subgroup analysis. Health Policy75(3),280–297 (2006).Crossref, Medline, Google Scholar
26 Sibbald S, Singer P, Upshur R, Martin D. Priority setting: what constitutes success? A conceptual framework for successful priority setting. BMC Health Serv. Res.9(1),43 (2009).Crossref, Medline, Google Scholar
27 Barber R, Boote JD, Parry GD, Cooper CL, Yeeles P, Cook S. Can the impact of public involvement on research be evaluated? A mixed methods study. Health Expect. doi:10.1111/j.1369-7625.2010.00660.x (2011) (Epub ahead of print).Medline, Google Scholar
28 Lloyd K, White J. Democratizing clinical research. Nature474(7351),277–278 (2011).Crossref, Medline, CAS, Google Scholar
29 Cowan K. The James Lind alliance: tackling treatment uncertainties together. J. Ambul. Care Manage.33(3),241–248 (2010).Crossref, Medline, Google Scholar
30 Tunis S, Korn A. The role of payers in the clinical research enterprise. In: The Role of Purchasers and Payers in the Clinical Research Enterprise: Workshop Summary. Institute of Medicine, National Academies Press, Washington, DC, USA (2002).Google Scholar
31 Bogart LM, Uyeda K. Community-based participatory research: partnering with communities for effective and sustainable behavioral health interventions. Health Psychol.28(4),391–393 (2009).Crossref, Medline, Google Scholar
32 Shalowitz MU, Isacco A, Barquin N et al. Community-based participatory research: a review of the literature with strategies for community engagement. J. Dev. Behav. Pediatr.30(4),350–361 (2009).Crossref, Medline, Google Scholar
33 Rowe G, Frewer LJ. A typology of public engagement mechanisms. Sci. Technol. Human Values30(2),251–290 (2005).Crossref, Google Scholar
34 Stern PC, Feinberg HV. Understanding Risk: Informing Decisions in a Democratic Society. National Research Council, Committee on Risk Characterization, National Academy Press, Washington, DC, USA (1996).Google Scholar
35 Renn O. A model for an analytic – deliberative process in risk management. Environ. Sci. Technol.33(18),3049–3055 (1999).Crossref, CAS, Google Scholar
36 Weinberg M. The role of other stakeholders in the clinical research enterprise. In: The Role of Purchasers and Payers in the Clinical Research Enterprise: Workshop Summary. Institute of Medicine, National Academies Press, Washington, DC, USA (2002).Google Scholar
37 Fearon JD. Deliberation as discussion. In: Deliberative Democracy. Elster J (Ed.). Cambridge University Press, NY, USA, 44–69 (1998).Google Scholar
38 Daniels N, Sabin J. Limits to health care: fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philos. Public Aff.26(4),303–350 (1997).Crossref, Medline, Google Scholar
39 Amaeshi KM, Crane A. Stakeholder engagement: a mechanism for sustainable aviation. Corp. Soc. Respons. Environ. Manage.13(5),245–260 (2006).Crossref, Google Scholar
40 Apostolakis GE, Pickett SE. Deliberation: integrating analytical results into environmental decisions involving multiple stakeholders. Risk Analysis18(5),621–634 (1998).Crossref, Google Scholar
41 Beierle TC, Konisky DM. Values, conflict, and trust in participatory environmental planning. J. Policy Anal. Manage.19(4),587–602 (2000).Crossref, Google Scholar
42 Carnes SA, Schweitzer M, Peelle EB, Wolfe AK, Munro JF. Measuring the success of public participation on environmental restoration and waste management activities in the US Department of Energy. Technol. Soc.20(4),385–406 (1998).Crossref, Google Scholar
43 Caron-Flinterman JF, Broerse JEW, Teerling J et al. Stakeholder participation in health research agenda setting: the case of asthma and COPD research in The Netherlands. Sci. Pub. Policy33(4),291–304 (2006).Crossref, Google Scholar
44 Halvorsen KE. Assessing public participation techniques for comfort, convenience, satisfaction, and deliberation. Environ. Manage.28(2),179–186 (2001).Crossref, Medline, CAS, Google Scholar
45 Laurian L, Shaw MM. Evaluation of public participation. J. Plan. Educ. Res.28(3),293–309 (2009).Crossref, Google Scholar
46 Webler T, Tuler S. Fairness and competence in citizen participation. Admin. Soc.32(5),566–595 (2000).Crossref, Google Scholar
47 Webler T, Tuler S, Krueger R. What is a good public participation process? Five perspectives from the public. Environ. Manage.27(3),435–450 (2001).Crossref, Medline, CAS, Google Scholar
48 Jefford M, Stockler MR, Tattersall MH. Outcomes research: what is it and why does it matter? Intern. Med. J.33(3),110–118 (2003).Crossref, Medline, CAS, Google Scholar
49 Clancy C, Collins FS. Patient-centered outcomes research institute: the intersection of science and health care. Sci. Transl. Med.2(37),37cm18 (2010).Crossref, Medline, Google Scholar
50 Sox HC. Comparative effectiveness research: a progress report. Ann. Intern. Med.153(7),469–472 (2010).Crossref, Medline, Google Scholar
51 Chalkidou K, Tunis S, Lopert R et al. Comparative effectiveness research and evidence-based health policy: experience from four countries. Milbank Quarterly87(2),339–367 (2009).Crossref, Medline, Google Scholar
52 Nass SJ, Moses HL, Mendelsohn J. A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program. National Academies Press, Washington, DC, USA (2010).Google Scholar
101 NIH. Project information: 5UC2CA148570–02, Center for Comparative Effectiveness Research in Cancer Genomics (CANCERGEN). http://projectreporter.nih.gov/project_info_description.fm?aid=7944022&icde=5662445 (Accessed 15 September 2011)Google Scholar
102 Gliklich R, Leavy M, Velentgas P et al. Identification of future research needs in the comparative management of uterine fibroid disease. www.effectivehealthcare.ahrq.gov/ehc/products/152/642/DEcIDE31_UterineFibroid_03-07-2011.pdf (Accessed 15 September 2011)Google Scholar
103 O’Haire C, McPheeters M, Nakamoto E et al. Engaging stakeholders to identify and prioritize future research needs. Methods future research needs report no 4. www.effectivehealthcare.ahrq.gov/reports/final.cfm (Accessed 15 September 2011)▪▪ Documents experiences of stakeholder engagement for the purposes of identifying and prioritizing future research needs.Google Scholar
104 Preskill H, Jones N. A practical guide for engaging stakeholders in developing evaluation questions. RWFJ Evaluation Series. www.rwjf.org/files/research/49951.stakeholders.final.1.pdf (Accessed 15 September 2011)Google Scholar
105 Buckland S, Hayes H, Ostrer C et al. Public information pack (PIP). Involve support unit. www.invo.org.uk/wp-content/uploads/2011/12/PIP1whatisitallabout.pdf (Accessed 11 January 2012)▪▪ Document developed by INVOLVE (UK) that distinguishes between levels of patient involvement including consultation, collaboration and user control.Google Scholar
106 Carlson JJ, Thariani R, Roth J et al. Value of research analyses in research prioritization of cancer genomic applications. AcademyHealth Annual Research Meeting. www.academyhealth.org/files/ARM/2011/PosterPresentations.pdf (Accessed 15 September 2011)Google Scholar
107 Esmail L, Roth J, Rangarao S et al. What factors do stakeholders consider in research prioritization? A qualitative analysis in cancer genomics. AcademyHealth Annual Research Meeting. www.academyhealth.org/files/ARM/2011/PosterPresentations.pdf (Accessed 15 September 2011)Google Scholar

Vol. 1, No. 2

Metrics

Downloaded 702 times

History

Published online 5 March 2012

Published in print March 2012

Information

Keywords

Acknowledgements

The authors gratefully acknowledge the contributions of the External Stakeholder Advisory Group who helped inform portions of the manuscript. They would also like to acknowledge the review and comments of our colleague E Tambor and thank her for her thoughtful input.

Financial & competing interests disclosure

This work was partially funded by the Center for Comparative Effectiveness Research in Cancer Genomics through the American Recovery and Reinvestment Act of 2009 by the National Cancer Institute, NIH under Agency Award # 5UC2CA148570-02. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

Ethical conduct of research

The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.

PDF download