Review

Stakeholder engagement in research: a scoping review of current evaluation methods

*Author for correspondence: Tel.: +1 215 503 8011;

E-mail Address: jenny.martinez@jefferson.edu

Department of Occupational Therapy, Jefferson College of Rehabilitation Sciences, Thomas Jefferson University, Philadelphia, PA 19107, USA

Search for more papers by this author

Carin Wong

Mrs TH Chan Division of Occupational Science & Occupational Therapy, Herman Ostrow School of Dentistry, University of Southern California, Los Angeles, CA 90089, USA

Search for more papers by this author

Catherine Verrier Piersol

Department of Occupational Therapy, Jefferson College of Rehabilitation Sciences, Thomas Jefferson University, Philadelphia, PA 19107, USA

Search for more papers by this author

Dawn Clayton Bieber

Genesis Rehab Services, Kennett Square, PA 19348, USA

Search for more papers by this author

Bonita L Perry

Communication Department, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104, USA

Search for more papers by this author

Natalie E Leland

Department of Occupational Therapy, School of Health & Rehabilitation Sciences, University of Pittsburgh, Pittsburgh, PA 15260, USA

Search for more papers by this author

Published Online:18 Nov 2019https://doi.org/10.2217/cer-2019-0047

View article

Abstract

Aim: Evaluating stakeholder engagement can capture what meaningful engagement in research entails, how it develops, and how it is experienced by all collaborators. We conducted a scoping review of recent approaches for evaluating engagement in research and present a descriptive overview of our findings. Methods: We searched peer-reviewed journal articles published worldwide in English between January 2013 and June 2018. Results: Our final sample consisted of 17 articles. Various approaches for evaluating stakeholder engagement were identified including qualitative approaches, surveys and engagement logs. Discussion & conclusion: We identified evaluation approaches that varied in quality, detail and methods. Valid, systematic and inclusive approaches that are developed with research partners and are inclusive of diverse perspectives are an important area for future research.

Keywords:

Papers of special note have been highlighted as: • of interest; •• of considerable interest

References

1. Morris ZS , Wooding S , Grant J . The answer is 17 years, what is the question: understanding time lags in translational research. J. Royal Soc. Med. 104(12), 510–520 (2011). Crossref, Medline, Google Scholar
2. McGlynn EA , Asch SM , Adams J et al. The quality of health care delivered to adults in the United States. N. Engl. J. Med. 348(26), 2635–2645 (2003). Crossref, Medline, Google Scholar
3. Boaz A , Hanney S , Borst R et al. How to engage stakeholders in research: design principles to support improvement. Health Res. Policy Syst. 16(60), 1–9 (2018). Medline, Google Scholar
4. Forsythe LP , Frank L , Tafari T et al. Unique review criteria and patient and stakeholder reviewers: analysis of PCORI's approach to research funding. Value Health 21(10), 1152–1160 (2018). Crossref, Medline, Google Scholar
5. Forsythe LP , Frank LB , Workman TA et al. Patient, caregiver and clinician views on engagement in comparative effectiveness research. J. Comp. Eff. Res. 6(3), 231–244 (2017). • Prior study indicating stakeholders’ perspective on the importance of stakeholder engagement in research.Link, Google Scholar
6. Esmail L , Moore E , Rein A . Evaluating patient and stakeholder engagement in research: moving from theory to practice. J. Comp. Eff. Res. 4(2), 133–145 (2015). •• Provides a systematic review that identifies the knowledge gap in methods to engage stakeholders in research.Link, Google Scholar
7. Concannon TW , Fuster M , Saunders T et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J. Gen. Intern. Med. 29(12), 1692–1701 (2014). •• Systematic review that identified the variability in how stakeholder engagement is reported.Crossref, Medline, Google Scholar
8. Bowen DJ , Hyams T , Goodman M et al. Systematic review of quantitative measures of stakeholder engagement. Clin. Transl. Sci. 10(5), 314–336 (2017). Crossref, Medline, CAS, Google Scholar
9. Forsythe LP , Ellis LE , Edmundson L et al. Patient and stakeholder engagement in the PCORI pilot projects: description and lessons learned. J. Gen. Intern. Med. 31(1), 13–21 (2016). • Provides a description of stakeholder engagement in research.Crossref, Medline, Google Scholar
10. Patient-Centered Outcomes Research Institute. PCORI's stakeholders (2018). https://www.pcori.org/about-us/our-programs/engagement/pcoris-stakeholders Google Scholar
11. Sheridan S , Schrandt S , Forsythe L et al. The PCORI Engagement Rubric: promising practices for partnering in research. Ann. Fam. Med. 15(2), 165–170 (2017). Crossref, Medline, Google Scholar
12. Lavallee DC , Williams CJ , Tambor ES et al. Stakeholder engagement in comparative effectiveness research: how will we measure success? J. Comp. Eff. Res. 1(5), 397–407 (2012). Link, Google Scholar
13. Goodman MS , Sanders Thompson VL , Arroyo Johnson C et al. Evaluating community engagement in research: quantitative measure development. J. Commun. Psychol. 45(1), 17–32 (2017). •• Previous study that emphasized the need for investigating stakeholder engagement as a research practice and document its impact through evaluation.Crossref, Medline, Google Scholar
14. Ray KN , Miller E . Strengthening stakeholder-engaged research and research on stakeholder engagement. J. Comp. Eff. Res. 6(4), 375–389 (2017). •• Previous study that identifies a lack of consensus on standards to report stakeholder engagement activities.Link, Google Scholar
15. Patient-Centered Outcomes Research Institute. Our story (2017). https://www.pcori.org/about-us/our-story Google Scholar
16. Boivin A , Richards T , Forsythe L et al. Evaluating patient and public involvement in research. Br. Med. J. 363, k5147 (2018). Crossref, Medline, Google Scholar
17. Kimminau KS , Jernigan C , LeMaster J et al. Patient vs community engagement: emerging issues. Med. Care 56(Suppl. 1), S53–S57 (2018). Crossref, Medline, Google Scholar
18. DeCamp LR , Polk S , Camacho Chrismer M et al. Health care engagement of limited English proficient Latino families: lessons learned from advisory board development. Prog. Community Health Partnersh. 9(4), 521–530 (2015). Crossref, Medline, Google Scholar
19. Arksey H , O’Malley L . Scoping studies: towards a methodological framework. Int. J. Social Res. Methodol. 8(1), 19–32 (2005). Crossref, Google Scholar
20. Patient-Centered Outcomes Research Institute. PCORI in the literature (2017). https://www.pcori.org/literature/research-articles Google Scholar
21. Ellis LE , Kass NE . How are PCORI-funded researchers engaging patients in research and what are the ethical implications? AJOB Empir. Bioeth. 8(1), 1–10 (2017). Crossref, Medline, Google Scholar
22. Moher D , Liberati A , Tetzlaff J et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6(7), e1000097 (2009). Crossref, Medline, Google Scholar
23. Concannon TW , Meissner P , Grunbaum JA et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J. Gen. Intern. Med. 27(8), 985–991 (2012). Crossref, Medline, Google Scholar
24. Brodt A , Norton CK , Kratchman A . So much more than a “pair of brown shoes”: triumphs of patient and other stakeholder engagement in patient-centered outcomes research. Patient Exp. J. 2(1), 43–49 (2015). Crossref, Google Scholar
25. Brookman-Frazee L , Stahmer A , Stadnick N et al. Characterizing the use of research-community partnerships in studies of evidence-based interventions in children's community services. Adm. Policy Ment. Health 43(1), 93–104 (2016). Crossref, Medline, Google Scholar
26. Mackie TI , Sheldrick RC , de Ferranti SD et al. Stakeholders’ perspectives on Stakeholder-Engaged Research (SER): strategies to operationalize patient-centered outcomes research principles for SER. Med. Care 55(1), 19–30 (2017). Crossref, Medline, Google Scholar
27. Morain SR , Whicher DM , Kass NE et al. Deliberative engagement methods for patient-centered outcomes research. Patient 10(5), 545–552 (2017). Crossref, Medline, Google Scholar
28. Morton KL , Atkin AJ , Corder K et al. Engaging stakeholders and target groups in prioritising a public health intervention: the Creating Active School Environments (CASE) online Delphi study. BMJ Open 7(1), e013340 (2017). Crossref, Medline, Google Scholar
29. Greene SM , Brandzel S , Wernli KJ . From principles to practice: real-world patient and stakeholder engagement in breast cancer research. Perm. J. 22, 17–232 (2018). Medline, Google Scholar
30. Chu JT , Chan SS , Stewart SM et al. Exploring community stakeholders’ perceptions of the enhancing family well-being project in Hong Kong: a qualitative study. Front. Public Health 5, 106 (2017). Crossref, Medline, Google Scholar
31. Gesell SB , Potvin Klein K , Halladay J et al. Methods guiding stakeholder engagement in planning a pragmatic study on changing stroke systems of care. J. Clin. Transl. Sci. 1(2), 121–128 (2017). Crossref, Medline, Google Scholar
32. Saunders T , Mackie TI , Shah S et al. Young adult and parent stakeholder perspectives on participation in patient-centered comparative effectiveness research. J. Comp. Eff. Res. 5(5), 487–497 (2016). Link, Google Scholar
33. Nowell WB , Curtis JR , Crow-Hercher R . Patient governance in a patient-powered research network for adult rheumatologic conditions. Med. Care 56, S16–S21 (2018). Crossref, Medline, Google Scholar
34. Warren NT , Gaudino JA , Likumahuwa-Ackman S et al. Building meaningful patient engagement in research: case study from ADVANCE clinical data research network. Med. Care 56, S58–S63 (2018). Crossref, Medline, Google Scholar
35. Bindels J , Baur V , Cox K et al. Older people as co-researchers: a collaborative journey. Ageing Soc. 34(6), 951–973 (2014). Crossref, Google Scholar
36. Gibbons MC , Illangasekare SL , Smith E et al. A community health initiative: evaluation and early lessons learned. Prog. Community Health Partnersh. 10(1), 89–101 (2016). Crossref, Medline, Google Scholar
37. Kelly G , Wang S , Lucas G et al. Facilitating meaningful engagement on community advisory committees in patient-centered outcome research. Prog. Comm. Health Partnersh. 11(3), 243–251 (2017). Crossref, Medline, Google Scholar
38. Kim KK , Knodyakov D , Marie K et al. A novel stakeholder engagement approach for patient-centered outcomes research. Med. Care 56, S41–S47 (2018). Crossref, Medline, Google Scholar
39. Forsythe LP , Carman KL , Szydlowski V et al. Patient engagement in research: early findings from the Patient-Centered Outcomes Research Institute. Health Aff. 38(3), 359–367 (2019). Crossref, Google Scholar
40. Crocker JC , Ricci-Cabello I , Parker A et al. Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis. Br. Med. J. 363, k4738 (2018). Crossref, Medline, Google Scholar
41. Forsythe L , Heckert A , Margolis MK et al. Methods and impact of engagement in research, from theory to practice and back again: early findings from the Patient-Centered Outcomes Research Institute. Qual. Life Res. 27(1), 17–31 (2018). • Previous study that initially addresses the need for assessment and models to capture stakeholder engagement.Crossref, Medline, Google Scholar
42. Brett J , Staniszewska S , Mockford C et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient 7(4), 387–395 (2014). Crossref, Medline, Google Scholar
43. Cottrell E , Whitlock E , Kato E et al. Defining the benefits and challenges of stakeholder engagement in systematic reviews. Agency for Healthcare Research and Quality, MD, USA, AHRQ Report No.: 14-EHC006-EF (2014). Google Scholar

Vol. 8, No. 15

Metrics

Downloaded 358 times

History

Received 8 April 2019

Accepted 23 September 2019

Published online 18 November 2019

Published in print November 2019

Information

Keywords

Financial & competing interests disclosure

Research reported in this publication was funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (IHS-1608-35732). The statements in this publication are solely the responsibility if the authors and no not necessarily represent the views of the PCORI, its Board of Governors or Methodology Committee. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

Author contributions

J Martinez, C Wong, and NE Leland conceived the presented idea and designed the study. J Martinez and C Wong preformed the scoping review with the guidance of NE Leland who was also available to resolve any questions about the inclusion of articles in the sample. J Martinez, C Wong, and NE Leland wrote the manuscript with support and review from all authors. DC Bieber and BL Perry provided a review of the manuscript to ensure that research stakeholders’ perspectives were represented. CV Piersol provided extensive writing support and mentorship for the article resubmission.

Acknowledgments

We thank K Saric for assistance with formulating and implementing our search strategy, as well as L Terhorst for support with technical editing of the manuscript.

Financial & competing interests disclosure

Research reported in this publication was funded through a Patient-Centered Outcomes Research institute (PCORI) Award (IHS-1608-35732). The statements in this publication are solely the responsibility if the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

PDF download