Addressing challenges of evaluating health communication campaigns

Contents

I Introduction
II Health communication evaluation challenges
III Comprehensive evaluation efforts to address challenges
IV Demonstrating causal inference in health communication efforts
V Conclusion

--Submitted by Sophie Rosa

I Introduction

Health communication is one of the key health promotion approaches that enable people to increase control over, and improve their health. [1] However, demonstrating the effect of health communication efforts on behaviour change and isolating the effects of a campaign on a given health outcome are challenging and often require extensive resources. In fact, researchers examining the effectiveness of various major health communication campaigns report a prevalent lack of proper design and evaluation rigor. [2-5] In many cases,  major campaigns that have been  examined were not grounded in any validated health promotion models and theories [2], while more than half did not have any formative research efforts. [3,4] In his review, Noar (2006) reported that most campaigns examined had weak evaluation designs that failed to remove threats to internal validity. [6] The purpose of this article is to discuss the most prevalent challenges in the evaluation of health communication campaigns and offer practical solutions to adequately demonstrate causal inference of health communication efforts.

II Health communication evaluation challenges

Rarely can the evaluation of a health communication campaign rival the reliable evidence provided by randomized control trials. [7] Many other variables interact with health communication efforts to indirectly contribute to the actual behaviour change, making it challenging to isolate the effects of a campaign on a given health behaviour. [8–10]  Furthermore, campaigns require sufficient exposure and time to achieve the level of reach necessary to generate measurable behaviour change. When evaluation efforts do manage to attribute behaviour changes to a given health communication campaign, it is often with lesser magnitude. [11]  

III Comprehensive evaluation efforts to address challenges

Opting for an outcome evaluation plan that focuses solely on behaviour change to assess the impact of a health communication campaign would imply that the campaign’s message alone can bridge the gap between knowledge and behaviour change. Many other variables come into play in behaviour change; for instance, demographics, psychometrics and behavioural traits. [9, 10] As such, careful considerations must be given to how these variables affect behaviour change at each phase of the campaign cycle – design, implementation and evaluation.

An evaluation approach to assess a campaign’s effectiveness should not be seen as a one-time activity, but as a series of activities from the outset to the end of the campaign. Outcome evaluation activities should be balanced with both sound formative research and process evaluation activities that generate timely, relevant and credible evidence, based on valid and reliable data collection and analysis.

Formative research is a critical step that will inform key campaign decisions. [12, 13] For instance, a sound situational assessment will provide insights into the behaviour of interest such as facilitators and barriers to change. [12, 14] A thorough audience analysis will identify determinants of behaviour change, effective messaging strategies and appropriate channels and vehicles to communicate key messages. [12, 13] Finally, pre-testing of messages and communication products will ensure the campaign resonates with the target audience. [13]

A process evaluation will assess if all the dependencies are falling into place and if the campaign cycle unfolds seamlessly; both critical to ensure success. [12] Process evaluation occurs throughout the campaign cycle and includes, but is not limited to, monitoring implementation efforts. [12, 13] It will also show if your formative research efforts were thoroughly conducted. If your results indicate they were, then you can reasonably deduce that your key messages and communication products informed by your formative research will resonate with and reach your target audience. Process evaluation should also include an assessment of your outcome evaluation activities; having a flawless evaluation plan will not yield valuable data if the evaluation efforts deployed lack rigor and quality.    

As suggested earlier, outcome evaluation in health communication poses many challenges and does not lend itself well to randomized control trials; however, planning teams can consider alternative methodologies. For instance, quasi-experimental designs, such as pre- and post-test control group designs, panel designs and interrupted time series designs can provide valuable insights into the impact of health communication campaigns. [8] However, as suggested by Noar et al. (2009), these designs are limited in terms of their ability to establish causal inference in health communication. [5]    

IV Demonstrating causal inference in health communication efforts

To complement quasi-experimental designs, teams can adopt an incremental approach to evaluation based on available evidence and strong foundational models or theories for behaviour change and message development to demonstrate causal inference.

For instance, take a healthy eating campaign targeting college students using the Health Belief Model [9] as a foundational model and the theory of message framing. [15] Based on its formative research, the planning team would favour the health belief model since it has been consistently identified as a strong predictive model for healthy eating behaviours. [16–18] Formative research would also highlight message framing as a theory to consider in the development of the messages. [19–20]

Furthermore, the planning team would uncover during its formative research efforts gender differences in the effect of message framing and perceived susceptibility on healthy eating in college students. [17] As a result, they would opt for separate messages and materials specifically tailored to males and females. For females, the campaign would highlight the potential losses associated with poor eating habits, while the messages for males would focus on increasing their perceptions of susceptibility. [17] As for the choice of channel and vehicles, formative research would suggest point-of-purchase interventions to further maximize reach and impact. [21]

Such methodological rigour during the formative research phase, complemented by thorough process evaluation efforts, can supplement outcome evaluation efforts and help establish causal inference. As such, evaluation efforts should be woven throughout the entire campaign cycle to effectively demonstrate health communication impact.

V Conclusion

Complementing outcome evaluation methods with formative and process evaluation methodologies can address some of the challenges in assessing the impact of health communication campaigns. In many health communication efforts, the measures of a campaign’s effectiveness are limited to a few indicators of the actual desired outcome, for instance, fruit and vegetable intake, healthy food purchase and reduction in processed food consumption as measures for healthy eating. [2–6] However, other moderating or mediating variables come into play in the development and maintenance of healthy eating behaviours, such as self-efficacy and gender. [17–20] As such, time and resources should be invested in formative research efforts to inform the design of campaigns, guide the development of messages and address the current challenges inherent to health communication evaluation. In order to adequately demonstrate causal inference, each phase of the campaign cycle – design, implementation, and evaluation – should be informed by timely, relevant and credible evidence gathered through valid and reliable data collection methods and sound analysis.   

References

World Health Organization. The Ottawa Charter for Health Promotion: First International Conference on Health Promotion [Internet]. Geneva: World Health Organization; 1986 [cited 2014 Dec 2]. Available from: http://www.who.int/healthpromotion/conferences/previous/ottawa/en/

Noar SM, Palmgreen P, Chabot M, Dobransky N, Zimmerman RS. A 10-year systematic review of HIV/AIDS mass communication campaigns: Have we made progress? Journal of Health Communication. 2009; 14:15-42.

Noar SM. Challenges in evaluating health communication campaigns: Defining the issues. Communication Methods and Measures. 2009; 3:1-11.

Randolph W, Viswanath K. Lessons learned from public health mass media campaigns: Marketing health in a crowded media world. Annual Review of Public Health. 2004;25:419-437.

Noar SM, Palmgreen P, Zimmerman R. Reflections on evaluating health communication campaigns. Communication Methods and Measures. 2009;3:105-114.

Noar, SM. A 10-year retrospective of research in health mass media campaigns: Where do we go from here? Journal of Health Communication. 2006;11:21-42.

Hornik, RC. Public health communication: making sense of contradictory evidence. In: Hornik, RC, editor. Public Health Communication: Evidence for Behavior Change. Mahwah, NJ: Lawrence Erlbaum Associates Inc: 2002. p. 1–22.

Abbatangelo-Gray J, Cole GE, Kennedy MG. Guidance for evaluating mass communication health initiatives: summary of an expert panel discussion sponsored by the Centers for Disease Control and Prevention. Eval Health Prof. 2007; 30:229–253

Rosenstock IM. The Health Belief Model and preventive health behavior. Health Educ Monogr 1974; 2:354-386.

Glanz K, Rimer BK, Lewis FM. Health Behavior and Health Education: Theory, Research, and Practice. 3rd edn. San Francisco, CA: Jossey-Bass; 2002.

Hornik R, Kelly B. Communication and diet: an overview of experience and principles. J Nutr Educ Behav. 2007; 39:S5–S12.

Patton MQ. Utilization-focused evaluation. 4th ed. Thousand Oaks, CA: Sage Publications; 2008.

Schiavo R. Health Communication; from theory to practice. 2nd ed. San Fransisco, CA: Jossey-Bass; 2014.

Ontario Agency for Health Protection and Promotion (Public Health Ontario), Meserve A, Bergeron, K. Focus on: Six strategic steps for situational assessment. Toronto, ON: Queen’s Printer for Ontario; 2015. Available from: http://www.publichealthontario.ca/en/eRepository/FocusOn-Situational_Ass...

Rothman AJ, Salovey P. Shaping perceptions to motivate healthy behaviour: the role of message framing. Psychol Bull. 1997;121:3–19.

Deshpande S, Basil MD & Basil DZ. Factors Influencing Healthy Eating Habits Among College Students: An Application of the Health Belief Model, Health Marketing Quarterly. 2009;26(2):145-164.

Garcia K, Mann T. From ‘I wish’ to ‘I will’: Social-cognitive predictors of behavioral intentions. Journal of Health Psychology. 2003;8:347 – 360.

Von Ah  D, Ebert S, Ngamvitroj A, Park N, Kang DH.  Predictors of health behaviours in college students. Journal of Advanced Nursing. 2004;48:463 – 474.

Gerend MA, Maner JK. Fear, Anger, Fruits, and Veggies: Interactive Effects of Emotion and Message Framing on Health Behavior. Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association. 2011;30(4):420–423.

Rothman AJ, Bartels RD, Wlaschin J, Salovey P. The strategic use of gain- and loss-framed messages to promote health behavior: How theory can inform practice. Journal of Communication. 2006;56:S202–220

Busher LA, Martin KA, Crocker S. Point-of-Purchase Messages Framed in Terms of Cost, Convenience, Taste, and Energy Improve Healthful Snack Selection in a College Foodservice Setting, Journal of the American Dietetic Association. 2001;101(8):909-913.