Back to top

Online learning resources from the National Collaborating Centre for Methods and Tools -- (1) An effective way to promote and support skill development in evidence-informed practice and (2) Critical Appraisal of Research: Some resources

Online learning resources from the National Collaborating Centre for Methods and Tools -- (1) An effective way to promote and support skill development in evidence-informed practice

Contents

I Introduction
II How are the modules used?
III Are the modules effective?
IV Conclusion

--Submitted by Jeannie Mackintosh, National Collaborating Centre for Methods and Tools

I Introduction

When promoting evidence-informed practice, we must acknowledge the barriers to using research evidence in practice identified by many practitioners, including a shortage of time, a lack of access to research evidence, and inadequate skills necessary to critically appraise, interpret and apply the research found. The National Collaborating Centre for Methods and Tools (NCCMT) is committed to supporting evidence-informed public health by providing resources to address each of these barriers. The NCCMT website includes time-saving tools, links to credible sources of research evidence, and learning resources to help develop competence and confidence in finding, appraising and applying research evidence in practice.

One key resource offered by the NCCMT is a suite of free, self-paced modules that provides a solid foundation in the principles and skills required to implement NCCMT’s seven-step process of evidence-informed public health. The modules are based on material provided in NCCMT’s popular in-person workshops but reach beyond the limits of geography and time, providing access to remote learners and those on limited budgets. Learning is enhanced by interactive elements and realistic public health scenarios. Users create and login to an account in NCCMT’s Learning Centre to access the modules.

The suite includes the following titles:

  • Introduction to Evidence-Informed Decision Making in Public Health,
  • Quantitative Research Designs 101: Addressing Practice-Based Issues in Public Health
  • Searching for Research Evidence in Public Health
  • Critical Appraisal of Intervention Studies
  • Critical Appraisal of Systematic Reviews
  • Critical Appraisal of Qualitative Studies
  • Assessing the Applicability and Transferability of Evidence
  • Implementing KT Strategies in Public Health
  • Evaluating KT Strategies in Public Health

Time required to complete the modules varies from an estimated three to four hours to an estimated six to eight hours for the critical appraisal modules. Learners can stop whenever they need to and continue where they left off when they next login. Feedback from participants has shown that users appreciate the self-paced format, the easy navigation and interactive elements of the modules. A typical comment received through online feedback states, “I could complete the module at my own pace, and I could come back to parts of it when I wanted.” Another user appreciated the practical scenarios, stating “I was able to apply the information to an actual activity. The hands-on practice was perfect for me.” To date, NCCMT users have successfully completed over 4,000 learning modules.

While most module users are from Canada (52%) and the United States (35%), module users come from over 58 countries. They represent numerous roles and positions and varied levels of education with students and public health nurses making up the two largest user groups.

II How are the modules used?

Practitioners complete modules to demonstrate professional development. One learner from Manchester, UK, shared, “Even though I have worked in the NHS for 33 years, working through the [modules] has really been eye opening, excellent study material without the jargon etc.  it has made me want to start studying again.”

Some organizations require incoming staff to complete the Introduction to Evidence-Informed Decision Making module as part of their orientation. A nurse educator in Washington, D.C. who has “been actively involved on our Nurse Research Council for several years,” states that “[t]he modules created here are the best I have seen anywhere.”

The modules are also used by university and college educators at institutions in Canada, the US, Europe. One professor states, “I'm registered with and using the Learning Centre. I'm asking the students to register as well and go through part of the searching module. I hope this will create a long relationship between the students and the NCCMT.”

Users wanting to earn a certificate of completion for a module are required to complete the pre-assessment and must score at least 75% on the knowledge post-test. Of the modules started, 65% have been successfully completed. The modules are seen as useful resources to support evidence-informed practice; even after earning their certificates, many learners continue to access the modules.

III Are the modules effective?

Self-efficacy and knowledge are assessed before starting and after completing each module. Statistically significant increases in knowledge and self-efficacy from pre- to post-module completion have been demonstrated for each module. For more information on the evaluation of these online modules, please contact the NCCMT directly.

IV Conclusion

NCCMT’s modules address some of the barriers to evidence-informed practice. Learners work at their own pace; interactive elements and realistic scenarios allow learners to develop and practise new skills; and supplementary resources further enhance learning and understanding. Use of the modules has been shown to increase knowledge and self-efficacy. Furthermore, user feedback indicates that the online format provides an effective way to learn about evidence-informed public health practice.

To enquire about how you can use these online modules for your own professional development, for team training, or to augment classroom learning, please contact the National Collaborating Centre for Methods and Tools. To explore the modules in the Learning Centre, create and login to your free account. For more information about the products and services available from the, please visit the NCCMT website at http://www.nccmt.ca/.

Online learning resources from the National Collaborating Centre for Methods and Tools -- (2) Critical Appraisal of Research: Some resources from the National Collaborating Centre for Methods and Tools

Contents

I Introduction
II What is critical appraisal?
III How can NCCMT help you efficiently appraise research articles?
IV Want to know more?

I Introduction

Many public health practitioners are looking to research evidence to guide their decisions about programs and interventions. But in the process of examining the research evidence, it’s important to determine whether the research we are relying on is of good quality. That’s where critical appraisal comes in. If we want to improve our decisions by using research, we will need to be selective about which research we use.

II What is critical appraisal?

Critical appraisal is the process of judging the quality of a research study’s methods. In other words, we want to answer the question: Were the methods used in this study good enough that I can be confident in the findings?  

Sometimes we can find ‘pre-appraised’ sources, such as in a synopsis, which means that someone has already done the appraisal for us. But if we are reviewing a study with no other information about the quality of the research, we need to bring our own critical eye to the article.  

We frequently see problems associated with a lack of critical appraisal. It seems that people can find research to bolster any conclusion they want to support. Debates about water fluoridation, for example, are often fueled by reliance on poor quality research evidence (or sometimes, no research evidence at all!) Critical appraisal is a step in evidence-informed decision making that moves us away from bias, preference, or opinion, and into the realm of reliable, high quality evidence.

Appraisal is the third step in the NCCMT‘s approach to evidence-informed public health (EIPH)  (see What is Evidence-Informed Public Health? At http://www.nccph.ca/docs/EIPH_Factsheet_EN.pdf).  Appraisal is an important step, but one that is often problematic because people don’t have time to thoroughly review research evidence, or because they lack the skills to appraise research with confidence. NCCMT has several resources that can build critical appraisal skills.   Although building these skills can take some time up front, the investment pays off when critical appraisal becomes familiar.  NCCMT tools can improve the efficiency of your appraisal process by introducing a system and structure you can use for critical appraisal.  
By critically appraising research, you ensure that you are:

  • Basing your decisions on high quality research evidence
  • Helping to create an environment in which good quality research is valued

III How can NCCMT help you efficiently appraise research articles?

Three comprehensive skill-building learning modules are available online that focus on Critical Appraisal of Intervention Studies, of Systematic Reviews, and of Qualitative Studies. The modules provides in-depth guidance about the process usingrealistic public health scenariosto illustrate how critical appraisal works in practice (see the modules in NCCMT’s Learning Centre at http://www.nccmt.ca/learningcentre/index.php).

Another NCCMT online learning module - Quantitative Research Designs 101: Addressing Practice-Based Issues in Public Health - provides guidance on determining whether the research design is the most appropriate design  for answering the question being asked. Additional modules are in development now: Critical appraisal of qualitative research and Critical appraisal of systematic reviews.  All modules can be found in the Learning Centre on the NCCMT website.

The NCCMT site also provides resources such as a guide to available tools for critical appraisal, and other publications relevant to this step in the EIPH process. For example, see A Compendium of Critical Appraisal Tools for Public Health Practice.

Also on the NCCMT site, you will find links to sources of pre-appraised evidence. Health Evidence is a site that provides easy access to current review-level research evidence through a searchable online registry.  On that site you can find material related to evidence of the effectiveness of community water fluoridation. Public Health + provides easy access to pre-appraised sources, where someone has already done the critical appraisal for you, so you can be confident that the public health research available there is of good quality. Although evidence on Public Health + will be of good quality it is important to note that not all evidence shared is at the review-level; this resource also includes evidence from primary studies (http://www.nccmt.ca/public_health_plus/all/1/list-eng.html).

IV Want to know more?

Contact the National Collaborating Centre for Methods and Tools to find out about more about critical appraisal. Sign up for the NCCMT Weekly Roundup to be sure that you hear about upcoming events and new resources as they become available at http://www.nccmt.ca/subscribe-eng.html.

Resources