Literature searching: easier than finding a needle in a haystack
II Some considerations when searching the literature
--Submitted by Allison Meserve, Health promotion consultant and Kara DeCorby, Senior knowledge product advisor, Public Health Ontario
Over the past 20 years, there has been increased emphasis on using research evidence in decision making for health promotion. Searching research evidence is included as a component of major planning models,[2-4] is mandated in the Ontario Public Health Standards (OPHS) and is a competency required for public health and health promotion practice.[6,7] Research evidence is an important factor in evidence informed decision making (EIDM), along with local context, community and political influences, available resources and public health expertise.
Literature searches which are conducted without using proper methodology, can lead to erroneous conclusions, and more importantly, planning, implementing or continuing programs which are ineffective at achieving desired health outcomes. Resources and tools are available to complete a comprehensive systematic search of the published research evidence.
II Some considerations when searching the literature
Literature searches come in many forms including systematic reviews, which systematically search, appraise quality and synthesize the research available on a particular topic; scoping reviews, which aim to determine the breadth and nature of the research literature and typically do not include quality appraisal; and rapid reviews, which use similar methods to a systematic review, including quality appraisal, but may limit the scope of the search to reduce the time and resources necessary to complete it.  The purpose for a literature search can vary as well including to write a briefing note for senior decision-makers, the background section of a grant proposal or a manuscript for publication in a peer-reviewed journal. No matter the purpose of the search or what type of review is carried out, the steps and methodology are similar.
The first step is to determine your research question. When defining a problem, and how to address it, there are three questions for which research evidence can be useful:
- What is the situation or problem?
- What is making the situation better or worse?
- What can we do about the situation? 
The research questions for your literature review will differ depending on what stage of the problem definition you are at. For example, to answer the first two questions, you may need to consult the literature on the epidemiological, behavioral or environmental determinants of the problem  which can highlight who is affected by the situation (e.g., groups of people more adversely affected by the situation due to their position in society, newcomer status, gender or sexuality), and factors which may be facilitating or inhibiting positive health outcomes (e.g., low-income, high networks of social support, the issue viewed as important by political decision makers). An example research question for this first step in defining the problem is, “What are risk factors associated with the development of large for gestational age (LGA) infants and what do the Ontario-specific data indicate about these risks?” 
Once you have determined what the problem is, who is affected by it and what facilitators and barriers currently exist, you will look to the research again to determine what policies, interventions or other health promotion strategies have been effective to address the situation. These research questions typically follow the PICO format – Population, Intervention, Comparison (or Setting) and Outcome. An example research question for this stage is, “What types of community-based interventions are being implemented to promote active play in children and youth aged 0 to 12, and are they effective?”  Resources to aid you in determining your research question can be found at the end of this article.
While this article describes a broad approach and some of the considerations for searching, keep in mind that searching is a skill that requires practice to do well.  Searching efforts need to fit the research question, and there is no one-size fits all strategy. Resources at the end of this article can help further develop your skills, and we recommend consulting an experienced librarian or designated information specialist for expert support and advice for a comprehensive search strategy, particularly when a documented search strategy is required. In the meantime, we draw attention to some considerations for a broad approach, which we hope can inform your strategies when searching.
Start broad and focus
It is generally recommended that you take a broad approach and gradually increase your focus until your search results are relevant. In doing this, you’re striking a balance between specificity and sensitivity (see Resources below for where to look for a description of these two concepts). A key question to ask yourself in deciding whether you’ve struck this balance is whether you are comfortable that you would likely have captured all of the key references on your topic. If the answer to that question is yes, then it is unlikely that broadening your search to capture more is going to dramatically change what you would conclude from the results.
Use a combination of indexing and keyword terms
When you search a resource, whether it is a database, online list, or a combination of sources, you need to consider indexing and keyword terms. Indexing terms are the proprietary terms that a database uses to catalogue or ‘tag’ articles it includes so that users can find them.  For example, Medline uses Medline Subject Headings (MeSH) to index included papers. Keywords are the terms generated by the searcher that can be used as a basic search, but which are not applied as a specific tag by the database itself.  For example, the searcher might know that an intervention goes by an informal name (e.g., search terms for active play interventions should include playground and recess) or that a certain idea is addressed using various terms, some of which might not be included in a database’s indexing system (e.g., high-risk behaviour, risky behaviour).
Use PICO to guide your search
Design your search strategy to address the current state of the literature on your PICO question. Although we suggest using PICO, we note that other frameworks for searchable questions are available for your consideration, with PICO being the framework we use most often in public health. Your PICO question and its components will help you determine the synonyms to include in your strategy; e.g., if your population includes children and adolescents aged 10–15 years, you would want to ensure the indexing terms selected in each database include the full range of 10–15 years specified in your PICO. Also consider synonyms for the terms within your PICO; for example, if you are searching for interventions that use mobile technology, you may want to consider using the indexing terms for “Cell Phones” and also search “mobile phones” as keywords to capture articles that use either. Additionally, there are different spellings used by the databases (e.g., Medline and CINAHL) and users need to include various spellings in their search strategy. 
Start at the top of the 6S pyramid and work your way down
When you develop your search strategy, consider that you’re looking for the best available evidence on your question. One aspect of the best available evidence is level of evidence, which in public health is defined by the 6S Pyramid. The 6S pyramid is outlined with instructional modules on the state of the evidence in the learning centre for the National Collaborating Centres for Methods and Tools (NCCMT) web site.  The pyramid provides guidance to search for the most highly-synthesized evidence first, saving time where others have already reviewed the evidence and summarized. 
Consider when to be creative
Your search needs to address the current state of the evidence and where the best available evidence is most likely to be found. Where you choose to search may be very different depending on what you expect to be the state of the literature. For example, should you be looking for barriers to physical activity in adults, you would expect there to be quite a bit of literature around that topic and you might feel comfortable that most of the established work in the area has been published, and you may then restrict your search to published literature. However, if you’re searching a newer topic such as the use of internet groups for peer interaction in teens and tweens, you may be unsure whether the published literature will hold the answers you need. A topic that is quite new might only require a search of the most recent 3–5 years of literature, or you might choose to include unpublished or grey literature to see whether a broader search strategy will harvest more relevant/representative results. Grey literature resources are listed below.
Effective, systematic searches of the research evidence can, and should, be carried out by health promotion professionals. The thoroughness and depth of a search will depend on time available, experience of those conducting the search, and number of people available for the search. Effort dedicated to an effective search however supports evidence-informed public health programs and policies.
The end of this article provides some of the resources available for all stages of your literature search. Health Promotion Capacity Building services at PHO (HPCB) and Health Evidence both provide free support and assistance with how to find and use evidence for decision making.
The learning centre at the National Collaborating Centres for Methods and Tools (NCCMT) provides training modules for all steps in the EIDM model, including literature searches: http://www2.nccmt.ca/learningcentre/index.php#main3.html NCCMT’s learning centre also provides a list of all relevant sources of public health evidence at each of the levels in the 6S pyramid.
Health Evidence appraises the quality of systematic reviews and sometimes provides summaries of the appraised reviews. You can search for articles by the quality of the review, when it was published, target audience, health topic, type of intervention and many other categories at http://www.healthevidence.org The Health Evidence website also includes tools which can be used to guide and track your search, including:
- Developing an Efficient Search Strategy using PICO;
- Resources to Guide and Track your Search; and
- Keeping Track of Search Results: a Flowchart
The Grey Literature Report (http://www.greylit.org) and the OPHLA grey literature database (http://ophla.pbworks.com/w/page/35497970/Public%20Health%20Grey%20Litera...) both provide access to grey literature in public health. Neither of these sites appraise the quality of literature posted, so quality appraisal will need to be done prior to determining whether the grey literature should be used.
1. Dobbins M, Rosenbaum P, Plews N, Law M, Fysh A. Information transfer: what do decision makers want and need from researchers? Implementation Sci. 2007;2(1):20. Available from: http://www.implementationscience.com/content/pdf/1748-5908-2-20.pdf
2. Ontario Agency for Health Protection and Promotion (Public Health Ontario). Planning health promotion programs: introductory workbook. 4th ed. Toronto, ON: Queen's Printer for Ontario; 2015. Available from: https://www.publichealthontario.ca/en/eRepository/Planning_health_promot...
3. Bartholomew K, Parcel GS, Kok G, Gottlieb NH, Fernandez ME. Planning health promotion programs: an intervention mapping approach. 3rd ed. San Francisco, CA: Jossey-Bass; 2011.
4. Gielen A, McDonald E, Gary T, Bone L. Using the PRECEDE-PROCEED model to apply health behavior theories. In: Glanz K, Rimer BK, Viswanath K, editor. Health behavior and health education: theory, research and practice. 4th ed. San Francisco, CA: Jossey-Bass; 2008. p. 407-33.
5. Ontario. Ministry of Health and Long-Term Care. Ontario public health standards, 2008. Revised October, 2015 [Internet]. Toronto, ON: Queen's Printer for Ontario; 2014 [cited 2016 Jan 11]. Available from: http://www.health.gov.on.ca/en/pro/programs/publichealth/oph_standards/d...
6. Public Health Agency of Canada. Core competencies for public health in Canada. Release 1.0. Ottawa, ON: Her Majesty the Queen in Right of Canada, represented by the Minister of Health; 2008. Available from: http://www.phac-aspc.gc.ca/php-psp/ccph-cesp/pdfs/cc-manual-eng090407.pdf
7. Pan-Canadian Network for Health Promoter Competencies. Pan-Canadian Health Promoter Competencies. Version 5. [Internet]. Ottawa, ON: Pan-Canadian Network for Health Promoter Competencies; 2014 [cited 2015 Jan 11]. Available from: https://static1.squarespace.com/static/52cb0336e4b0e90fb28b6088/t/54748e...
8. National Collaborating Centre for Methods and Tools. A model for evidence-informed decision making in public health [Internet]. Hamilton, ON: National Collaborating Centre for Methods and Tools; 2009 [cited 2016 Jan 11]. Available from: http://www.nccmt.ca/pubs/FactSheet_EIDM_EN_WEB.pdf
9. Yost J, Dobbins M, Traynor R, DeCorby K, Workentine S, Greco L. Tools to support evidence-informed public health decision making. BMC Public Health. 2014;14:728. Available from: http://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-14-728
10. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91-108.
11. Ontario Agency for Health Protection and Promotion (Public Health Ontario), Meserve A, Bergeron, K. Focus on: Six strategic steps for situational assessment [Internet]. Toronto, ON: Queen's Printer for Ontario; 2015 [cited 2016 Jan 11]. Available from: http://www.publichealthontario.ca/en/eRepository/FocusOn-Situational_Ass...
12. Ontario Agency for Health Protection and Promotion (Public Health Ontario), Vanderlaan A, Harrington D. Evidence Brief: Risk factors associated with the development of large for gestational age (LGA) infants in Ontario. Toronto, ON: Queen’s Printer for Ontario; 2015. Available from: https://www.publichealthontario.ca/en/eRepository/Risk_Factors_LGA_Infan...
13. Health-evidence.ca. Developing an efficient search strategy using PICO [Internet]. Hamilton, ON: Health Evidence; 2009 [cited 2016 Jan 11]. Available from: http://www.healthevidence.org/practice-tools.aspx#PT2
14. Ontario Agency for Health Protection and Promotion (Public Health Ontario), Jarvis J, Oei T. Evidence Brief: Promoting active play for children 0-12: a review of community-based interventions. Toronto, ON: Queen’s Printer for Ontario; 2015. Available from: https://www.publichealthontario.ca/en/eRepository/Promoting_Active_Play_...
15. Coughlan M, Cronin P, Ryan F. Doing a literature review in nursing, health and social care. London: SAGE Publications; 2013.
16. DiCenso A, Bayley L, Haynes RB. ACP Journal Club. Editorial: Accessing preappraised evidence: fine-tuning the 5S model into a 6S model. Ann Intern Med. 2009;151(6):JC3,2, JC3-3.
17. National Collaborating Centre for Methods and Tools. NCCMT Learning Centre: what is evidence-informed public health? [Internet]. Hamilton, ON: National Collaborating Centre for Methods and Tools; 2012 [cited Dec 30 2015]. Available from: http://www2.nccmt.ca/learningcentre/index.php?lang=en#main3.html
18. Robeson P, Dobbins M, DeCorby K, Tirilis D. Facilitating access to pre-processed research evidence in public health. BMC Public Health. 2010;10:95. Available from: http://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-10-95