Types of evidence
Hierarchy of Evidence
Setting the search strategy
Types of evidence
Is your question about treatment, exploration, aetiology, diagnosis or prognosis? The type of question will inform the type of research evidence you need to look for to address your clinical question
Secondary evidence or secondary research:
A type of research that involves the summary, collation, and synthesis of findings from existing or already completed primary studies. A systematic review, using either meta-analytic analysis or narrative synthesis, is the primary methodology in secondary research.
A systematic review summarises the results of available carefully designed studies in response to a research question. A high quality systematic review is considered to be the most reliable source of evidence to guide clinical practice.
Podcast: What is a systematic review?
Features of a systematic review include (Clarke 2011)
- Clear aims with pre-determined eligibility and relevance criteria for studies
- Transparent and reproducible methods
- Rigorous search designed to locate all eligible studies
- An assessment of the validity of the findings of the included studies
- Systematic presentation and synthesis of the included studies
Useful resources for the steps in developing, conducting and writing a systematic review:
Steps for writing a systematic review protocol
Steps for developing a search strategy
Data extraction
Quantitative extraction
Qualitative extraction
Data synthesis
Qualitative synthesis
- Methods for the synthesis of qualitative research: a critical review
- Synthesis of qualitative research
Quantitative synthesis
Further reading
Useful websites to find systematic reviews:
The Cochrane Library
The Joanna Briggs Institute
Centre for Reviews and Dissemination
The Campbell Collaboration
PubMed
Further reading
Hemingway P. What is a systematic review, 2nd ed. Evidence based med 2009.
Green S. Systematic reviews and meta-analysis. Singapore Med J 2005 Jun; 46(6):270-3.
Centre for cognitive ageing and cognitive epidemiology. Systematic reviews and meta-analyses: a step-by-step guide.
Primary evidence or primary research:
Any research that is conducted on human subjects or animals (or parts thereof), and which therefore requires ethical approval.
A beginner’s guide to ethics approval
National Statement on Ethical Conduct in Human Research
Observational research: Research in which there is no deliberate intervention, and subject choices (and their effects) are observed and measured.
Cross sectional: the observation of a defined population at a signal point in time or time interval. Exposure and outcome are determined simultaneously.
Case control: where characteristics of a known group of subjects with a specific disease (cases) are compared with matched controls. Matching is usually by key confounding variables.>
Cohort study: a group of subjects is recruited and their disease and exposure status is measured over time.
Further reading
Experimental research: A planned attempt to assess the effects of a prospectively-delivered intervention. Interventions can also be called treatment(s), or manipulation of features of subjects’ environments.
N=1 (Single Case Experimental Design) (SCED): where one individual is given, at least, one intervention, and the effect of that intervention is compared with pre-intervention (baseline) measures. Within-subject variability is established by the distribution of data of repeated measures taken from that individual.
Using Case Studies in Research
Pre-post study: where the one group of subjects is given the same intervention, and this group of subjects acts as its own controls. Outcome measures are taken before (pre, baseline), the intervention/treatment is delivered, and the outcome measures are taken again (post-intervention).
Pretest-posttest designs explained
Quasi-experimental study: two groups of subjects are compared, usually in different time periods or under different circumstances. One group usually is the ‘control’ in which no intervention occurs (usual care) and the second group receives the intervention. This is an experimental study approach to negotiate around situations where having a concurrent control/intervention study in the one organisation may result in violation of the study protocol (for instance where control subjects may end up receiving the intervention). This can occur when the control and intervention are being delivered concurrently in one hospital in two wards (one control, one intervention) and the control ward staff start delivering the intervention (or vice versa).
Quasi-experimental study: designs, elements and examples
Randomised controlled trial: an experimental study conducted in a sample of subjects randomly selected from a known population. Intervention arms are randomly assigned.
Stolberg HO, Norman G, Trop I. Randomised controlled trials. Am J Roentgenol 2004; 183: 1539-1544.
Further reading
Activity
iCAHE MS Excel training booklet - A short course in data handling and statistical function
Qualitative research: A method of inquiry which aims to gather an in-depth understanding of human behaviour and the reasons that govern such behaviour. The qualitative method investigates the why and how of decision making, not just what, where, when and how much. Hence, smaller but focused samples are more often needed than large samples.
Podcast: Qualitative Research: Points for beginners (11MB)
Qualitative researchers use different data collection techniques such as focus groups, interviews, ethnography and participant observation and document analysis.
Common qualitative research methodologies are:
Participatory Action Research (PAR): seeks to understand the world by trying to change it, collaboratively and reflectively. Aims to examine the political structures that disempower marginalised, deprived and oppressed groups of people to find ways in which these structures can be changed. It aims to transform “social realities”.
Ethnography: is aimed at exploring cultural phenomena. It reflects the knowledge and the system of meanings in the lives of a cultural group. Ethnography is a means to represent graphically and in writing, the culture of a group of people. The typical ethnography is a holistic study and so includes a brief history, and an analysis of the terrain, the climate, and the habitat. It observes the world (the study) from the point of view of the subject (not the participant ethnographer) and records all observed behaviour.
Reeves S, Kuper A, Hodges BD. Qualitative research methodologies: ethnography. BMJ 2008:a1020
Grounded theory: is a research methodology that seeks to develop theory that is grounded in data systematically gathered and analysed. Rather than beginning with a hypothesis, the first step is data collection, through a variety of methods. From the data collected, the key points are marked with a series of codes, which are extracted from the text. The codes are grouped into similar concepts in order to make them more workable. From these concepts, categories are formed, which are the basis for the creation of a theory, or a reverse engineered hypothesis. This contradicts the traditional model of research, where the researcher chooses a theoretical framework, and only then applies this model to the phenomenon to be studied.
Noerager Stern P. Grounded theory methodology: Its uses and processes.
Holton JA. Grounded theory as a general research methodology.
Phenomenology: is the study of a phenomenon and aims to study everyday situations from the view point of the experiencing person. It describes the “subjective reality” of an event, as perceived by the study population.
Lester S. An introduction to phenomenological research, Stan Lester Developments, Taunton.
Further reading
Patton MQ. Qualitative research & evaluation methods (3rd ed.) 2002. Thousand Oaks, CA: Sage Publications
Mixed Methods research: The following articles describe what a mixed methods research is (and what it is not), when it should be used, how it is designed and the methodological challenges associated with conducting mixed methods investigation.
The Nature and Design of Mixed Methods Research
Steps in Conducting a Scholarly Mixed Methods Study
Mixed methods in Biomedical and Health Services Research
Major mixed methods designs:
Convergent parallel design (also known as Triangulation design): The purpose of the convergent design is to obtain different but complementary data on the same topic to best understand the research problem. This design is used when the researcher wants to triangulate the methods by directly comparing and contrasting quantitative statistical results with qualitative findings for corroboration and validation purposes.
Example of a study which utilised this approach:
Explanatory Sequential Design: This is a mixed methods design in which the researcher begins by conducting a quantitative phase and follows up on specific results with a second phase – qualitative phase. The qualitative phase is intended to explain the initial quantitative results in more depth. This design is most useful when the objective is to assess trends and relationships with quantitative data and also explain the mechanism or reasons behind the resultant trends.
Example of a study which utilised this approach:
Exploratory Sequential Design: The primary purpose of this mixed methods design is to generalise qualitative findings based on a few individuals to a larger sample gathered during the second phase – quantitative phase. This design is particularly useful when the objective is to develop and test an instrument because one is not available or to identify important variables to study quantitatively when the variables are unknown. It is also appropriate when one wants to generalise qualitative results to different groups, to test aspects of an emergent theory or classification, or to explore a phenomenon in depth and measure the prevalence of its dimensions.
Example of a study which utilised this approach:
Embedded Design: This is a mixed methods approach where the researcher combines the collection and analysis of both quantitative and qualitative data within a traditional quantitative research design or qualitative research design. The premises of this design are that a single data set is not sufficient, that different questions need to be answered, and that each type of question requires different types of data.
Example of a study which utilised this approach:
Reference
Creswell J, Plano Clark V. Designing and conducting mixed methods research. 2nd edition, Sage Publications, California USA.
Further reading
The use of "mixing" procedure of mixed methods in health services research
Choosing a Mixed Methods Design
Expert opinion:
This refers to the views of professionals who have expertise in a particular form of practice or field of inquiry, such as clinical practice or research methodology.
top
Hierarchy of Evidence
The hierarchy of evidence is a way of ranking the relative authority of various research designs based on the degree of bias within their methodologies. There is no standard hierarchy of evidence; however there is agreement on the position in the hierarchy of different types of research. Secondary research (systematic reviews, meta-analyses) rank at the top of the hierarchy because they include data from multiple primary studies. Primary researches rank next to secondary research and expert opinion is ranked lowest in the hierarchy.
Further reading
Setting the search strategy
Podcast: Developing a search strategy.
Example Question: What is the evidence for the effectiveness of Mediterranean diet in patients with type II diabetes?
|
P (population) |
Middle aged man with type 2 diabetes |
|
I (intervention) |
Mediterranean diet |
|
C (comparison) |
Other types of diet |
|
O (outcome) |
Glycaemic level |
Select relevant databases
The following databases are commonly used to find evidence-based practice related information:
MEDLINE: contains bibliographic and abstract coverage of biomedical literature including coverage in the areas of allied health, biological and physical sciences, humanities and information science as they relate to medicine and health care, communication disorders, population biology, and reproductive biology.
CINAHL: covers nursing, biomedicine, health sciences librarianship, alternative/complementary medicine, consumer health and several allied health disciplines. This database also offers access to health care books, nursing dissertations, selected conference proceedings, standards of practice, educational software, audiovisuals and book chapters.
PEDro: provides physiotherapists and others access to bibliographic details and abstracts of randomised controlled trials and systematic reviews in physiotherapy.
OTseeker: a database that contains abstracts of systematic reviews and randomised controlled trials relevant to occupational therapy. Trials have been critically appraised and rated to assist Occupational Therapists to evaluate their validity and interpretability. These ratings will help in judging the quality and usefulness of trials for informing clinical interventions.
Cochrane Library: covers all areas of health care; provides high quality evidence to inform people providing and receiving care, and those responsible for research, teaching, funding and administration at all levels. The library consists of 6 databases: Cochrane database of systematic reviews (CDSR), Database of abstracts of reviews of effects (DARE), Cochrane central registrar of controlled trials (CENTRAL), Cochrane methodology register (CMR), Health technology assessment database (HTA), and NHS economic evaluation database (NHSEED).
Google Scholar: Google Scholar is a free service for searching scholarly literature. It includes peer-reviewed papers, theses, books, preprints, abstracts and technical reports.
Choose key words from your question
To develop a search plan, identify the main concepts from your question (PICO or the other formats) and determine the relationships between these concepts. Break your topic into several parts to make it more manageable.
Example
Concept 1: Type 2 diabetes
Concept 2: Mediterranean diet
Concept 3: Glycaemic index
These concepts are the keywords you will use to search for research evidence. Remember to include similar words (synonyms) or alternative spellings, complete names or abbreviations, scientific or common names. In cases when a database has its own indexing language, or controlled vocabulary (e.g. medical subject headings (MeSH)), the search terms should include these index terms.
Example
Concept 1: type 2 diabetes, type II diabetes, Diabetes Mellitus, adult onset diabetes, maturity onset diabetes etc.
Concept 2: Mediterranean diet, diet, dietary pattern, Mediterranean, Mediterranean food, Mediterranean eating, Mediterranean nutrition etc.
Concept 3: Glycaemic index, Glycaemic level, blood glucose, blood sugar etc.
Think about how these keywords relate to one another, use Boolean operators (OR, AND) as appropriate. When you combine keywords using OR, your search will find items containing either one or both of your keywords. Therefore, use OR to make your search broader, and to combine words with similar meanings. On the other hand, when you combine keywords using AND, your search will only find items containing both of your keywords.
Most databases allow you to use a symbol to search for all words beginning with a particular base. These are called truncation symbols and may vary depending on the database software (?, *, $)
e.g. typing learn* in the Cochrane Library would find learn, learns, learner, learners, learning
Another symbol, usually called a Wildcard, may be used to search for alternative spellings or forms
e.g. typing organi?ation in the Cochrane Library would find both organisation and organization, while typing col?r would find both color or colour.
For a step-by-step guide on developing a systematic search strategy see: A short course in searching the literature: A basic introduction to developing a search strategy, database hints, and executing a search
For a step by step guide in searching for evidence (PubMed example), refer to the following article:
