Systematic Review Protocol       

A review of nursing research literature to identify the frequency of research methodologies used

                   

Review team: Professor David Richards Neil Pumphrey Vania Coulthard

Background  The current problem Approximately one million healthcare research publications are produced worldwide annually, at a cost of some $100 billion dollars. However, much of this research is currently wasted due to the findings being unusable by clinicians or patients (Chalmers & Glasziou, 2009). Four main areas of waste were identified: the choice of research questions (choosing the wrong questions for research); the quality of research design and methods (doing studies that are unnecessary, or poorly designed); the adequacy of publication practices (the failure to publish relevant research promptly, or not at all); and the quality of reports of research (biased or unusable reports of research). Chalmers and Glasziou concluded that the accumulation of waste over the four stages results in only 15% of research not being wasted. The situation regarding nursing research is just as bad. Rahm Hallberg (2006) scanned 210 articles produced in the same year from two international nursing science journals, and found that only 15% of the studies published carried strong evidence for practice. Mantzoukas (2009) reviewed the abstracts of 2547 studies published in the leading top ten nursing research journals (based on impact factor scores) from the years 2000 and 2006. The studies published were mostly descriptive (47%) with very little in the way of secondary research (i.e., meta-analysis, meta-synthesis or systematic reviews), which is a method of bringing together and synthesising data from primary research, and is widely regarded as an essential element of evidence-based practice research. In other words, nursing practitioners are being deprived of research evidence that they can utilise in day-to-day practice in order to potentially change and improve patient care.

Why is it important? Nursing is a quintessential ‘complex intervention’. It has a critical role to play in meeting health and social care challenges – an aging population, chronic diseases and new endemics at the fore of European health concerns. Increasingly, nurses engage in a wide range of highly complex activities, many of which take place in multiple care environments including acute medicine, chronic care facilities, community and residential homes. Changes in health care organisation internationally (e.g. short hospital periods, growing responsibility for patient self-care) place more health care in the hands of nurses, increasing the scope, the overall need for nursing care and for that care to rest on a solid evidence base (Richards & Borglin, 2011).

 

1

What can be done? Hallberg has challenged researchers to re-focus their activities to develop knowledge for nursing that is both useful and that can be readily translated into practice. This re-focus needs to develop knowledge, ‘in a step wise manner, a series of studies from descriptions, theory development, testing, exploring possible explanations, refining models or theories and testing them and implementing valid knowledge in practice. In particular, we need to learn more about the implementation process and how to make it successful.’ (Hallberg, 2009). Part of the problem is that many nurse researchers lack the knowledge, skills and infrastructure to undertake programmatic, multidisciplinary, theory based and integrated research. The next generation of researchers must be equipped with knowledge and skills in sequenced, mixed-methods, complex intervention strategies. In order to bring about the changes necessary to improve nursing research, the Researching Complex Interventions for Nursing (REFLECTION) Research Network Programme, funded by Research Councils and Academies from eight different European countries, is leading the drive to re-focus nursing research activity and develop knowledge for nursing that is both useful and that can be translated into practice. The network aims to move European research in nursing to an experimental, longitudinal, generalisable and implementation/transition focused position.

Why it is important to do this review Before the REFLECTION network can achieve its aims, it is essential to fully understand the extent of the problem in today’s nursing research literature. This review will analyse the literature in order to obtain a baseline measure of the extent to which different research methodologies are currently being used.

Objective To identify, appraise and synthesise research reports in (European) nursing in 2010 to identify the frequency of different primary and secondary research methods used and the extent of translational, mixed/multi-methods and programmatic research into care provided to patients by nurses.

 

2

Methods Criteria for inclusion and exclusion of studies Inclusion Criteria – Types of studies All articles where data is collected from research involving nursing practice will be considered. Studies using one methodology (e.g. RCT, phenomenological study) or studies that combine several different methodologies (e.g. survey design and a semi-structured interview) will be considered. No one type of methodology will be excluded. Only articles written in English language will be considered.

Inclusion Criteria – Participants Articles where data is collected from nurses, the consumers, or potential consumers, of nursing care including members of the public and carers of people in receipt of nursing interventions. NURSE is used as shorthand to include the broad spectrum of professional qualifications such as nursing, mental health nursing, midwifery, health visiting, community and family nursing, and care delivered by members of nursing teams without professional qualification. CONSUMERS or potential consumers of nursing care include patients, members of the public (for example school children, people receiving public health education from nurses); no restrictions on environments such as hospital, community, primary care etc.

Inclusion Criteria – Time and Place Reports published by researchers, conducted in Europe in the year 2010. Europe is defined as any of the following 47 countries: Albania, Andorra, Armenia, Austria, Azerbaijan, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Georgia, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Liechtenstein, Lithuania, Luxemburg, Malta, Moldova, Monaco, Montenegro, The Netherlands, Norway, Poland, Portugal, Romania, Russian Federation, San Marino, Serbia, Slovak Republic, Slovenia, Spain, Sweden, Switzerland, The former Yugoslav Republic of Macedonia, Turkey, Ukraine, United Kingdom.

 

3

Exclusion Criteria 

Studies that do not collect data: editorials/commentaries.



Studies involving educating nurses (as opposed to clinical research) where patients are not included.



Book reviews.



Studies testing medical equipment.



Studies conducted in non-European countries.

Search Strategy Study identification will include both manual and electronic searching strategies. The top 10 nursing journals (identified by impact factor) will be hand-searched and all articles that fulfil the inclusion criteria will be retrieved. Simultaneously, we will conduct electronic searches using the databases listed below and the search terms included in Appendix A.

Electronic databases The following electronic databases will be searched for articles published in 2010 (Centre for Reviews and Dissemination, 2009; Higgins & Green, 2011): CINAHL, MEDLINE, PsychINFO, EMBASE, Cochrane Database, MedlinePlus.

Data collection and extraction Selection of studies A team based in the University of Exeter will perform the manual and electronic searches. Studies identified will be scanned independently by two review authors who will exclude the studies according to the criteria above, on the basis of titles and abstracts. Full copies of the studies deemed eligible by one of the review authors will be retrieved for closer examination. If there is uncertainty or disagreement, consensus will be reached by discussion and consultation with a third review author. A log of all studies which initially appear to meet the inclusion criteria but are excluded on retrieval of the full text will be detailed in a table of excluded studies. A record will be kept of the reasons for exclusion. The included studies will then be distributed to an independent team from the REFLECTION network. The REFLECTION team will extract data from the included studies based on the  

4

criteria below. The data will be sent back to the team based in Exeter where it will be collated and synthesised.

Data extraction Data will be extracted and categorised from included studies according to the following criteria:

1. Can the study be classified as either primary (Y/N) or secondary (Y/N) research? Primary research is classified as research where new data is being collected in the study, as opposed to data being analysed from already published sources or from datasets collected for another purpose. Secondary data is classified as one or more of the following five subcategories: 

Systematic literature review: identifies, appraises and synthesises research literature.



Meta-analysis: the statistical combination of quantitative findings from several studies.



Meta-synthesis: brings together the findings from similar qualitative studies at an abstract level as to produce second order interpretations and develop theories.



Secondary analysis: includes a retrospective analysis of gathered data for another study, but addresses new questions from an alternative perspective.



Analysis of routine data: an analysis of data that is routinely collected for other purposes (e.g. mortality rates in hospitals) that was not intended to be collected for the study.

2. What method is being used in the study? Studies will be categorised according to the methodology or methodologies employed, and whether or not the data collected is numeric. Experimental studies: studies that involve the manipulation of an independent variable. These types of studies tend to collect numeric data: 

 

Experimental study type 1: studies where there is a manipulation of an independent variable (e.g. administering an experimental treatment), subjects are randomly assigned to different conditions and there is a control group. These types of studies are often referred to in the literature as ‘randomised control trials’. 5



Experimental study type 2: studies in which two or more groups of participants are observed before and after an intervention is implemented. This design includes a control group, but lack of randomisation to conditions means that the experimental and comparison group cannot be assumed as equivalent. These types of studies are often referred to in the literature as ‘non-equivalent control group before-after design’ or ‘pre-experimental design’.



Experimental study type 3: studies that involve collecting data from subjects over a period of time with the intervention being implemented during that period, but do not include a control group or randomisation of subjects. These types of studies are often referred to in the literature as ‘time series design’ or ‘uncontrolled pre-post design’.

Observational studies: in this type of study no attempt is made to change behaviours or conditions, and data collected tends to be numeric: 

Correlational (retrospective): a phenomenon observed in the present is linked to a phenomenon occurring in the past (e.g. the link between cigarette smoking in the past and rates of lung cancer today – “How many people with lung cancer have been smokers?”).



Correlational (prospective): the research begins with a presumed cause and then goes forward to the presumed effect (e.g. the link between the amount of cigarette smoking today and rates of lung cancer in the future – “How many smokers will contract lung cancer in future?”).



Case-control: this involves comparing a control group (e.g. a group without a certain illness) with a case group (e.g. a group of people with same illness), where both groups share similar background factors (e.g. age, working conditions, etc.) that could be linked to the condition. (“Amongst people with similar backgrounds, what are the differences between people who have lung cancer and those who do not?”)



Descriptive research: this is where studies observe, describe and document aspects of a situation, where there is no attempt to describe the relationship between variables (“What is the prevalence of lung cancer?”).

The following types of studies generally collect non-numeric data:

 



Ethnography: provides a framework for studying the meanings, patterns and experiences of a defined cultural group in a holistic fashion.



Phenomenology: is an approach to thinking about people’s life experiences, and asks, what is the essence of this phenomenon experienced by people and what does it mean.

6



Grounded Theory: studies using grounded theory have a primary purpose of generating a theory from data that can explain a pattern of behaviour that is problematic and relevant to the participants involved in the study.



Critical Theory: is concerned with a critique of existing social structures; critical researchers strive to conduct inquiries that involve collaboration with participants and foster enlightened self-knowledge and transformation.



Feminist Research: whose foundations are those of critical theory, however focuses on how gender domination and discrimination shapes women’s lives and their

3. Is the study programmatic research? (Y/N) If a study is found to be part of a series of planned, purposeful and theoretically linked studies, then it will be categorised as programmatic research. In order to decide that a study is part of a program of research, the study should include reference to other work conducted by the research team or individual researcher which directly links to the study in question. These other studies should be clearly articulated as connected to the objectives of the study in question. Aspirational studies (i.e. those planned but where there is no evidence of execution) do not count as programmatic research. There has to be EVIDENCE of other related studies being undertaken by the researchers, not just good ideas in the research recommendations/discussion part of the paper.

4. Is the study translational research? Translational research in nursing is the process of turning appropriate theories or pilot interventions into nursing interventions that can be used in a widespread way for the treatment of patients. There are three phases of translational research:

 



Phase 1: A study which takes prior theoretical or empirical knowledge and uses it to construct a nursing intervention is a Phase 1 clinical trial and is often called "bench to bedside". This is the research process that investigates and translates non-clinical research results into clinical applications and tests their safety and efficacy.



Phase 2: A study which takes a potential nursing intervention and tests it in a clinical population is a Phase 2 study. This phase examines how findings from clinical science, shown to be efficacious and safe treatments established in Phase 1 translational research, function when they are applied in routine practice.



Phase 3: A study which takes a proven nursing intervention and investigates the uptake of the intervention in a routine nursing environment is a Phase 3 study. This phase adds the necessary information to convert treatments and prevention strategies,

7

shown to be effective and cost-effective in Phase 2 Translational Research, into sustainable solutions.

5. Where in the Medical Research Council guidelines does the study fit? Studies that are selected will be analysed to see where they stand in relation to the MRC (Craig et al., 2008) guidelines. The first step will be to see if the study makes reference to the MRC guidelines and at which stage; if not, the stage the study has reached should be determined, based on the following criteria: 

Development (identifying the evidence, theory development, modelling)



Feasibility/piloting (testing procedures, estimating recruitment, determining sample size)



Evaluation (assessing effectiveness, understanding change process, assessing costeffectiveness)



Implementation (dissemination, surveillance and monitoring, long term follow-up)

6. Is it a study into a nursing intervention? (Y/N) Selected studies will be categorised into studies that are nursing intervention research and those that are not. Nursing intervention research is defined as, “studies either questioning existing care practices or testing innovations in care that are shaped by nursing’s values and goals, guided by a strong theoretical basis, informed by recent advances in science, and designed to improve the quality of care and health of individuals, families, communities and society” (Naylor, 2003).

Missing Data Where crucial information is ambiguous or missing, attempts will be made to contact the author(s) of the study. If outcome data are missing, attempts will be made to contact authors. If no contact is possible and the required data are unavailable, we will exclude the study from the main review although the results will be described qualitatively.

 

8

Data Synthesis The data extracted from the included studies will be presented as follows: 1. Percentages will be calculated from raw values to show the percentage of studies that are primary research and the percentage that are secondary research. Both raw values and percentages will be presented. Furthermore, of the studies that are classified as secondary research, frequency counts pertaining to the five subcategories of secondary research will be calculated and presented. 2. The frequencies of each experimental study subtype and observational study subtype will be calculated and presented. Also, studies will be subdivided into quantitative and qualitative studies based on whether the data collected was numeric or not. A study encompassing the use of both qualitative and quantitative methods will be termed mixed-methods research. Where one type of method alone is used it will be classified as single method. Percentages, along with raw values of how many quantitative, qualitative, mixed-methods and single method studies will be calculated and presented. 3. Percentages from raw values will be calculated to show the percentage of studies that are categorised as programmatic research. Both raw values and percentages will be presented. 4. Percentages from raw values will be calculated to show the percentage of studies that are categorised as translational research and both will be presented. Furthermore, of the studies that are classified as translational research, frequency counts pertaining to the three phases of translational research will be calculated and presented. 5. Frequency counts of where the included studies stand in relation to the four stages of the MRC guidelines will be calculated and presented. 6. Percentages from raw values will be calculated to show the percentage of studies that are categorised as nursing intervention research and those that are not. Both raw values and percentages will be presented. The results presented will be accompanied by a narrative analysis. The narrative will be written by the lead reviewer and then checked independently by two other reviewers who will then feed back with comments. Any disagreements will be decided by all three reviewers.

References Centre for Reviews and Dissemination. (2009). Systematic Reviews. CRD’s guidance for undertaking reviews in health care. York: Centre for Reviews and Dissemination, University of York. Available from: http://www.york.ac.uk/inst/crd. Accessed 27 November 2011. Chalmers I, Glasziou P. (2009) Avoidable waste in the production and reporting of research  

9

evidence. The Lancet, 374, 86 – 98. Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337: a1655. Hallberg, I. (2009) Moving nursing research forward towards a stronger impact on healthcare practice? Int.J.Nurs.Stud, 46, 407 – 412. Higgins, J. P. T., Green, S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2009. Available from www.cochrane-handbook.org. Mantzoukas, S. (2009). The research evidence published in high impact nursing journals between 2000 and 2006: A quantitative content analysis. International Journal of Nursing Studies, 46, 479 – 489. Morse, J. M. (1994). “Emerging from the data: the cognitive process of analysis in qualitative enquire”, in Morse, J M. (Ed.), Critical Issues in Qualitative Research Methods, Sage, Thousand Oaks, CA. Naylor, M. D. (2003). Nursing Intervention Research and Quality of Care: Influencing the Future of Healthcare. Nursing Research, 52, 380 – 385. Polit, D., Beck, C. (2006). Essentials of Nursing Research, Methods, Appraisals, and Utilization. Philedelphia, PA: Lippencott, Williams and Wilkins. Richards D, Borglin G. (2011) Complex interventions and nursing: Looking through a new lens at nursing research. Int.J.Nurs.Stud, 48, 531 – 533. Woolf, H. S. (2009). The Meaning of Translational Research and Why it Matters. JAMA, 299, 211 – 213.

 

10

Appendix A – Electronic Search Terms Search terms will be developed with the aid of an information specialist.

 

11