Interpreting Research in Nephrology Nursing Evaluating Quantitative Research Reports Cynthia L. Russell

Editors Note: This is the third of a series of columns contributed by the ANNA Research Committee to assist nephrology nurses in understanding research approaches and methodologies and evaluating research, .uantitative research is evaluated for several reasons. You may be trying to decide if the findings are worihy of incorporation into your practice or you may ^ attempting to determine the current state of the reseaK:h in a particular area. In the evaluation process, you will objectively review the strengths and the weaknesses of a report. Ultimately, you will determine whether the strengths of the work are greater than its weaknesses, whether the results can be incorporated into practice, and where the fmdings direct the next study. Evaluating a quantitative research report may initially seem like a daunting task, even if you regularly read research. However, by using a systematic approach, you can become more comfortable and proficient in evaluating quantitative research. Numerous articles have been published on evaluating research reports (Beck, 1990; Pieper, 1993; Ryan-Wengar, 1992; Soeken, 1985; Summers, 1991). In addition, multimedia CD-ROMs have become available to assist with critiquing research for clinical practice use (Alderman, 199H; Beyea, 1998}. The purpose of this article is to provide the knowledge and tools needed to successfully evaluate quantitative research reports.

Components of a Qualitative Research Report ihoLigh journal criteria vary, most qualitative research reports contain five sections: research problem, review of the literature, methods (design, sampling plan, instrumentation, procedure, human subjects protection), data analysis and results, and discussion.

needs to be studied, including both who and what will be studied. A problem statement example follows: Because of the long waiting time, there is a need for nursing interventions to assist individuals waiting for deceased donor renal transplantiition. After you read the problem statement, you should have an idea of what completion of the study included. You should detennine whether the problem makes a significant contribution to the science and whether it is relevant to your practice. R e v i e w of the Literature i This section provides the foundation for helping the reader understand what the current state of the evidence is in the selected area of study. The review of the literature should present pertinent findings from selected research reports in an organized and clear fashion. Frequently, reviews are organized by headings thett correspond to key study concepts. The review must also present an evaluation of the quality of the pertinent literature noting strengths and weaknesses. As you read the review of literature, it should move from the broad to the specific with the last section of the review clearly delineating the need for the study. For example, a review of literature on interventions for those awaiting deceased donor renal transplantation would begin with a section summarizing literature on the experience of waiting for a tnmsplant imd then move to a section on nursing interventions used to assist those waiting. The review of the literature would then conclude with the statement of the gaps in existing literature and how the current study will address those gaps. The review of the literature may contain a section on the theoretical or conceptual framework. If presented, you should assess whether the theoretical or conceptual framework is clearly described including concepts and relationships. The problem statement should flow directly from the theoretical or conceptual framework.

Research Problem The introduction of the report .should thoroughly describe the background of the research problem so tbat the need for the study is apparent. The author must build a case from the existing literature that the problem is of sufficient merit to justify further research. For example, if the study's puipose is to measure the effect of infonnation and support on hope and uncertainty in individuals awaiting deceased donor renal tiansplantation, the introduction would describe the number of people waiting for renal transplant, the potential impact that interventions could make in this area, and the problems with extent research. The statement of the problem should flow directly from the introduction and should conclude this section. The statement of the problem broadly identifies what Cynthia L. Russell, PhD, RN, AfSCNS, is Assistant i\ofessor. University of Missouri- Columbia, Sinclair School of Nursing, Columbia, MO; and a member of ANNA's Central Missouri Chapter.

I

I

Methods The methods section describes the steps used by the reseaicher to carry out the study. This section includes the design, sampling plan, instrumentation, procedure, and the protection of human subjects. Design. The design delineates the plan or blueprint of the study. Non experimental designs, which include descriptive and correlational designs, examine phenomena as they naturally occur, so no manipulation is involved. A descriptive design allows the researcher to describe the This column is compiled by the ANNA Research Committee to assist nephrology nurses in understanding research approaches and methodoiogies and evaiuating research. For additional information, contact Patricia A. Cowan, PhD. RN; ANNA Research Committee through the ANNA National Office; East Holly Avenue/Box 56; Pitman. NJ 08071-0056; [email protected].

NEPHROLOGY NURSING JOURNAL • January-February 2005 • VoL 32, No. 1

61

Evaluating Quantitative Research Reports

Table 1 Criteria for Research Report Evaluation Research Problem 1. is the problem clearly stated? 2. Is the problem significant? Review of the Literature 1. Is the literature summarized? 2. Is the literature critically evaluated? 3. Are gaps and inconsistencies in the literature described? 4. Is the literature current and the review complete? 5. If presented, is the theoretical or oonceptual framework clearly described including concepts and relationships? 6. Does the problem clearly link to and flow from the theoretical or conceptual framework? Design 1. Is the design clearly stated? Sample 1. 1. Is the sample clearly identified? 2. Is it clear how the sample will be obtained? 3. Is the relationship between the sample and the target population clearly delineated? 4. Is the rationale for the sample size provided? Instrumentation t. Is it clear which instruments will measure which concepts? 2. Is the rationale for instrument selection acceptable? 3. Is the reliability for each instrument described and acceptable? 4. Is the validity for each instrument described and acceptable? Procedure 1. Are sufficient details provided in the procedure? 2. Is the procedure written clearly? 3. Does the procedure flow logically? 4. Are al! steps of the procedure clearly stated?

77ie sample included the first 50 participants agreeing to participate who were on the deceased donor renal transplantation waiting list at a university-affiliated hospital in the Midwest

Protection of Human Subjects 1. Has the researcher provided sufficient protection of human subjects? Data Analysis and Results 1. Is the data analysis section well organized? 2. Is the statistical method used for analysis appropriate for the research question{s) and/or hypothesis and level of measurement? 3. Are tables and graphs clearly labeled? 4. Do the tables and graphs complement the text? Discussion 1. Does the discussion clearly flow from the data? 2. Does the discussion place the study's findings in context with what is already known? 3. If a theoretical or conceptual framework is presented, are the nature of the findings discussed in the context of the framework? 4. If the author presents interpretations of the findings are these clearly distinguished as such? 5. Are justifications offered for the author's conclusions? 6. Are study limitations provided? 7. Are implications for practice and future research delineated?

62

characteristics of the sample, while a correlational design assists the researcher in examining relationships between variables (Polit & Beck, 2004). On the other hand, experimental designs involve three key components: (a) manipulation of the independent variable, (b) use of a control gioup, and (c) randomization into groups (Polit & Beck, 2004). Experimental studies use the most powerful designs. These designs allow the researcher to control for extra variables that may interfere with the researcher's ability to tell if the measured effect was due to the manipulatioii of Ihe independent variable or due to interference from tlu- undesired extra variables. Quasi-experimental designs lack one of the three key experimentaJ components. The reader can anticipate the data analysis plan once the design is known. For example, if an experimental study is planned, the reader can anticipate use of inferential statistics such as t-tests for data analysis. If a descriptive study is planned, then descriptive statistics are anticipated e.g. means, modes, medians. All studies have research questions. Research questions goiide and direct the study. A well-developed research question includes the population and variables to be studied. At the completion of the study the research questions should be answered. Correlational, quasi-experimental, and experimental studies also have hypotheses. A hypothesis is a statement of the relationship between variables predicted by the researcher. Sampling plan. Thi.s section must clearly describe who was asked to participate in the study and how they were identified, the characteristics of the target population (tlie population to which the findings are generalized), the sampling procedure, and the size uf the sample. An example of a well-developed sampling plan follows:

(Russell & Brown, 2002, p. 202). Instrumentation. The instruments used to gather data for the study must be clearly and thoroughly described. The researcher should delineate which concepts each instrument will measure. Instrumentation may involve interviews, questionnaires, scales, observation, and/or biophysiological measures. Reliability and validity data should be reported for each instrument. Reliability is the instilment's ability to accurately and consistently measure the concept (Brink & Wood, 2001). Validity is the instni ment's ability to measure what it is supposed to measure (Brink & Wood, 2001). The following is an example of a well-developed description of an instrument: Depression was measured using the Beck Depression Inventory (BDI) (Beck et al., l!)(il). This 21 item self-administered, selfreport scale addresses mood, pessimimi, sense of failure, lack of satisfaction, guilty feeling, sense of punishment, self-hate, selfaccusation, self- punitive wishes, crying spells, irritability, social withdrawal, indedsiveness, body image, work inhibition, sleep disturbance, fatigability, loss of appetite, iveight, loss, somatic preoccupation, and loss of libido. The BDI has high internal con-

NEPHROLOGY NURSING JOURNALm January-February 2005

32, No. 1

sistency with ranges from .73 to .92 with a mean of .86 Steer, & Garbin, 1988). The BDI has a split- half reliability co-efficient of.93 (Beck et ai., 1988) (Russell & Brown, 2002). Procedure. The procedure should be the "recipe" for the reseaich process with sufficient details provided so that you can easily follow the process. The procedure should be written very clearly and flow logically. All steps of the procedure should be describedfially.An example follows.

descriptive statistics follows:

Participants were randomly assigned to either the control group or the treatment group. Those placed in the control group received no intervention phone calls or mailings, which was the current .standard of care. Those randomized into the treatment group received support, which included phone calls and mailings, once every month for six months. Because the current average waiting time at the institution was 8 months for blood ^oup A, and longer for other blood groups, a six month intervention was selected. During the phone calls, patients were asked if they had any questions or concerns that they woutd like to ask about waiting on the transplant list. TTie researchers documented key words and phrases stated by the subjects in response to the questions. The mailings were spomored by Signature Pharmaceuticals. This program involved sending cm initial welcoming letter and subsequent newsletters which provided information on pertinent transplantation issues such as medications, diet, exercise, organ allocation, waiting times, and current media topics. A web site, which could be accessed for information, was also provided by Signature. Both the control and treatment groups completed the Herth Hope Index (HHI) and the MisheVs Uncertainy in Illness for Adults Scale (MUIS- A) at the beginning of the study and six months later (Russell & Brown, 2002, p. 2()H).

(SD = 1084, range 1-4752) (Russell & Brown, 2002, p. 20H). If the design is experimental or quasi-experimental, you should anticipate the use of inferential statistics. Inferential statistics answer questions about relationships between variables and differences between groups (Holcomb, 2002). An outstanding quick reference guide for assessing an author's appropriate use of statistics based on the research question and level of measurement is available (Ryan-Wengar, 1992). Many studies set the level of statistical significance as pr Gertice Hoicomb, Z.C. (2002). Interpreting basic .statistics (3rd ed.). Los Aiiiic Ciiacovi'lli Angeles: Pyrczak Publishing. JamesMiiieo RECRUITING •-'''""/''" Pieper, B. (l^O^^). Basics of critiquing a research article. Joumai of ET Nursing, 20, 24.V 2S0. Jeff Kozma Kelly Winder Polit, D., & Beck, C.T. (2004). Nursing research: Principles and methods (7th cd.). Philadelphia: Lippincott, Williams, & Wilkins. .Amly Lawrence jt>h,i faon, Russell, C.L, & Brown. K (2(K)2). The efFecls of information and support on individuals awaiting transplant. Process in Transptaniaiimi, 72(3), 201-207 Ryan-Weng-ar, N.M. (1J)!)2). Guidelines for critique of a research report. Heart & Lung, 21(4), 35H-4O1. Soeken, K.L. (U)85). Critiquing research: Steps for complete evaluation on an article. AORNJoumal, 41[5\ 882-893. Stimmers, S. (li)SM). Defining components of the research process ^'Dialysis & Transplantation is our only busiDcss." needed to conduct and critique ^\ixdie%. Joumai of Post Anesthesia (315)768-3322 * Fax: (315)768-4349 * Toll Free: (888) 38-RENAL Nursing, 6(1), 50-55.

SPECIALISTS

Nurses!

v^'W'w, dialysis recni iting.com

REGISTERED NURSES & HEALTH CARE PROFESSIONALS:

Make your job work for you. Join us and remember why you got into nursing in the first place. What does it mean to be a Magnet Hospital? To be considered one of the finest hospitals in New Jersey? We like to think it's a lot of things.

ACUTE AND CHRONIC DIALYSIS FACILITIES NEED YOU.

An environment, culture and a philosophy committed to the advancement of the profession and the individual and one that fosters continual growth, learning and development.The constant challenge that comes with working with a diverse patient load. Respect. Flexibility.To be honest, there are as many reasons as there are nurses,

WE'RE HERE TO MAKE IT HAPPEN. -

and we invite you to find out for yourself.

Clinical Coordinator, Hemodialysis Our Hemodjalysis program is one of the finest in New Jersey and in ouf newly constructed units we treat both the adult and pediatric

Quik Travel Staffing

populations affected by acute and chronic conditions.

The flexible, friendly, in-your-corner healthcare travel e m p l o y m e n t company, offering t e m p o r a r y and permanent assignments with chronic and acute Dialysis facilities. You're serious about your career— and we're serious about getting the best situation for y o u .

We are seeking a coordinator to manage the overall activities of the Hemodialysis unit. Must have 3-5 years of hemodialysis experience and 2-3 years of supervisory experience, and BCLS/ACLS and CNN certifications. To find out what's been attracting some of the state's best nurses to the magnet hospital known ds Morristown Memorial, contact us today. Apply online at www.atlanticKealth.org or send your resume to [email protected]. Equal Opportunity Employer.

^ ^ Morristown Memorial Hospital y \ r i A M T ( c :

H

E

A

L

T

H

www.atLantichealth.org

S Y S T E F V I

COMMIHED TO TOP SALARIES & GREAT BENEFITS COMPLETION & REFERRAL BONUSES FREE HOUSING, TRAVEL & LOCAL TRANSPORTATION PERSONALIZED, ROUND THE CLOCK SERVICE

810-454-2230 NEPHROLOGYNURSINGJOURNAL January-February2005

Vol. 32. No. 1

^'"