Involving nursing students into clinical research projects: Reliability of data and experiences of students?

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. © 2020 The Authors. Journal of Clinical Nursing published by John Wiley & Sons Ltd 1Department of Nursing and Health Promotion, Faculty of Health Sciences, Oslo Metropolitan University, Oslo, Norway 2Institute of Clinical Medicine, Campus Ahus, University of Oslo, Blindern, Norway 3Health Services Research Unit, Akershus University Hospital, Lørenskog, Norway

The proportion of older citizens in the population is steadily increasing, and malnutrition is already creating high costs at both individual and system levels (Rojer et al., 2016;Volkert, Beck, Cederholm, Cereda, et al., 2019).
Nutrition is a basic nursing care, and nurses play a key role in monitoring patients' nutritional needs and identifying patients who are undernourished or who are at risk of malnutrition (Cate et al., 2019;. Despite knowledge about the importance of good nutritional status, weaknesses in the treatment of malnutrition have been documented (Norman et al., 2008;Ray et al., 2014).
Learning is a subjective process that takes place through activity and reflection in encounters between students and teachers, not through students passively receiving theoretical knowledge (Biggs & Tang, 2011). Faster implementation of practice in new methods, guidelines and quality indicators can be achieved by familiarising students with them as part of their clinical training. In this way, they can increasingly integrate evidence-based practice into their daily work and thereby better safeguard patients' basic nutritional needs.
Rapid changes in the healthcare system require that nurse graduates bring new knowledge to the practice setting (Cusson, Meehan, Bourgault, & Kelley, 2020). Nursing students are potential agents of change for better practice if they receive good training in preventing and treating malnutrition during the course of their training. To gain an understanding of evidence-based practice in their future nursing role, it is important that nursing students participate in clinical research projects (Muraraneza, Mtshali, & Bvumbwe, 2020;Ramsay, Wicking, & Yates, 2020).

| BACKG ROU N D
As part of a large-scale interdisciplinary clinical research project entitled 'Nutritional status and care of older patients', 173s-year nursing students performed nutritional screening on 508 patients (48.8% women) aged over 70 as part of a coursework requirement during their hospital practice placement Eide, Saltyte Benth, Sortland, Halvorsen, & Almendingen, 2015;Eide, Šaltytė Benth, Sortland, Halvorsen, & Almendingen, 2016;. A total of 20 units belonging to either a medical department or a surgical department participated in the study.
Stratification was used to increase the representativeness of the sample. The coursework requirement requires completion of the Nutrition Risk Screening (NRS) 2002 form (Kondrup, Rasmussen, Hamberg, & Stanga, 2003), weighing and measuring patients, and collecting relevant documentation from electronic health records.
In the first year of study, students have studied nutrition and nutritional status in the simulation laboratory in smaller groups of approximately 15 students. They were taught the symptoms of malnutrition, the risk of developing malnutrition, dehydration and malnutrition. Moreover, they received teaching and practical training in how to weigh, measure and calculate BMI. They were also trained to calculate weight loss and weight gain in %. The students were trained to conduct a dietary interview and mapped each other's nutritional status using various screening tools. They were taught how to calculate energy and fluid needs based on individual patient needs. Particular emphasis was placed on how to apply Nutritional Risk Screening (NRS 2002) and on energy and nutrient dense diets. They were trained to increase the energy and nutritional content of various dishes by adding cream, butter and eggs, without affecting the taste of the dish. They also acquired knowledge about various nutritional drinks and snacks, and how they could enrich snack foods. As part of the study, students measured patients' weight and height when possible by using measuring equipment available in the departments. The students worked together, often in pairs, and checked that their measurements were correct. Each student participated in at least two of nine nutritional screening days. The results showed that as much as 45% of the patients examined at the hospital were at nutritional risk.
In guidelines from 2009, the Norwegian authorities recommended K E Y W O R D S hospital care, learning outcomes, nursing students, nutritional screening, older patients, research, undernutrition What does this paper contribute to the wider global clinical community?
• Older hospitalised patients are considerably at risk of malnutrition.
• Research collaboration with nursing students in their hospital practice on nutritional risk screening provide reliable data.
• Participation in research increases nursing students' engagement for improved nutritional care practices. that all patients who are admitted to hospital be screened for malnutrition on admission by NRS 2002 and weekly thereafter or as part of an individual, medically based plan (Norwegian Directorate of Health, 2009). Our study showed that recommended nutritional care practice in compliance with national and international guidelines was not implemented in clinical hospital practice. Older patients were almost never assessed for nutritional risk, malnutrition and nutritional risk was clearly undertreated, and documentation of nutritional status and treatment was unsatisfactory. Overall, the results show that many older hospitalised patients do not receive the nutritional care they need and that a need as basic as nutrition seems to be ignored and given little priority. The involvement and use of nursing students in the research project were substantial.
Since malnutrition threatens patient safety and quality and makes proper nursing practice more difficult, we wanted to examine the quality of the screening data to which the students contributed and which formed the basis for our clinical study.
The aims were to assess reliability of the screening data collected by second-year students in a bachelor programme in nursing (n = 30). Furthermore, to examine second-year nursing students' (n = 173) evaluations of participation in nutritional screening of older hospitalised patients.

| Reliability test
Two second-year nursing students (S1 and S2) who were not part of the ordinary screening (Eide, Saltyte Benth, et al., 2015) performed screening of 30 selected older patients ≥70 years admitted to the surgical and medical departments in a Norwegian hospital.
Each patient's age, height, weight, body mass index and nutritional risk were measured three times: first by one of the ordinary 173 nursing students (SS) and thereafter by S1 and S2. The results of the reliability tests were recorded on separate forms with different numerical codes and colours. These forms were linked with the ordinary screening form completed by SS earlier the same day. The students who participated in the screening received an envelope with a screening form and detailed information on how the screening should be carried out. All the envelopes contained identical nonelastic measuring tapes. The patient's height was measured, while the patient stood upright against a wall, without shoes. The patient's heels were positioned against the wall, and the patient was asked to stand as erect as possible. The height was measured to the nearest centimetre. If the patient could not stand upright, the patient's half arm span was used. The patient's weight was measured to the nearest 0.1 kg, without shoes and outer clothing, in either standing or sitting position, using the department's regular equipment (not calibrated weighing scales). The time lapse between the measurements was no more than approximately 2 hr.
Based on the patient's height and weight, the students calculated their body mass index (BMI), defined as weight (kg) divided by the square of their height (m) (kg/m 2 ). Nutritional risk was assessed based on nutritional status and disease severity on a score from 0-3 (Kondrup et al., 2003). Normal nutritional status and no disease gives a score of 0; weight loss of 5%-10% over the past 3 months and/or 50%-75% of required dietary intake for more than 1 week give a score of 1; weight loss of 10%-15% over the past 3 months and/or BMI of 18.5-20 kg/m 2 and/or 25%-50% of required dietary intake for more than 1 week give a score of 2; and weight loss >15% over the past 3 months and/or BMI < 18.5 kg/m 2 and/or 0%-25% of required dietary intake for more than 1 week give a score of 3. A clinical nutritionist and teaching coordinator were on hand for the nursing students (SS, S1 and S2) on the screening days.

| Feedback from the nursing students
In addition to testing the reliability of the screening procedure itself, we wanted to gain knowledge about the students' experiences of screening older patients admitted to hospital for nutritional risk.
At the end of each academic year, an anonymous electronic questionnaire was sent to all the students who had participated in the nutritional screening (n = 173) asking about their experiences when performing nutritional screening. We conducted this survey using QuestBack, a web-based survey and reporting tool where data collection is performed via email containing a link to an online questionnaire. The programme protects the students' anonymity. By using QuestBack, we wanted to collect data on what challenges the students encountered when performing nutritional screening. They were asked questions about how easy it was to measure height and weight, find details about the patient's normal weight and previously recorded weight, and what it was like using the NRS 2002 screening form. They were also asked questions about how instructive it had been to participate in the nutritional screening days and whether the nutritional screening had increased focus on nutrition among the students themselves and among healthcare personnel in the departments where they had their practice placements.

| Ethical principles for medical research
The study was carried out in compliance with the guidelines in the Helsinki Declaration and was approved by the data protection officers at the university hospital participating in the study. The study was developed with multidisciplinary cooperation between the researches, the collegium at the bachelor nursing programme, representatives from the university hospital and other experts in the field. The hospital management, the older patients, the nursing staff and the nursing students were provided with written and verbal information about the study. Verbal informed consent was obtained from the older patients. Given that the data were anonymised, the study was exempt from assessment by the Regional Committee for Medical and Health Research Ethics.

| Statistical analyses
The continuous variables age, weight, height and BMI were presented as means and standard deviations (SD), while frequencies and percentages were used to describe nutritional status. Agreement between measurements of age, weight and height performed by three students were assessed graphically in a Bland-Altman plot and by calculating 95% limits of agreement (LoA). LoA define an interval where 95% of the differences between measurements in a population are expected to lie. Acceptable LoA were predefined as ±1 year for age, ±2 kg for weight and ±3 cm for height. Bias, defined as the mean difference between the measurements performed by two students, was tested by one-sample t test. Weighted kappa statistic with a 95% confidence interval was used to assess the agreement between NRS 2002 scores. The students' experiences after the screening were summarised in the form of frequencies and percentages. The statistical programme IBM SPSS Statistics version 26 for Windows was used to perform all the statistical analyses. The results with p-values less than 5% were regarded as statically significant. All tests were two-sided. The EQUATOR STROBE Statement-checklist of items that should be included in reports of observational studies is followed (see File S1). All analyses were performed on anonymised data by a statistician.

| Reliability test
Descriptive statistics showed that the differences in mean age, weight, height or BMI measured by S1 and S2 were marginal (Table 1). Consequently, no significant bias was found between S1 and S2 for weight, height and BMI, while age was measured equally by S1 and S2 resulting in no bias at all (Table 2). However, the differences between SS and S1/S2 were slightly greater, with small but significant bias in weight and BMI (Table 2). No significant bias was found in measurements of height. According to the LoA, 95% of differences between SS and S1/S2 in age ( Figure 1) and differences between SS and S1/S2 and between S1 and S2 in weight ( Figure 2) and height ( Figure 3) are expected to lie in a wider interval than the predefined limits. For BMI, 95% of LoA were also quite wide (Table 2 and Figure 4), even though no predefined values were set. On the other hand, it appears as if the wide LoA are mostly due to very few but large deviations between two measurements. Interestingly, the 95% LoA were notably narrower when assessing agreement between S1 and S2 than between SS and S1/ S2. The weighted kappa (Table 2) for NRS 2002 scores indicates poor agreement between SS and S1/S2 and between S1 and S2, and the wide confidence intervals reflect few samples with paired NRS 2002 scores.

| Feedback on what it was like to participate in the research project
In total, 82 (47.4%) of 173 students who participated in the nutritional screening completed the QuestBack survey (Table 3). Of these, a total of 70.5% responded that they partly or fully agreed that it was easy to use the NRS 2002 screening form. A slightly larger proportion of the students partly or fully agreed that it was easier to measure patients' height (59.0%) than to measure patients' weight (52.6%). A total of 41.0% of the students responded that they partly or fully agreed that it was easy to find information on normal weight from patients, while as much as 39.7% of the students responded that they partly or fully disagreed that it was easy to find information on normal weight from patients. As much as 70.5% of the students responded that they partly or fully disagreed that it was easy to find information on patient's previously recorded weight in records. A total of 37.2% of the students partly or fully agreed that participating in the nutritional screening was instructive, and 34.1% of the students partly or fully agreed that their interest in nutrition increased after participating in the nutritional screening. Only 12.8% of the students responded that they partly or fully agreed that the TA B L E 1 Descriptive statistics for the ordinary nursing students (SS) and the two students (S1 and S2) who conducted the reliability analyses for the hospitalised patients aged over 70  (Kondrup et al., 2003): normal nutritional status and no disease gives a score of 0; weight loss of 5%-10% over the past 3 months and/or 50%-75% of required dietary intake for more than 1 week gives a score of 1; weight loss of 10%-15% over the past 3 months and/or BMI of 18.5-20 kg/m 2 and/or 25%-50% of required dietary intake for more than 1 week gives a score of 2; weight loss >15% over the past 3 months and/or BMI < 18.5 kg/m 2 and/or 0%-25% of required dietary intake for more than 1 week gives a score of 3.  (Kondrup et al., 2003): normal nutritional status and no disease gives a score of 0; weight loss of 5%-10% over the past 3 months and/or 50%-75% of required dietary intake for more than 1 week gives a score of 1; weight loss of 10%-15% over the past 3 months and/or BMI of 18.5-20 kg/m 2 and/or 25%-50% of required dietary intake for more than 1 week gives a score of 2; weight loss >15% over the past 3 months and/or BMI < 18.5 kg/m 2 and/or 0%-25% of required dietary intake for more than 1 week gives a score of 3.

TA B L E 2
Reliability test results presented as bias (mean difference between two measurements) between two students and 95% limits of agreement (LoA) and as weighted kappa with 95% confidence interval (CI) F I G U R E 1 Bland-Altman plot for agreement between students in age (weight, height, BMI) measurements. Straight line shows bias, while dashed lines show 95% limits of agreement nutritional screening had increased focus on nutrition in the department where they had their practice placements.

| D ISCUSS I ON
Previously, in a large cross-sectional study (Eide, Saltyte Benth, et al., 2015;Eide et al., 2016), it was shown that there was a strong need and great potential for quality improvement in nutritional care practice to ensure that malnutrition among older hospitalised patients is sufficiently prevented and treated. All the data were collected by the nursing students in practice, and the validity of the conclusions therefore depends on the reliability of the data they submitted. Methodical challenges can arise when many individuals working in different wards perform nutritional screening (Rasmussen, Holst, & Kondrup, 2010). We therefore conducted a reliability analysis in order to say something about the quality of the data collected by the nursing students. Our findings show that the ways in which the students screened the patients were relatively similar, and the QuestBack survey showed that they found participating in the screening easy.
The students experienced challenges in finding previously recorded information on weight and data on the patient's normal weight. Because malnutrition can occur from rapid weight loss, it is important to collect information about the patient's normal weight . However, our study shows that patients are not weighed on admission or readmission to hospital. Systematic measurements of weight and height constitute objective goals in assessments of nutritional risk and mal-  (Kondrup et al., 2003). This may explain why more than two/thirds of the students responded that they fully or partly agreed that it was easy to use the NRS 2002 nutritional screening form. This is a positive finding, because preventing and treating malnutrition should be part of their work (Cate et al., 2019). By making screening a compulsory coursework requirement in the programme, we gave our students relevant job-oriented training and work experience in evidence-based clinical nursing practice.
A potential model for developing nursing students' methodical competence and ability to work according to evidence-based practice could be where the teaching staff at universities and university colleges have several research projects in clinical practice together with practice placements, where the teaching staff supervises and the bachelor students actively contribute. In this way, competence building would be achieved both in the study programme and in the practice placement, which we believe will generate a greater sense of ownership among both students and practice placements. At the hospitals, research is part of their activities, and professional and research nurses are key actors and collaborative partners. This will bring training, research and practice closer together.
The students had different opinions about how instructive it had been to participate in the nutritional screening days. On the other hand, two-thirds of the students responded that they fully or partly disagreed that the nutritional screening had increased focus on nutrition in the departments. This may reflect a lack of sense of ownership of the research project on the part of the departments, but also that less focus is placed on nutrition than on other aspects. Nurses also reported that little time or priority was given to nutrition in a busy working day (Eide, Saltyte Benth, et al., 2015). A recent study showed that overall attitudes of nursing students did not change after experiencing a semester long introductory research subject (Ramsay et al., 2020). Students may find the research topic to be challenging to learn, while academics find it challenging to engage students in learning about research (Ramsay et al., 2020).
It is interesting to note that the students reported that it was easy to find information on diagnoses in patient records yet not so easy to find information on nutritional measures. Could it be the case that more focus is placed on diagnosis than on nutrition?
Nurses shall conduct their work in accordance with the requirements to professional responsibility and diligent care that can be expected based on their qualifications, the nature of their work and placement staff could conduct assessments in addition to the students, and quantitative studies could be combined with qualitative studies. One major study strength is that we actually tested the reliability of the student screening data and that the students obtained prevalence data in agreement with data from international studies in which skilled nurses and other health professionals were responsible for the screening. This study highlights that it is right to trust the results obtained from nutritional screening performed by students (Eide, Saltyte Benth, et al., 2015;Eide et al., 2016).
Students practised the research techniques they learned, applying didactic content to clinical application. This is a win-win situation where student research is good for the nursing student and for the patient. We found no other studies that have investigated nursing students' reliability in using NRS 2002 or in performing anthropometric measurements. Despite the study limitations, the present study documents the need for nutritional training of nursing students both in the simulation lab and in a clinical setting in order to prevent and treat undernutrition.

| CON CLUS ION
The students delivered nutritional screening data of high quality, and they responded that nutritional screening was easy to perform. The results from the cross-sectional must therefore be deemed to hold high quality and to be valid. However, the results indicate that there is great potential for increased focus both on research and on nutrition. To the best of our knowledge, this is

| RELE VAN CE TO CLINI C AL PR AC TI CE
Collaborating with nursing students contributes with valuable data for practice and research. Moreover, it increases students' engagement for improved nutritional care practices for older patients.

ACK N OWLED G EM ENTS
Thanks to the students and staff at both the hospital and the university. A special thanks to Helene K Eide, Clinical Dietitian.

CO N FLI C T O F I NTE R E S T
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article. Note: Response alternatives: 1-Fully disagree, 2-Partly disagree, 3-Neither agree nor disagree, 4-Partly agree, 5-Fully agree, −1-Don't know.