Development of the Hybrid Clinical Practicum Environment Scale for Nursing Students

Seoyoung Yoon; Hye-Ah Yeom

doi:10.4040/jkan.24016

Articles

Page Path: HOME > J Korean Acad Nurs > Volume 54(3); 2024 > Article

Research Paper Development of the Hybrid Clinical Practicum Environment Scale for Nursing Students: Seoyoung Yoon, Hye-Ah Yeom; Journal of Korean Academy of Nursing 2024;54(3):340-357.
DOI: https://doi.org/10.4040/jkan.24016
Published online: August 31, 2024

¹Accreditation & Evaluation Team 2, Korean Accreditation Board of Nursing Education, Seoul, Korea

²College of Nursing, The Catholic University of Korea, Seoul, Korea

Corresponding author: Hye-Ah Yeom,
Email: yha@catholic.ac.kr

prev next

2,719 Views
90 Download
1 Crossref
0 Scopus

prev next

Full Article

Download PDF

Abstract

Purpose
This study aimed to develop a Hybrid Clinical Practicum Environment Scale for Nursing Students (HCPES-NS) and verify its validity and reliability.
Methods
The HCPES-NS was constructed following the DeVellis guidelines. The initial items were written based on a literature review and individual in-depth interviews. Content validity was verified through an expert panel review. To confirm the validity and reliability of the scale, a survey was conducted with 449 nursing students enrolled in 12 nursing colleges. Data were analyzed using item analysis, exploratory factor analysis, confirmatory factor analysis, concurrent validity, and reliability tests.
Results
Factor analysis showed that the HCPES-NS consists of 15 items on five subdomains: clinical site atmosphere, interpersonal relationship, alternative online practicum contents, provision of learning information, and clinical performance facilitation. A higher score indicated a more positive perception of the clinical practicum environment. The concurrent validity of the HCPES-NS was confirmed by its positive correlation with the Clinical Learning Environment Scale (r = .77). The Cronbach’s α reliability of the HCPES-NS was .84.
Conclusion
The HCPES-NS is both valid and reliable. This scale reflects the clinical practicum environment and includes an online practicum factor. It may be used effectively by faculty members and educators to evaluate nursing students’ perceptions of clinical practicum environments.

Keywords: Environment; Students, Nursing; Factor Analysis, Statistical; Reproducibility of Results

J Korean Acad Nurs. 2024 Aug;54(3):340-357. English.
Published online Aug 13, 2024.
https://doi.org/10.4040/jkan.24016

Original Article

Development of the Hybrid Clinical Practicum Environment Scale for Nursing Students

Seoyoung Yoon

,¹ and Hye-Ah Yeom

Author information

Author notes

- ¹Accreditation & Evaluation Team 2, Korean Accreditation Board of Nursing Education, Seoul, Korea.
- ²College of Nursing, The Catholic University of Korea, Seoul, Korea.
Address reprint requests to: Yeom, Hye-Ah. College of Nursing, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Korea. Tel: +82-2-3147-8432, Fax: +82-2-532-6537, Email: yha@catholic.ac.kr

Received February 14, 2024; Revised June 01, 2024; Accepted June 03, 2024.

This is an Open Access article distributed under the terms of the Creative Commons Attribution NoDerivs License. (http://creativecommons.org/licenses/by-nd/4.0/) If the original work is properly cited and retained without any modification or reproduction, it can be used and re-distributed in any format and medium.

Abstract

Purpose

This study aimed to develop a Hybrid Clinical Practicum Environment Scale for Nursing Students (HCPES-NS) and verify its validity and reliability.

Methods

The HCPES-NS was constructed following the DeVellis guidelines. The initial items were written based on a literature review and individual in-depth interviews. Content validity was verified through an expert panel review. To confirm the validity and reliability of the scale, a survey was conducted with 449 nursing students enrolled in 12 nursing colleges. Data were analyzed using item analysis, exploratory factor analysis, confirmatory factor analysis, concurrent validity, and reliability tests.

Results

Factor analysis showed that the HCPES-NS consists of 15 items on five subdomains: clinical site atmosphere, interpersonal relationship, alternative online practicum contents, provision of learning information, and clinical performance facilitation. A higher score indicated a more positive perception of the clinical practicum environment. The concurrent validity of the HCPES-NS was confirmed by its positive correlation with the Clinical Learning Environment Scale (r = .77). The Cronbach’s α reliability of the HCPES-NS was .84.

Conclusion

The HCPES-NS is both valid and reliable. This scale reflects the clinical practicum environment and includes an online practicum factor. It may be used effectively by faculty members and educators to evaluate nursing students’ perceptions of clinical practicum environments.

Keywords

Environment; Students, Nursing; Factor Analysis, Statistical; Reproducibility of Results

INTRODUCTION

Learning refers to the process of acquiring knowledge or change in behavior as a result of experience [1]. In nursing education, clinical practicum is the process of directly applying theoretical material to patients in clinical settings and acquiring practical nursing knowledge. Clinical practicum is a core part of nursing education and allows nursing students to form beliefs, values, and attitudes to become professional nurses after graduation [1]. In Korea, nursing students should take at least 22 credits of clinical practicum courses for graduation [2], imposing nursing colleges the issue of evaluating their needs and satisfaction with clinical practicum.

A clinical practicum environment is a set of stable characteristics that are unique to a particular clinical environment and influence the behavior of individuals within that environment [3]. The clinical practicum environment is an essential component of education because it is where healthcare students can apply theoretical knowledge, acquire clinical skills, and develop problem-solving and clinical reasoning skills [4]. The characteristics of the clinical practicum environment that affect students’ learning experience in nursing education include physical space, psychosocial and interaction factors, organizational culture, and teaching and learning components [5]. In clinical practicum, students rely on a supportive atmosphere that comprises psychological and educational aspects for learning [6, 7], a meaningful learning situation that enables a teacher-student relationship and an educational atmosphere [7, 8, 9]. Individualized guidance and continuous feedback based on a student’s learning needs, goals, and stages of learning are essential factors for a positive learning experience [10, 11, 12]. For example, orientation to clinical practicum by faculty or clinical preceptors (e.g. learning objectives, overall regulations of the clinical sites, clinical practicum schedule and evaluation system, etc.) has been shown to increase students’ satisfaction with the clinical practicum environment [13].

The new interpersonal relationship and environments experienced during clinical practicum can be stressful for nursing students [14]. Nursing students’ satisfaction with clinical practicum can be affected by feelings of welcome and appreciation in inter-professional teamwork [15, 16] and a sense of belonging to the clinical setting [17]. Particularly positive experiences in the clinical practicum environments can influence nursing students’ decision to remain in the healthcare field [5] and increase their favor for future employment in the clinical setting [18], negative experiences can hinder learning outcomes of nursing students and exacerbate the international nurse shortage [19, 20]. For effective clinical practicum, a practicum environment that matches the needs of students must be provided [21], and continuous evaluation of the clinical practicum environment is required [22].

As of 2021, the enrollment capacity of university for nursing in Korea has increased by approximately 66% compared to 2009 [23]. As the number of nursing students increases, securing high-quality clinical practicum settings is an important issue at nursing schools. Most healthcare institutions have indefinitely suspended or deferred clinical practicum for nursing students in consideration of safety during the COVID-19 pandemic situation since 2020 [24]. Thus, changes in the method of conducting clinical practicum have occurred in the clinical practicum settings, such as running an online practicum [25]. These changes in the clinical practicum method have necessitated re-operationalization of clinical practicum environments and incorporating the clinical practicum attributes of the previously developed measurement scales and the newly derived environmental components perceived by nursing students.

The scales currently used to measure the clinical practicum environment of nursing students include the Clinical Learning Environment and Supervision Scale (CLES) [26, 27], the Clinical Learning Environment Inventory [6], the Clinical Learning Environment, Supervision and Nurse Teacher Scale [28, 29], and the Placement Evaluation Tool [30]. Although these scales measure environmental components of clinical practicum and have been widely used in nursing education research, they were developed primarily in the context of onsite clinical practicum and have limitations in measuring hybrid clinical practicum environments that include online components. Additionally, existing scales evaluate the overall quality of clinical practicum environment focusing on social and organizational factors rather than assess students’ subjective perceptions on personal performance and learning activities they experience during clinical practicum. It is necessary to develop an up-to-date scale that reflect the recent changes in clinical practicum environment for nursing students in a multifaceted manner. Therefore, this study aimed to develop the Korean Hybrid Clinical Practicum Environment Scale for Nursing Students (HCPES-NS) that can comprehensively measure the clinical practicum environment recognized by nursing students. We further examined the validity and reliability of the new scale.

METHODS

1. Study design

This study was a methodological study to develop the HCPES-NS, a scale to assess the nursing students’ perception of clinical practicum environment, and to verify the validity and reliability of the scale.

2. Scale development process

The development of the HCPES-NS was carried out in eight steps based on the guidelines by DeVellis [31] (Figure 1).

Figure 1
Steps of scale development.

Download Figure

1) Development of scale

(1) Identify components

The conceptual model of learning environments by Gruppen et al. [32] was used as the conceptual framework of this study to conceptualize learning environments in health professional education. Gruppen et al. [32] divided the learning environment into two main domains: the psychosocial and material environments. The psychosocial domain includes personal, social, and organizational factors. The personal factor at the psychosocial level refers to how individual learners interact with a learning environment and establish their personal growth and professional identity. The social factor relates to interactions with others and engagement in social relationships. The organizational factor includes how individuals interact with policies and respond to organizational performance measures, culture, and leadership [32]. The material domain of the conceptual model refers to physical/virtual spaces factor. The physical space includes spaces where learning or clinical practicum occurs and objects that affect learning. Physical spaces may include university campuses, lecture or conference rooms, and clinical practicum institutions, and refer to both the quantity and quality of spaces related to specific learning activities. Virtual spaces include e-learning context, curriculum management tools, portable devices, and computer networks [32]. Using this conceptual framework, the clinical practicum environment of nursing students was conceptualized into psychosocial and material domains. The conceptual framework was used as a guideline to derive the factors and detailed attributes of the initial items of the HCPES-NS. The constituent factors of each domain were specified through literature reviews and individual in-depth interviews.

The typology of initial items derived from a review of the literature was categorized into two themes, psychosocial aspect (personal, social, and organizational factors) and material aspect (the factor for physical/virtual spaces). The personal factor in the psychosocial domain included items related to nursing students’ intra-personal characteristics and subjective feelings for the clinical sites such as clinical performance, professional identify, aptitude, and safety [1, 2, 4, 5]. The social factor in the psychosocial domain included items related to nursing students’ interpersonal connectedness to the clinical department, which included relationships and communications with patients, preceptors, and the nursing staff at the clinical site [11, 12, 15, 16]. The organizational factor in the psychosocial domain included items related to institutional characteristics that may impact nursing students’ experiences at the clinical site such as institutional support, rules and guidelines, systematic management, and organizational atmosphere [2, 6, 7, 8, 9]. The physical/virtual spaces factor in the material domain involved physical attributes of the clinical site including types and locations of practicum institutions, materials or equipment, and orientation or meeting spaces for nursing students [2, 5, 32].

The attributes of the initial scale items were identified through a review of the relevant literature to specify the detailed components of the clinical practicum environment for nursing students. The literature, either in Korean or English, was searched through domestic and international databases, including Research Information Sharing Service, Korean studies Information Service System, DBpia, Google Scholar, Cumulative Index to Nursing and Allied Health Literature, PubMed, and Science Direct. Search of the literature was performed using keywords including “nursing clinical practicum environment,” “clinical practicum environment,” “clinical learning environment,” “nursing students clinical education environment,” and “nursing student clinical environment.” Literature of published learning environment theories and models, conceptual analysis of the clinical practicum environment and clinical practicum environment measurement scales, and studies on nursing students were included. As a result, a total of 4,681 studies were searched, of which 2,179 studies were selected for primary analysis after excluding 2,502 duplicates. Of the literature pool, a total of 25 studies were finally reviewed after excluding literature in which study participants were not nursing students or the theme was not relevant to practicum in clinical settings.

Individual in-depth interviews were conducted using the content analysis method to confirm attributes of the concept of the clinical practicum environment for nursing students derived from the literature review and to derive the initial items of the scale. The interviews were conducted online by the author. Individual in-depth interviews with ten individuals (4 nursing students with clinical practicum experience, 3 nursing professors in charge of clinical practicum courses, and 3 nurses with more than 5 years of clinical experience) were conducted via online in January, 2022.

The interview took approximately 40 to 60 minutes, once per participant. Among the interview participants, the nursing professors were faculty whose majors were adult nursing, women’s health nursing, and nursing management. The duration of education experiences of the professors were from 3 to 12 years. The nurses were working at general hospitals and had clinical experiences of 6 to 16 years. The main questions of the interview were structured based the conceptual framework of learning environment [32] and contained personal, social, organizational, and physical/virtual spaces aspects of clinical practicum environment for nursing students. Examples of interview questions include, “What do you expect from clinical practicum?”, “What are the important things that you consider in relation to a clinical practicum institution?”, and “How do you feel about changes in your clinical experiences due to COVID-19?”. The interview participants were encouraged to freely discuss their thoughts about what they considered important for each component of the clinical practicum environment from the perspectives of nursing students. Results of the interviews reflected that the four conceptual factors by Gruppen et al. [32] was specified into seven factors. This specification was found in social, organizational, physical/virtual spaces factors. The social factor in the conceptual model was specified into social relations and social atmosphere of the clinical site. Social relations involved interpersonal relationship with clinical preceptors, patients, and nursing staff that nursing students experienced during clinical practicum. Social atmosphere of the clinical site included support and interest to nursing students, teamwork and educational support of the department, and positive attitude toward nursing students. Organizational factor of the conceptual model was specified into provision of learning information and management system for clinical practicum. Provision of learning information involved providing nursing students orientation about clinical department, safety education, and learning contents for clinical practicum. Management system for clinical practicum included consistent preceptorship, explanation and feedback for students’ practicum. Physical/virtual spaces factor of the conceptual model was specified into physical attributes and online practicum component. Physical attributes included distance to clinical site, diversity in clinical institution type, availability of convenience facility, and mealtime. Online practicum contents referred to provision of various learning contents, feedback about online practicum, and perceived helpfulness of online practicum contents. Unlike other factors, personal factor were remained as a single domain, as all of its contents were intra-personal attributes related to individual students’ clinical performance. Throughout the in-depth interviews, the attributes of nursing students’ perception on clinical practicum environments were identified in detail, and the initial pool of 42 priori themes were derived.

(2) Generate on item pool

The 42 themes were refined into sentences in a descriptive way. The items were composited based on the suggestions that a statement did not contain multiple meanings, there were no words with ambiguous meanings, and each item did not inquire about various situations in one item [31]. The constituent factors derived from the literature reviews and in-depth interviews were refined into seven factors: clinical performance facilitation (9 items), interpersonal relationship (6 items), provision of learning information (6 items), clinical site atmosphere (5 items), organizational management system (6 items), physical component (6 items), and alternative online practicum contents (4 items).

(3) Determine the format for measurement

In this study, a 5-point Likert scale (1 point = strongly disagree, 2 points = disagree, 3 points = neutral, 4 points = agree, 5 points = strongly agree) was used to measure the clinical practicum environment perceived by nursing students. The total HCPES-NS score was determined by summing up the scores of all the items. A higher score corresponded to a more positive perception by the nursing students on the clinical practicum environment.

(4) Have initial item pool reviewed by experts

To ensure that the initial items of the HCPES-NS reflected the concept of the practicum environment for nursing students, a group of experts reviewed the contents of the scale. Lynn’s criterion [33] suggested that the number of experts for content validity verification should be 3 or more and 10 or fewer; in this study, a group of 10 experts (4 nursing faculty, 5 nurses, and 1 faculty majoring in education) participated in the evaluation of the content validity. Each expert evaluated the content validity of each item using a 4-point Likert scale (1 point = irrelevant, 2 points = irrelevant and needs to be modified, 3 points = relevant but requires some modification, 4 points = very relevant). As a result, all 38 out of 42 items showed an item level content validity index (I-CVI) of .80 or higher [33], four items with I-CVI below .80 were deleted (‘clinical performance facilitation’ 3 items, ‘clinical site atmosphere’ 1 item). The scale-level content validity index of the scale was .91 [34], which secured the content validity of the initial items. From the expert review, four items with ambiguous or overlapping meanings were deleted (1 item from each factor: ‘interpersonal relationship’, ‘provision of learning information’, ‘organizational management system’, ‘physical component’), and two items containing multiple meanings in each item were split into four items (‘interpersonal relationship’ 1 item, ‘clinical site atmosphere’ 1 item). This process resulted in 36 preliminary items.

(5) Consider inclusion of validation items

A pilot test was conducted to evaluate the clarity, overall understanding, appropriateness of the items length, and the time required to respond to the questionnaire before running the main survey [31]. To recruit participants for the pilot test, requests for approval for participant recruitment were sent via e-mail to the deans of the three nursing colleges. Based on the standard that the sample size is suitable for 15~30 participants [35], the pilot test was conducted on 25 nursing students who were enrolled in the third or fourth grade, had experience in clinical practicum in at least 1 hospital department, and voluntarily agreed to participate in the study. Those who only experienced lab or online clinical practicum were excluded. The data were collected through an online Google Forms in March, 2022. The comprehensiveness of each items was measured on a 5-point Likert scale (1 point = very difficult, 2 points = somewhat difficult, 3 points = moderate, 4 points = somewhat easy, 5 points = very easy). Of 25 participants, 23 were female (92.0%) and 19 (76.0%) were senior students. The mean age of the participants was 23.2 ± 1.96 years. The average number of clinical departments where the nursing students conducted clinical practicum was 5.56 ± 3.43. The online survey took an average of 7 minutes to complete. The overall average scores were 4.50 ± 0.51 for clarity of the item, 4.61 ± 0.50 for understandability of the item, and 4.72 ± 0.57 for adequacy of the item length. Therefore, there were no corrections or deletions for 36 items after the pilot test.

2) Evaluation of scale

(1) Administer items to a development sample

① Selection of subjects

The participants of the large sample survey were nursing students. Inclusion criteria were individuals who were enrolled in the third or fourth grade of the undergraduate program in nursing, had experience in clinical practicum in at least 1 clinical department, expressed their intention to share their experiences and perceptions, understood the purpose of the study, and voluntarily agreed to participate in the study. Because the HCPES-NS is a hybrid scale that includes both onsite and online clinical practicum components, nursing students who only experienced lab practicum or online practicum without clinical site visit were excluded for study participants. To meet the criteria that the number of samples should be at least 200 to conduct factor analysis [36, 37], 400 or more participants were required to perform both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). A total of 460 nursing students were recruited from eight cities in the consideration of a dropout rate of 13% in each factor analysis. Of the data collected, data from 449 subjects were used for the final analysis after excluding eleven cases with incompleteness in responses.

② Data collection and measurements

To recruit participants for the main survey, requests for approval for participant recruitment were sent via e-mail to the deans of the 12 nursing colleges using a convenience sampling approach. The data were collected through online Google Forms from April to June, 2022. To verify the concurrent validity of the HCPES-NS, the CLES, a scale developed by Dunn & Burnett [26] and translated by Han [27], was used. The CLES was used after obtaining approval for use from the original developer [26] and the author who translated and adapted it into Korean [27]. The Cronbach’s α of the CLES was .89 in this study. The survey consisted of general characteristics (5 items), information about clinical practicum experience (3 items), preliminary items of the HCPES-NS (36 items), and the CLES items (19 items) for concurrent validity testing [27].

(2) Evaluate the items

Item analysis, EFA, and CFA were conducted to verify the construct validity of the scale.

It is desirable to conduct EFA and CFA on different groups of samples [37]. In this study, IBM SPSS/WIN 22.0 (IBM Co.) was used for case random sampling for classifying the subject pool into EFA (n = 218) and CFA (n = 231) groups. The concurrent validity of the HCPES-NS was examined with the Pearson correlation coefficients between the HCPES-NS and the CLES [27]. The internal consistency reliability of the HCPES-NS was examined with Cronbach’s alpha.

(3) Optimize scale length

A comprehensive evaluation of the results of previous steps was conducted to optimize the scale. The items arranged by the factors were randomly distributed in the item list to prevent fixed responses. Through this process, the final version of the HCPES-NS was confirmed.

3. Data analysis

The data were analyzed using IBM SPSS/WIN 22.0 and IBM AMOS 26.0 programs (IBM Co.). The general characteristics of the subjects were examined using descriptive statistics. The constructive validity of the HCPES-NS was verified by item analysis, EFA, and CFA. EFA was first performed by Kaiser–Mayer–Olkin (KMO) and Barlett’s spherical test to determine data suitability. Principal component analysis (PCA) with varimax rotation were used to extract factors based on an Eigenvalue of 1.0 [38]. CFA was performed using a structural equation model using the maximum likelihood estimator. The goodness-of-fit was evaluated using χ², χ²/df, standardized root mean square residual (SRMR), root mean square error of approximation (RMSEA), and goodness of fit index (GFI), Tucker–Lewis index (TLI), and comparative fit index (CFI). The reference values for pre-determined cutoffs for each fitness index were p > .05 for χ², χ²/df ≤ 3.0, SRMR ≤ .08, RMSEA ≤ .10, GFI ≥ .90, TLI ≥ .90, and CFI ≥ .90 [39].

Convergent validity was examined with standardized factor loading > .50, critical ratio (C.R.) > ± 1.97 and strict criteria of average variance extracted (AVE) > .50, and construct reliability (CR) > .70 [40, 41]. Verification of discriminant validity was performed by calculating the AVE and the square value of the correlation coefficient of each factor [41]. The concurrent validity of the HCPES-NS was examined with the correlation of the scale with the CLES. The internal consistency reliability of the HCPES-NS was tested using Cronbach’s α.

4. Ethical considerations

This study was approved by the Institutional Review Board of The Catholic University of Korea before collection of the data (IRB No. MC21EASI0136). Because students are a vulnerable population, voluntary participation and the provision of ample information about study participation process were ensured before the survey. Participants were informed about the purpose and process of the study and that their participation in the survey would be kept confidential. They were also explained that the participation is voluntary and can be withdrawn at any time without any disadvantages. Each participant was compensated a gift for their participation in the study.

RESULTS

1. General characteristics of participants

Of 449 participants, 400 were women (89.1%) and 443 (98.7%) were senior students. The mean age of the participants was 24.2 ± 3.08 years, with a range from 21 to 47 years. About 61.0% (n = 273; 60.8%) were enrolled in university, and 39.2% (n = 176) were enrolled in college (4-year). In terms of clinical practicum institution, the majority of the participants (n = 432; 96.2%) reported general hospitals with 300 beds or more. The average number of clinical departments where the nursing students practiced was 7.43 ± 3.20 (Table 1).

Table 1
General Characteristics of Participants (N = 449)

Download Table

2. Construct validity of the Hybrid Clinical Practicum Environment Scale for Nursing Students

1) Item analysis

The normality of the items was evaluated with mean, standard deviation (SD), skewness, and kurtosis. The mean value of each item ranged from 2.82 to 4.74, with the SD ranging from 0.48 to 1.45. The skewness did not exceed the absolute value of 3 and the kurtosis did not exceed the absolute value of 8, confirming the normality of all items [31]. The Cronbach’s α for all 36 HCPES-NS items was .89. The appropriateness of each item was evaluated using the corrected item-total correlation coefficient. Of the three items (No. 3, 9, and 35) that showed the correlation coefficient lower than .30 and were supposed to be deleted from the item pool due to their low contributions [42], item No. 3 was determined to be retained as it was conceptually essential to explain clinical practicum environment for nursing students. The alpha values with item deletion were analyzed before factor analysis to identify and delete items that lowers the reliability of the scale. As a result, 34 items were used for EFA (Table 2).

Table 2
Result of Item Analysis (N = 449)

Download Table

2) Exploratory factor analysis

The data suitability for factor analysis was confirmed by the KMO and Bartlett’s spherical tests. In this study, PCA with varimax rotation were used to extract factors based on an Eigenvalue of 1.0. Items were selected based on factor loadings ≥ .50, commonality ≥ .50, and accumulative variance > 60.0%. Items with factor loadings < .50 that did not load on any factor were deleted [38]. The primary EFA with 34 items resulted in a total of 9 factors. Seven items with a factor load value of less than .50 were deleted, resulting in 27 items. As the findings of secondary EFA on 27 items, 8 factors were extracted. After deleting six items with a factor load value of less than .50, the tertiary EFA was conducted on 21 items, resulting in 6 factors.

The first factor (5 items) was named ‘clinical site atmosphere’ and consisted of the educational atmosphere of the clinical practicum departments and the guidance of the preceptors for nursing students. The second factor (4 items) was called ‘interpersonal relationship’ and reflected nursing students’ difficulty with communications with preceptors and the patient. The third factor (3 items) was named ‘alternative online practicum contents’ and was composed of virtual contents that can replace onsite clinical practicum and the levels of support from faculty perceived by nursing students in learning online clinical practicum. The fourth factor (3 items) was called ‘provision of learning information’. This factor consisted of items regarding the provision of information to nursing students on systems and protocols to prepare for emergencies, safety and accident prevention, and orientation before clinical practicum. The fifth factor (2 items) was named ‘clinical performance facilitation’ and was composed of the individual items that nursing students regard as essential for improving clinical performance. The sixth factor (4 items) was called ‘physical component’. It consisted of physical aspects of clinical practicum institutions (amenities, distance, time, etc.) that are related to the clinical practicum of nursing students. The KMO value (.80) and Bartlett’s spherical test (χ² = 1,718.37, p < .001) were statistically significant, conforming the data suitability for factor analysis [38]. The cumulative explanatory power of the factors was 64.4%. The communality ranged from .54 to .84, and the factor loadings ranged from .51 to .89 (Table 3).

Table 3
Results of Exploratory Factor Analysis (N = 218)

Download Table

3) Confirmatory factor analysis

CFA was conducted on 231 participants to determine the fit, convergent validity, and discriminant validity of the model derived by EFA. In this study, items with standardized factor loading < .50 were sequentially deleted because they may impede the model goodness-of-fit [43]. As a result of the primary CFA, 2 items with standardized factor loading of .50 or less were eliminated. As a result of the secondary CFA, 4 items with standardized factor loading value of .50 or less were sequentially removed for the final analysis. The model goodness-of-fit test showed that goodness-of-fit indices of the model met the standard values (χ²/df = 1.91, SRMR = .05, RMSEA = .06, GFI = .92, TLI = .92, CFI = .94), suggesting that the model properly reflected the clinical practicum environment perceived by nursing students (Table 4). The value of χ² did not meet the reference value (χ² = 153.10, p < .001). However, since χ² value tends to be easily rejected due to its strict standards and sensitivity to sample size, it is necessary to evaluate the model fit with other goodness-of-fit indices along with χ² value [39]. During the process of CFA, six items were deleted due to the low standardized factor loading. As a results of the CFA, it was found that the final model of the HCPES-NS consisted of 5 factors and 15 items (Appendix 1).

Table 4
Results of Confirmatory Factor Analysis (N = 231)

Download Table

In terms of convergent validity of HCPES-NS factors, the factors met or was generally close to the recommended values in standardized factor loading (.58 to .99), C.R. (6.45 to 10.53), AVE (.43 to .77), and CR (.69 to .87), except for AVE in two factors and CR in one factor. Therefore, the convergent validity of the HCPES-NS was supported (Table 4) [41]. Discriminant validity among factors was confirmed by that the AVE values (.58) were greater than the squared correlation coefficient value (.34) (Table 4) [41]. The discriminant validity among the factors was secured, as the correlation coefficients of all factors ± of the standard error multiplied by two were from −.02 to .71, which did not include the absolute value 1 [40].

3. Concurrent validity of the Hybrid Clinical Practicum Environment Scale for Nursing Students

The concurrent validity of the HCPES-NS was examined with the Pearson correlation coefficients between the HCPES-NS and the CLES [24]. There was a high correlation between the two scales (r = .77, p < .001), supporting the concurrent validity of the HCPES-NS. There were also significant positive correlations among the subdomains of the HCPES-NS and the CLES. The CLES was significantly correlated with the domains of clinical site atmosphere (r = .73, p < .001), interpersonal relationship (r = .44, p < .001), alternative online practicum contents (r = .30, p < .001), provision of learning information (r = .51, p < .001), and clinical performance facilitation (r = .50, p < .001).

4. Reliability of the Hybrid Clinical Practicum Environment Scale for Nursing Students

The internal consistency reliability of the HCPES-NS was examined with Cronbach’s α. In this study, the Cronbach’s α of the HCPES-NS was .84. Cronbach’s α reliabilities were .84 for factor 1, .70 for factor 2, .86 for factor 3, .70 for factor 4, and .80 for factor 5.

DISCUSSION

This study aimed to develop the HCPES-NS and examined the validity and reliability of the scale. The HCPES-NS was developed according to the guidelines by DeVellis [31]. A total of 42 initial items were derived from a review of the relevant literature and in-depth interviews with ten individuals. Through content validity test and pilot test, the initial item pool was reduced into 36 items. Psychometric testing of the scale was performed on 449 nursing students. Through item analysis, two items were deleted, resulting in 34 items. The EFA on the 34 items resulted in 21 items with 6 factors. The final CFA demonstrated that the scale comprises 15 items with 5 factors, including clinical site atmosphere, interpersonal relationship, alternative online practicum contents, provision of learning information, and clinical performance facilitation. The scale was developed in the COVID-19 pandemic context and differs from existing tools in that it is a hybrid scale that includes both onsite and online clinical practicum environment components.

The first factor, ‘clinical site atmosphere’ consisted of items on the educational atmosphere for guiding nursing students in clinical practicum sites and the role of clinical preceptors. Nursing students may perceive the clinical practicum environment positively when receiving clinical guidance by the preceptors at the clinical practicum site. These results are consistent with findings of previous research that nurse managers’ interests in and support for nursing students’ clinical practicum [5, 26, 27, 44] and the educational and positive atmosphere of the clinical practicum department are an important environmental factor for the efficient clinical practicum of nursing students [9, 28, 29].

The second factor, ‘interpersonal relationship’ consisted of attributes about social components experienced by nursing students during the clinical practicum such as the difficulty in communicating with the clinical preceptors and experiences of being ignored or feeling worthless. In addition to the items about students’ emotions and preceptors’ attitude toward students which were already included in existing scales [6, 26, 27, 28], a new item about ‘feeling worthless at clinical practicum departments’ was added in the HCPES-NS in this study. The literature review showed that communications with the patient is important for nursing students in conducting clinical practicum. However, in this study, the items on communications with the patient were removed as a result of CFA, and nursing students were found to be more influenced by clinical preceptors than patients during clinical practicum. Specifically, the rapport and emotional aspect of the relationship with the preceptors affected nursing students’ perception of the clinical practicum environments. This results supports the view that nursing students expect to be welcomed by clinical practicum departments and feel a sense of belonging during clinical practicum [15, 16, 17]. In addition, such social experiences from the clinical practicum department can not only affect nursing students’ views on the nursing profession [17] but also influence their decision to remain in the healthcare field [5]. Therefore, the formation of social relationships with clinical preceptors is an important factor of the clinical practicum environment for nursing students.

The third factor, ‘alternative online practicum contents’ reflects a variety of learning contents provided during online clinical practicum. As the COVID-19 pandemic has restricted on-site clinical practicum of nursing students at healthcare institutions, online clinical practicum has been emerged as an alternative methods and attributes about the online clinical practicum is a new environmental factor. The online practicum involves providing specific clinical situations to nursing students within an online clinical practicum environment using a virtual context. However, unlike on-site clinical practicum, it may limit nursing students to conduct direct observations and communication with patients and healthcare providers. During the individual in-depth interviews in this study, nursing students reported anxiety that their indirect experiences in online practicum might lead to their limited performance in patient care in actual clinical settings. Therefore, it is important to provide nursing students with various clinical scenarios and ample feedback along with continuous interests in their learning needs in replacing on-site practicum with online clinical practicum. In this study, only two items related to online clinical practicum were included in the final scale. Considering that each university may have different online education system, additional research is needed to identify various aspects of online clinical practicum in the future.

The fourth factor, ‘provision of learning information’ involved educations on emergency systems, clinical protocols, safety guideline, and course orientations provided by universities to nursing students. Differentiated from the previously developed scales that include items related to the nursing care performance and the general atmosphere of the clinical practicum department as main organizational factors [26, 27, 28, 30], the HCPES-NS included items about how students are provided with the information necessary for clinical practicum as a newly derived organizational factor, showing the learning needs of nursing students. Since clinical practicum is a leaning activity in a new and unfamiliar environment compared with an on-campus course on theory, providing information about the overview of learning contents before clinical practicum and how to respond to emergencies and accidents are essential for them to successfully proceed to clinical practicum. The fourth factor is consistent with the existing view that providing students with sufficient information before clinical practicum is a critical factor of the clinical practicum environment [13].

The fifth factor, ‘clinical performance facilitation’ addresses nursing students’ individual needs for clinical performance. Whereas previous scales focus on achievement of learning goals as a main personal factor [26, 27, 30], the HCPES-NS includes items about whether nursing students’ clinical competence has improved during clinical practicum as individual performance factors. During clinical practicum, nursing students primarily explore whether the clinical department is a place where they can improve clinical competence. When their clinical performance is enhanced through the clinical practicum, they perceive it as a good clinical practicum environment. Because the aim of clinical practicum is to apply theoretical knowledge to clinical cases [1], providing ample guidance for knowledge application and enriching experiences to nursing students are important to improve their clinical competence through clinical practicum.

In the process of factor analysis, several items related to physical component factor such as distance to clinical practicum departments, convenience facilities, and experiencing clinical practicum at diverse institutions were deleted and not included in the HCPES-NS. It is presumed that a nursing student’s perception on clinical practicum environment may be less influenced by physical components of the clinical departments or their needs for physical factors could be hindered during the COVID-19 pandemic period when alternative online practicum was utilized by nursing colleges. Consistent assessment of nursing students’ needs for physical components of a clinical practicum site should be conducted in future research.

The HCPES-NS reflects the characteristics of the concept for clinical practicum environment. The concurrent validity of the HCPES-NS was supported by its high level of correlation with the CLES (r = .77). Of the subdomains of the HCPES-NS, the domain of ‘alternative online practicum contents’ showed the lowest correlation with the CLES (r = .30). This may be because the domain has emerged as a new sub-factor for the clinical practicum environment for nursing students throughout this study and could not be well related to conventional measurements. In this study, the Cronbach’s α of the HCPES-NS was .84, demonstrating an acceptable level of internal consistency reliability as a newly developed scale [37]. The HCPES-NS can be used as a tool to assess nursing students’ subjective perception on the hybrid leaning environments in which onsite and online clinical practicum are combined. Further research is needed to evaluate the usefulness of the scale for estimation of students’ learning outcomes, such as satisfaction with clinical practicum curriculum, academic achievement, or adaptation to clinical setting after graduation.

This study has several limitations. As the study participants were recruited using a convenience sampling method, the generalization of the study results is limited. While both junior and senior nursing students were invited as study participants following the inclusion criteria, only six junior students participated in the survey. It will be necessary to include more nursing students from various regions and grades as study participants in future studies. In this study, only internal consistency reliability using Cronbach’s α value was reported as the reliability of the HCPES-NS. Other types of reliability such as test-rest reliability and interrater reliability should also be examined in future studies. In future research, the databased should be expanded to review a broader range of literature using those not used in this study. The validity and reliability of the HCPES-NS should be examined in future replication studies in the post-COVID era when clinical practicum environment is as stable as before the outbreak of COVID-19. Finally the HCPES-NS can be used for evaluating a nursing student’s subjective perception of clinical practicum environments of overall departments. Therefore, objective measurements for the quality of practicum environments should be further developed based on the researchers’ needs.

Nevertheless, the HCPES-NS adds to the literature by providing an up-to-date assessment of clinical practicum environment for nursing students and incorporating online learning contents into the scale. Existing scales have mainly focused on ward atmosphere, unit manager leadership style, practicum guidance, or safety [6, 26, 27]. The HCPES-NS differs from existing measurements in that it reflects specific needs of nursing students for clinical practicum, including clinical site atmosphere, interpersonal relationship, alternative online practicum contents, provision of learning information, and clinical performance facilitation. In the HCPES-NS, attributes of the clinical practicum environment derived from the in-depth interviews both from nursing students and clinical faculty are reflected multi-dimensionally. While the HCPES-NS was designed in the context of Korean nursing education, it contains items that can be easily understood by nursing students in other regions and can promote research in the area of educational interventions for a quality clinical practicum for nursing students.

CONCLUSION

The HCPES-NS developed in this study is a multidimensional scale composed of 15 items and 5 subdomains. Each item of the HCPES-NS is measured on a 5-point Likert scale, and the total score ranges from 15 to 75. A higher score means a more positive perception of the clinical practicum environment. The HCPES-NS is a short scale that is relatively easy to apply to nursing students. The HCPES-NS can be used to evaluate the clinical practicum environment and provide suggestions for faculty if further improvement on the clinical practicum environment is needed. The HCPES-NS can also be used as an instrument in future research to examine the effects of nursing education interventions to enhance the quality of clinical practicum environment. The HCPES-NS was developed primarily in Korean, and it is suggested to examine the psychometric properties of the scale in other languages in further research. As this study focused on nursing students, mostly senior students, enrolled in nursing colleges in Korea, future studies should include nursing students from various regions as study participants.

Notes

CONFLICTS OF INTEREST:The authors declared no conflict of interest.

AUTHOR CONTRIBUTIONS:

Conceptualization or/and Methodology: Yoon S & Yeom H.
Data curation or/and Analysis: Yoon S.
Funding acquisition: None.
Investigation: Yoon S.
Project administration or/and Supervision: Yeom H.
Resources or/and Software: Yoon S.
Validation: Yoon S & Yeom H.
Visualization: Yoon S & Yeom H.
Writing original draft or/and Review & Editing: Yoon S & Yeom H.

ACKNOWLEDGEMENTS

None.

DATA SHARING STATEMENT

Please contact the corresponding author for data availability.

Appendix 1

간호대학생의 하이브리드 임상실습환경 인식에 대한 설문조사

References

1. Schlairet MC. Educating nurses: A call for radical transformation, by Patricia Benner, Molly Sutphen, Victoria Leonard, and Lisa Day. Stanford, CA: Jossey-Bass, 2010. Cambridge Quarterly of Healthcare Ethics 2011;20(4):617–619. [doi: 10.1017/S0963180111000375]
  Article
1. Korean Accreditation Board of Nursing Education (KABONE). 2023 Accreditation standards of bachelor degree in nursing program [Internet]. Korean Accreditation Board of Nursing Education; 2023 [cited 2022 Dec 28].
  Available from: http://www.kabone.or.kr/notice/list/view.do?num=823.
1. Orton HD. Ward learning climate and student nurse response. Nursing Times 1981;23 suppl 17:65–68.
1. Pitkänen S, Kääriäinen M, Oikarainen A, Tuomikoski AM, Elo S, Ruotsalainen H, et al. Healthcare students’ evaluation of the clinical learning environment and supervision - a cross-sectional study. Nurse Education Today 2018;62:143–149. [doi: 10.1016/j.nedt.2018.01.005]
  Article
1. Flott EA, Linden L. The clinical learning environment in nursing education: A concept analysis. Journal of Advanced Nursing 2016;72(3):501–513. [doi: 10.1111/jan.12861]
  PubMed
  
  Article
1. Chan D. Development of the Clinical Learning Environment Inventory: Using the theoretical framework of learning environment studies to assess nursing students’ perceptions of the hospital as a learning environment. Journal of Nursing Education 2002;41(2):69–75. [doi: 10.3928/0148-4834-20020201-06]
  PubMed
  
  Article
1. Saarikoski M, Leino-Kilpi H. The clinical learning environment and supervision by staff nurses: Developing the instrument. International Journal of Nursing Studies 2002;39(3):259–267. [doi: 10.1016/s0020-7489(01)00031-1]
  PubMed
  
  Article
1. Cremonini V, Ferri P, Artioli G, Sarli L, Piccioni E, Rubbi I. Nursing students’ experiences of and satisfaction with the clinical learning environment: The role of educational models in the simulation laboratory and in clinical practice. Acta Bio-Medica 2015;86 Suppl 3:194–204.
1. D’Souza MS, Karkada SN, Parahoo K, Venkatesaperumal R. Perception of and satisfaction with the clinical learning environment among nursing students. Nurse Education Today 2015;35(6):833–840. [doi: 10.1016/j.nedt.2015.02.005]
  Article
1. Papastavrou E, Dimitriadou M, Tsangari H, Andreou C. Nursing students’ satisfaction of the clinical learning environment: A research study. BMC Nursing 2016;15:44 [doi: 10.1186/s12912-016-0164-4]
  PubMed
  
  Article
1. Saukkoriipi M, Tuomikoski AM, Sivonen P, Kärsämänoja T, Laitinen A, Tähtinen T, et al. Clustering clinical learning environment and mentoring perceptions of nursing and midwifery students: A cross-sectional study. Journal of Advanced Nursing 2020;76(9):2336–2347. [doi: 10.1111/jan.14452]
  PubMed
  
  Article
1. Antohe I, Riklikiene O, Tichelaar E, Saarikoski M. Clinical education and training of student nurses in four moderately new European Union countries: Assessment of students’ satisfaction with the learning environment. Nurse Education in Practice 2016;17:139–144. [doi: 10.1016/j.nepr.2015.12.005]
  PubMed
  
  Article
1. Lovecchio CP, DiMattio MJ, Hudacek S. Predictors of undergraduate nursing student satisfaction with clinical learning environment: A secondary analysis. Nursing Education Perspectives 2015;36(4):252–254. [doi: 10.5480/13-1266]
  PubMed
  
  Article
1. Kwak EM, Yun HJ, Park MA. Effects by incivility experienced and clinical learning environment of nursing students on clinical practice stress. Journal of the Korea Academia-Industrial Cooperation Society 2021;22(2):160–168. [doi: 10.5762/KAIS.2021.22.2.160]
1. Skaalvik MW, Normann HK, Henriksen N. Clinical learning environment and supervision: Experiences of Norwegian nursing students - a questionnaire survey. Journal of Clinical Nursing 2011;20(15-16):2294–2304. [doi: 10.1111/j.1365-2702.2011.03727.x]
  PubMed
  
  Article
1. Forber J, DiGiacomo M, Carter B, Davidson P, Phillips J, Jackson D. In pursuit of an optimal model of undergraduate nurse clinical education: An integrative review. Nurse Education in Practice 2016;21:83–92. [doi: 10.1016/j.nepr.2016.09.007]
  PubMed
  
  Article
1. Kim CH, Kim JY. Influence of nursing students’ clinical practice learning environment, self-leadership, and clinical practice belonging on nursing professionalism. Journal of Korean Academic Society of Nursing Education 2019;25(1):5–16. [doi: 10.5977/jkasne.2019.25.1.5]
  Article
1. Lamont S, Brunero S, Woods KP. Satisfaction with clinical placement--the perspective of nursing students from multiple universities. Collegian 2015;22(1):125–133. [doi: 10.1016/j.colegn.2013.12.005]
  PubMed
  
  Article
1. Babenko-Mould Y, Laschinger HKS. Effects of incivility in clinical practice settings on nursing student burnout. International Journal of Nursing Education Scholarship 2014;11(1):145–154. [doi: 10.1515/ijnes-2014-0023]
  Article
1. Welding NM. Creating a nursing residency: Decrease turnover and increase clinical competence. Medsurg Nursing 2011;20(1):37–40.
  PubMed
1. Oh YK, Kim EY. The effects of clinical learning environment on nursing students’ powerlessness and self-efficacy related to clinical practice. Journal of East-West Nursing Research 2018;24(1):36–43. [doi: 10.14370/jewnr.2018.24.1.36]
1. Quigley D, Loftus L, McGuire A, O’Grady K. An optimal environment for placement learning: Listening to the voices of speech and language therapy students. International Journal of Language & Communication Disorders 2020;55(4):506–519. [doi: 10.1111/1460-6984.12533]
  Article
1. Higher Education in Korea. Academyinfo data [Internet]. Higher Education in Korea; 2021 [cited 2022 Aug 7].
  Available from: https://www.academyinfo.go.kr/popup/main0810/list.do.
1. Kim WG, Park JM, Song CE. Development and application of an online clinical practicum program on emergency nursing care for nursing students. Journal of the Korea Entertainment Industry Association 2021;15(1):131–142. [doi: 10.21184/jkeia.2021.1.15.1.131]
  Article
1. You SY, Cho MY. Nursing students’ experiences of online adult nursing practicum in COVID-19. Journal of Learner-Centered Curriculum and Instruction 2021;21(10):385–398. [doi: 10.22251/jlcci.2021.21.10.385]
1. Dunn SV, Burnett P. The development of a clinical learning environment scale. Journal of Advanced Nursing 1995;22(6):1166–1173. [doi: 10.1111/j.1365-2648.1995.tb03119.x]
  PubMed
  
  Article
1. Han JY. Nursing students’ perceptions of clinical learning environment (CLE). Journal of the Korean Data Analysis Society 2010;12(5):2595–2607.
1. Saarikoski M, Isoaho H, Warne T, Leino-Kilpi H. The nurse teacher in clinical practice: Developing the new sub-dimension to the Clinical Learning Environment and Supervision (CLES) Scale. International Journal of Nursing Studies 2008;45(8):1233–1237. [doi: 10.1016/j.ijnurstu.2007.07.009]
  PubMed
  
  Article
1. Kim SH, Yoo SY, Kim YY. Validity and reliability of the Korean version scale of the Clinical Learning Environment, Supervision and Nurse Teacher Evaluation Scale (CLES+T). Journal of Korean Academy of Nursing 2018;48(1):70–84. [doi: 10.4040/jkan.2018.48.1.70]
  PubMed
  
  Article
1. Cooper S, Cant R, Waters D, Luders E, Henderson A, Willetts G, et al. Measuring the quality of nursing clinical placements and the development of the Placement Evaluation Tool (PET) in a mixed methods co-design project. BMC Nursing 2020;19:101 [doi: 10.1186/s12912-020-00491-1]
  PubMed
  
  Article
1. DeVellis RF. In: Scale development: Theory and applications. 4th ed. Sage Publications; 2017. pp. 105-204.
1. Gruppen LD, Irby DM, Durning SJ, Maggio LA. Conceptualizing learning environments in the health professions. Academic Medicine 2019;94(7):969–974. [doi: 10.1097/ACM.0000000000002702]
  PubMed
  
  Article
1. Lynn MR. Determination and quantification of content validity. Nursing Research 1986;35(6):382–386. [doi: 10.1097/00006199-198611000-00017]
  PubMed
  
  Article
1. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health 2007;30(4):459–467. [doi: 10.1002/nur.20199]
  Article
1. Lee EO, Im NY, Park HA, Lee IS, Kim JI, Bea JY, et al. In: Nursing research and statistical analysis. Soomoonsa; 2009. pp. 261-267.
1. Yu JP. In: Prof. Jongpil Yu’s structural equation model concept and understanding: Amos 4.0~20.0. Hannarae Publishing; 2012. pp. 149-278.
1. Hinkin TR. A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods 1998;1(1):104–121. [doi: 10.1177/109442819800100106]
  Article
1. Seo W, Lee S, Kim M, Kim J. Exploratory factor analysis in psychological research: Current status and suggestions for methodological improvements. Journal of Social Science 2018;29(1):177–193. [doi: 10.16881/jss.2018.01.29.1.177]
  Article
1. Bae BR. In: Structural equation modeling with Amos 27. Chungram Publishing; 2021. pp. 251-280.
1. Noh KS. In: The proper methods of statistical analysis for dissertation: SPSS & AMOS. Rev. ed. Hanbit Academy; 2019. pp. 138-192.
1. Song JJ. In: SPSS/AMOS statistical analysis method. Rev. ed. 21Century; 2015. pp. 146-482.
1. Kline RB. In: Principles and practice of structural equation modeling. 4th ed. Guilford Press; 2015. pp. 7-113.
1. Choi CH, You YY. The study on the comparative analysis of EFA and CFA. Journal of Digital Convergence 2017;15(10):103–111. [doi: 10.14400/JDC.2017.15.10.103]
1. Phillips KF, Mathew L, Aktan N, Catano B. Clinical education and student satisfaction: An integrative literature review. International Journal of Nursing Sciences 2017;4(2):205–213. [doi: 10.1016/j.ijnss.2017.03.004]
  PubMed
  
  Article

Citations

Citations to this article as recorded by

Environmental and Individual Factors Associated with Clinical Practice Stress in Korean Nursing Students: A Scoping Review
Ui Rim Song
Korean Journal of Adult Nursing.2025; 37(4): 355. CrossRef

Cite

CITE: export Copy Download Format; Close

Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:

RIS — For EndNote, ProCite, RefWorks, and most other reference management software
BibTeX — For JabRef, BibDesk, and other BibTeX-specific software

Include:

Citation for the content below
Citation and abstract for the content below

Development of the Hybrid Clinical Practicum Environment Scale for Nursing Students

J Korean Acad Nurs. 2024;54(3):340-357. Published online August 31, 2024

DOI: https://doi.org/10.4040/jkan.24016

Related articles

Development and Effectiveness of Progressive Simulation Education Program on Medication Safety for Nursing Students