Effectiveness of Online Pedagogy of welding Skills during the COVID-19 Period: A Systematic Review

Ogumbe Boniface Ekwok¹ and Arikpo Sampson Venatius²

¹ Department of Technical Education, Cross River University of Technology, 1123 Calabar, Nigeria

²Department of Technical and Engineering Education, Universiti Teknologi Malaysia (UTM), Johor, 81310, Malaysia

ABSTRACT: Online learning was created as a result of the huge industry disruption that occurred in March of 2020. Online education presented unique difficulties for learning in an entirely virtual setting with a concentration on psychomotor skill development, calling into doubt its efficacy. Finding a practical replacement for lab-based hands-on activities and group or team-based learning experiences was extremely difficult for technology education. This study compares the COVID-19 time to the pre-COVID period to assess how effective online teaching and learning experiences were in technology-based education. Just 20 randomized controlled trial publications, out of which 15 studies were used in the meta-analysis, were included in the review after an electronic search of the literature. Using Review Manager 5.3 to examine the data, Cochrane’s 2 test and I2 were used to determine heterogeneity. The meta-analysis shows a very reliable sensitivity analysis and a substantial pooled effect size of (SMD = 4.49 @ 95%, CI = 2.37 – 3.63 @ p.00001) from the test scores in favor of the experimental group. The results of the included studies’ sub-item achievement test results reveal a statistically positive difference in every category of welding skill tested. Studies demonstrating the effectiveness of online pedagogy in the covid era are the source of the current statistics. These results point to the need for additional investigation into the development of psychomotor skills in other technology education courses.

1. Introduction

Over 80% of the world’s students (or roughly 1.723 billion learners as of April 21, 2020) were not in school as a result of the pandemic’s impact on educational systems around the world during the COVID-19 era, which resulted in the widespread shutdown of schools, universities, and institutions. Five (5) local closures and 191 countrywide closures, affecting about 99.4% of the global student population, have been enacted [1]. School closures have affected several stakeholders, including students, teachers, and families, in addition to having an economic and societal impact [2-4]. To solve this issue, UNESCO recommended using open educational resources and platforms, such as distance learning programs, to engage students remotely and minimize disruptions to the educational process [5, 6]. Higher education institutions (HEIs) across the world started operating remotely via online platforms for Emergency Remote Teaching and Learning (ERT&L) [7, 8]. However, many academics have questioned whether HEIs are ready to transition to the digital era of learning because of this [9].

During the outbreak, there was also a disruption in technology-based education [10]. Technical institutions were closed for the students, and educational and research activities stopped. This sudden halt due to the COVID-19 crisis has many implications on vocational education and training opening a new surge in the exploration of new ways for continuing education distantly.7 it wasn’t news that the COVID-19 crisis brought all laboratory-based research activities to a standstill. During this stay-at-home time, vocational educators have resorted to different creative ways of delivering practical skills remotely. Moodle, Blackboard, Zoom, IJILT, Google Classroom, Google Meet, Skype, Google Forms, Calendars, G-drive, Google Hangouts, Google Jam Boards, and Drawings have been the most widely used systems for online instruction during COVID-19. Other e-conferencing platforms, such as WebEx, Instagram, Twitter, Facebook Live, Google Meet, Skype, YouTube Live, WhatsApp, Google Drive, and Facebook Live were also utilized by some institutions [11]. Though, most of these remote learning platforms have been in use for a long time and were found to be useful during the crisis in Afghanistan, Africa and Far-eastern countries.13. Teachers and students were encouraged to make use of online study materials, either uploaded by the institutions or videos available on the internet for teaching and learning.14 Many institutions implemented small group problem-based learning (PBL) and case presentation and discussion to engage the students.15. Anderson and McCormick (2005) enumerated factors for which ERT&L should focus such as i. curriculum compliance; ii. inclusivity; iii. learner involvement; iv. innovative learning; v. effective learning; vi. formative evaluation; summative evaluation; vii. coherent, consistent, and transparent; viii. the device is easy to operate and use; and ix. cost-effectiveness. Additionally, [12] stresses the value of i. interaction between students and teachers, ii. student participation, iii. an active learning environment, iv. prompt feedback, v. attainment of learning objectives, and vi. respect for differences.

ERT&L allows synchronous or asynchronous teaching [13]. All of these different ERT&L models should be beneficial as long as the online learning principles are used correctly and effectively. Because when properly used, flipped classrooms or blended learning methods have been proven highly successful in remote teaching. A synchronous learning process allows for real-time communication between students and lecturers as well as between students themselves during class time. While, Ansikronus permits students to study at various times within a predetermined period, resulting in indirect engagement and communication. In a technology-based laboratory setting, psychomotor skills are typically taught using the Ansikronus model, allowing for a safe, controlled environment where students can master skills at their own pace. The benefits of simulation for learning psychomotor skills are numerous. Although the three learning domains of emotive, cognitive, and psychomotor are all present in psychomotor skills, they do not exist independently. Each learning domain has traditionally been taught separately, yet all are required when teaching technical content [14]. According to [15] experiential learning theory, believes that learning is a continuous process based on experience, students learn most successfully when the learning procedure incorporates pertinent psychomotor, cognitive, and emotional knowledge. The COVID-19 pandemic increased the difficulty to perform hands-on training effectively. Hence, virtual simulation (virtual reality (VR) and augmented reality (AR)) is the potential and timely solution for these problems. In the VR application, the users immerse in the virtual world and interact with the virtual objects while the AR application, on the other hand, displays the virtual information in front of the users’ vision when viewing the real-world environment allowing students to interact with virtual objects by using real objects. All learning domains must be included in technical skills in ERT&L. According to [16] allowing students to fully synthesize the related knowledge before applying it in a real-world setting. Reduced anxiety and enhanced self-confidence are attributes of motor skills that make it easier to transfer knowledge [17-19] . The objective s for students to more readily apply what they learn in simulation to actual settings because skills simulation matches real-life scenarios [20].

Effectiveness in the context of teaching often refers to the kinds of activities that result in or support learning [21, 22]. To put it another way, good teaching promotes student learning [23]. We refer to ERT&L effectiveness in this article as the capacity of e-learning to produce the desired output of equipping students with the knowledge and abilities that will increase their life opportunities and help them prepare for the workforce and self-employment. For better preparing the trainees for the certified examination, [24] propose adding new welding procedures to the virtual workshops.  Additionally, numerous earlier studies investigated the efficiency of VR or AR assistance for beginning welders [25]. Several research studies [26-28] of VR or AR welding training workshops reported that VR and AR techniques are recently attaining more attention in vocational welding training, especially during the COVID-19 pandemic. [24] evaluated the effectiveness and impact of the VR or AR welding training workshops on the learning experience and performance of the welders using professional organizations such as the American Welding Society (AWS) and The Welding Institute (TWI) authorized welding examinations. They discovered that the VR intervention group outperformed the conventional group, with a 41.6% increase in total certificates obtained. Additionally, [29] conducted a study in which they contrasted the effects of full VR and VR-integrated training on both cognitive and physical skills for welders. In the VR-integrated training group, real welding instruction made up 50% of the training time and 50% of the training was done in virtual reality. After the course, the actual welding inspection was needed by both groups. According to the findings, the VR-integrated training group outperformed the complete VR welding group in challenging welding positions. To attain the finest welding abilities, virtual and real-world practical training must complement and integrate [29, 30], use of the VR welding simulator allowed them to compare the welding quality scores between the expert and novice groups [31]. It is expected that even after this COVID-19 pandemic is over, some sort of e-learning and online evaluation strategies may take place in the vocational education system. Hence, to ensure that virtual training is effective in successfully transferring practical knowledge in vocational-based education, more reality must be added to it.

2. LITERATURE REVIEW

2.1          Effectiveness of ERT&L

Several studies [32, 33] show that online learning helps learners develop a variety of abilities and a high level of competence. According to research, e-learning allows students to learn at their own pace whenever and wherever they choose, promoting flexible learning and reducing expenses, labor costs, and time [34, 35]. Also, it has been found that students reinforce their understanding of the material by repeating it as many as they like by their learning pace and style [36]. By taking on their learning responsibilities, students perceive themselves as being more involved in the learning process [37]. Additionally, when technology is used effectively for online learning, educational activities are carried out outside of the traditional classroom setting. Several research on online learning has shown that the disparities in student learning rates are taken into account and that equal opportunities are provided for all students [38]. In some research, it has also been discovered that it permits the use of rich and easily available materials to provide long-lasting learning results [39]. All of these findings highlight the advantages and opportunities that online learning provides in terms of learning results. Course design, instructor actions, and student actions are the three primary categories included in a qualitative synthesis of the elements influencing online learning experiences within the community of inquiry framework [40]. As a result, qualities associated with course design that is effective in online learning experiences include learning materials, learning activities, collaboration and working in small groups, encouraging interaction, supporting a feeling of community, and clarity in course design and objectives. Additionally, while being present and giving feedback are described as instructor-related actions that are seen to have an impact, timely participation in class discussions and thoughtful responses to their peers’ questions are said to be crucial student actions in online learning environments [40]. During the Covid-19 period,  [41] carried out a quasi-experimental study to determine the impact of online learning on students’ academic performance. The posttest results for learners’ academic achievement revealed a significant impact of online learning, according to the authors. In other words, it may be claimed that online learning contributes to the achievement of the course’s learning objectives. 207 universities’ 1,255,022 students and 27,820 academic staff members participated in the national survey in Turkey. 90% of the students said they benefited from and thought the online teaching materials in their courses were adequate, and 52% of the respondents said that online education and learning during the pandemic time were on par with in-person education. [33] carried out a study that aims to determine the effectiveness of online learning during the Covid-19 pandemic using Primary data obtained from 115 respondents. The results of data analysis obtained from filling out student questionnaires concluded that the online learning system carried out during the COVID-19 pandemic is effective and inefficient. [42] evaluate the performance of online instruction utilizing Undiksha E-Learning during the COVID-19 pandemic in the Physical Education and Health Studies curriculum. The outcomes demonstrated that employing Undiksha’s E-Learning for online learning was quite successful. The authors discovered that student motivation for participating in online learning using Undiksha E-Learning was 77%, student understanding of the learning materials was 88%, and student learning outcomes were classified as 88%. Students’ concentration levels for paying attention to the lecturers’ explanations were 96%. [43] analyzes what students believe about the value of online learning. Few studies have examined learner satisfaction with online instruction, particularly during the transition from traditional learning methods to online learning, the authors noted, even though several studies have suggested that online education can be as effective as traditional education that requires attendance, students respond to online education differently, and their response is dependent on their skill with online tools, their technical capabilities to access online courses, and the way the instructors carry out learning activities.

2.2          Challenges to the effectiveness of ERT&L

Given the prevalence of contemporary communication technologies, the literature supports the use of ERT&L in higher education in terms of its utility, efficacy, and favorable impact on student performance. Even though closing schools were a necessary preventive measure to stop the spread of COVID-19, [44] reported that the sudden reliance on online learning required unprecedented efforts and innovative learning and teaching practices to support students during the crisis. The teaching approaches employed in online education should encourage learning in students [45]. Recent research [46] that takes into account various aspects of school teachers’ perspectives on e-learning in the UAE supports our study. Several more research like [47-49] were carried out to discover difficulties during this exceptional scenario. [50] used educators’ responses to the domains of socio-demographic data forms to assess the psychometric features of the Distance Education Attitudes Scale (DEAS). The findings indicated that there were no appreciable differences between these two Domains. The implementation of ERT&L during the COVID-19 crisis was found to be significantly influenced by instructors’ perceptions toward the efficacy of distance learning and by distance learning’s challenges. In their study on how Romanian institutions used only online teaching and learning to disseminate information during the Coronavirus epidemic, [51] took into account students’ perceptions of online learning and e-learning platforms. The authors uncovered that students are exposed to a variety of complementary technologies, as demonstrated by prior studies, online-based educational systems have several advantages for students. [52] carried out a survey to study Agricultural students’ perceptions and preferences toward online learning. The authors studied the student’s preferences for various attributes of online classes. This study focused on how students perceive online learning, and which format they prefer. In a related study [53] amid COVID-19 in Nepal, a survey was conducted to ascertain the advantages and difficulties of online education. The study found that the situation had several advantages, including encouraging online research, introducing participants to a global community, and giving individuals more independence. To understand the obstacles to online learning from the perspective of medical students in the Philippines, [54] conducted a study. The study is significant since it focuses on a developing nation and includes the viewpoint of students. Similar research was done by [55] to determine the strengths and shortcomings of Spanish universities during the epidemic. A theoretical model built on a time series was provided by Mittal et al. in 2021 to assess several aspects of the uptake of online learning at the time of the COVID-19 pandemic. Indian students’ perceptions of their readiness for the abrupt switch to online education were examined [56]. To address such a circumstance, the authors also published institutional governance rules. This study and ours are closely connected in that they both addressed the effects of the abrupt change; however, whereas our study will be applied to instructors, this one was applied to students. [57], the study found that senior management support and commitment significantly influenced university lecturers’ behavioral intentions to teach online. [58] The results of the study revealed that top management support through the direct involvement of top management and senior managers in mobilizing ICT resources and creating a cohesive and collaborative ICT use culture significantly influenced university lecturers’ behavioral intentions to teach online. Furthermore, [59, 60] found that institutional ICT policies, top management support, online teaching approaches, the availability of trustworthy, strong, and accessible ICT infrastructure, as well as opportunities for ICT training, had a significant impact on lecturers’ behavior intentions to teach online. [61] revealed that educational institutions that have ICT infrastructure that is easy for both lecturers and students to use support effective online teaching. On the other side, the same studies found that ICT infrastructure that is complex and challenging for professors and students to operate has a detrimental impact on lecturers’ behavioral intentions to teach online. [62] revealed that lecturers’ behavioral intentions to teach online were significantly influenced by the course design and approach they used. Interactive course materials and a dynamic, adaptable technology-mediated learning environment have a significant influence on lecturers’ behavioral intentions to teach online, according to a study by [63]. Using specialized technology like virtual reality (VR) and augmented reality (AR), authors in [64] presented solutions to the challenges of conducting research during a pandemic, although also expressed worries about the lack of firm ethical rules for this. Researchers in [65] explored the difficulties associated with operating a virtual reality (VR) lab while upholding tight regulations and sanitary standards for research and education. [66] identified certain significant difficulties, like the lack of internet connection and the restricted availability of resources during the COVID-19 outbreak, which are also faced by many students in Brazil [67]. The lack of policy standards for engineering course delivery online and of assessment methods for high-quality instruction was also mentioned in [67]

METHODOLOGY

Search process and inclusion criteria

Following the PRISMA guidelines [68], a comprehensive literature search was conducted for all articles published from 2019 through September 2022. Nine electronic databases were searched for ERT&L articles that discussed teaching and learning during the COVID-19 period (Research Gate, Academic Search Premier, Econ Lit, ProQuest, Scopus, Web of Science, Google Scholar, and ERIC). The search strategy concentrated on technical skill outcomes (automobile, building, electrical/electronics, metalwork, and woodwork), pedagogy (teaching and learning), programs (vocational education, technology education, and engineering education), and intervention (randomized controlled trials, controlled trials, evaluations). Platforms requiring the usage of hardware other than a typical computer monitor, keyboard, mouse, or touch-screen device as well as those with a hands-on technical training component were excluded. Studies that included e-learning as a component of a larger curriculum were only considered when the comparison group was given the same curriculum as the experimental group but without e-learning. Using Kirkpatrick’s four-tiered approach for evaluating training programs, studies reporting just self-reported opinions (level 1) without examining learning (level 2), transfer of acquired abilities into job behaviors (level 3), or student outcomes (level 4) were discarded [69]. Additionally, there were exclusions based on the publishing year and language, but there were none based on the field of research after appropriateness could not be determined after publications were originally disqualified by reviewing the title and abstract, the entire article was examined. As a supplementary search approach, the retrieved articles were cross-referenced for potential additions.

(See in PDF FILE)

Fig.1:  PRISMA technique of reporting items for systematic reviews or meta-analyses derived from [70].

The various databases’ literature searches turned up 174 possibly pertinent publications. After screening the publication’s titles and abstracts, 16 references were determined to be possibly relevant and their complete texts were collected. Four additional papers were discovered as prospective references and expert input. Publications found through the search were not included in our analysis because they had duplicate content, did not comply with age restrictions, did not relate to technology education, did not call for the application of motor skills, and were not related to Covid-19. A total of 18 studies provided information on the effect of ICT interventions on psychomotor skills and 2 studies ICT pedagogical skill proficiency were selected for inclusion in this review (See Tables 1 and 2). 10 studies representing 50% of those included were published in 2020; 4 papers representing 20% of those included were published 2021. 4 research, representing 20% of the studies were published in 2022, and 2 studies standing for 10% were published between 2017 and 2019.

Methodological quality assessment

Before articles were included in this study, two reviewers independently conducted a methodological quality examination to assess the chosen studies’ validity based on common sources of bias to see if they satisfy the requirements for inclusion as stated in the Cochrane handbook for a systematic review of treatments [71] using the Joanna Briggs Institute Meta-Analysis of Statistics Assessment and Review Instrument (JBI-MAStARI) [72]. Any discrepancies between the reviewers were settled by conversation otherwise, by there was a need to involve a third reviewer. The standardized data extraction tool from JBI-MAStARI was used by two reviewers to independently extract data from the papers included in the review. The two reviewers separately determined for each article whether it received a good rating (if the evaluated item was there) or a negative rating (if the evaluated item was absent). Specific information regarding the interventions, populations, study methodologies, and outcomes important to the review question and particular objectives were included in the data that is extracted.

The methodological quality of the included publications was evaluated using a 10-item quality evaluation scale for randomized control/pseudo-randomized trials and a 9-item scale for descriptive or case series studies (see Tables 1 & 2). Items with inadequate descriptions received a negative (absent) score. Reviewer agreement was predetermined at 80% for each article [73]. That is, for 8 of the 10 things and 7 of the 9 items in each article, reviewers had to concur that the items were present or absent. If there was less than 80% agreement, more debate led to a consensus. To assess the article’s overall quality, the scores were then added together. An article was assessed to have excellent methodological quality by [22, 73] if it received a score of 5 or higher for a controlled trial or a score of 6 or higher for a randomized controlled trial.

Data synthesis

Software called Revman 5.3 was used to examine the data. We utilized variance analysis to determine the standardized mean difference (SMD) and weight mean difference (WMD) for effect sizes based on sample size and 95% confidence intervals for each trial and the pooled trials (CIs). We used the fixed-effects model if heterogeneity was found after pooling the survey’s data; otherwise, the random-effects method was used. We adopted two meta-analysis techniques (fixed and random effect models). Utilizing Cochran’s (2) test p-value and the proportion of variation attributable to heterogeneity, we estimated heterogeneity (I²). According to [74-76], heterogeneity with p 0.10 and I² > 50% was considered significant, but I² -values of 25% to 50% were regarded as low, 50%-75% as moderate, and > 75% as high. We conducted a sensitivity analysis where heterogeneity was present to see if it had a substantial impact on the meta-findings. The analysis by removing each of these studies and then recalculating the pooled estimates for the other trials, we performed a sensitivity analysis. By removing each of these studies and then recalculating the pooled estimates for the other trials, we performed a sensitivity analysis. This is to make sure it did not materially affect the findings [77] and Using STATA 13.0, a funnel plot from the Begg’s rank correlation test and an Egger’s linear regression test are used to examine the publication bias [78].

Effect sizes and heterogeneity

The value calculated to determine the strength and direction of the association between two groups or variables is known as the “effect size”[79]. The effect values of the studies included in the analysis in this study were estimated using Hedges’ g value [80], a measure of the standardized mean difference. On the other hand, due to the limited circumstances in which the Fixed Effects Model (FEM) would be appropriate, the Random Effects Model (REM) was employed to calculate effect sizes in a meta-analysis [81]. Because factors related to various educational levels, courses, and intervention durations were considered in this study, the use of REM was preferred. The I² value, which provides more trustworthy results than the Q statistic, is calculated in the meta-analysis. It is known that there is a large level of heterogeneity at 75% and higher, but there is none.

Table 1: Methodological quality checklist for randomized control/pseudo-randomized trial

Paper

No.          Author & year                                      1              2              3              4              5              6              7              8              9              10                MQS       Ag

1. Opris, Ionescu, Costinas, &

Nistoran, 2020                                     Y             Y             Y             N             Y             N             Y             Y             Y             N                7              90%

2. Arif & Shafiullah, 2022 Y             N             Y             N             Y             Y             Y             Y             Y             N                7              80%
3. Bdair, 2021 N             N             Y             Y             N             Y             Y             Y             Y             Y                9              95%
4. Yasmin, 2022 Y             Y             Y             Y             Y             Y             N             Y             N             Y                8              80%
5. Khan & Abid, 2021 Y             N             Y             Y             Y             N             Y             N             N             N                5              80%
6. Suryaman & Mubarok, 2020 Y             Y             Y             Y             N             Y             Y             Y             Y             Y                6              90%
7. Cant & Cooper, 2017 N             Y             N             Y             Y             Y             Y             Y             Y             N                8              70%
8. Hassan, Mirza, & Hussain, 2020 N             Y             Y             N             Y             Y             N             Y             Y             N                9              75%
9. Mukhtar, Javed, Arooj, & Sethi,

2020                                                       Y             Y             Y             N             Y             Y             Y             N             Y             Y                6              80%

10. Sarpong, Dwomoh, Boakye, &

Ofosua-Adjei, 2022                            Y             Y             N             Y             Y             Y             Y             Y             N             Y                7              80%

11. Chan, Bista, & Allen, 2022 N             Y             Y             N             Y             N             N             Y             Y             Y                6              90%
12. Hameed, Husain, Jain, Singh, &

Sabina, 2020                                        Y             Y             Y             Y             Y             Y             Y             N             N             Y                8              80%

13. Djidu et al., 2021 Y             Y             N             Y             Y             Y             N             N             Y             N                8              85%
14. Angel-Urdinola, Castillo-Castro

& Hoyos, 2021                                    Y             N             Y             Y             Y             Y             N             Y             Y             N                9              100%

15. Agrawal & Pillai, 2020 N             Y             Y             N             Y             N             Y             N             N             Y                4              90%
16. Cen, Ruta, Al Qassem, & Ng, 2019 Y             Y             N             Y             N             Y             Y             Y             Y             N                8              95%
17. Chung, Tung, & Lou, 2020 N             N             Y             N             Y             N             N             Y             Y             Y                7              90%
18. E Armas, Tori, & Netto, 2020 Y             Y             N             Y             N             N             Y             N             N             N                3              100%
19. Prokić-Cvetković, 2020 N             N             Y             Y             Y             Y             N             Y             Y             Y                6              80%
20. Karstensen & Lier, 2020 Y             N             Y             N             Y             Y             Y             Y             N             N                5              90%

*Papers with positive scores                      13           13           15           12           16           14           13           14           13           10

(% of Papers)                                                65%        65%        75%        60%        80%        70%        65%        70%        65%        50%

*Control trials with scores > 5   = 4

*Randomized controlled trials with scores > 6 =

MQS: Methodology quality score, Ag: Agreement

1. Was the assignment to treatment groups truly random? 2. Were participants blinded to the treatment allocation? 3. Was allocation to treatment groups concealed from the allocator? 4. Were the outcomes of people who withdrew described and included in the analysis? 5. Were those assessing outcomes blind to the treatment? 6. Were the control and treatment groups comparable at entry? 7. Were groups treated identically other than for the named interventions? 8. Were outcomes measured in the same way for all groups? 9. Were outcomes measured reliably? 10. Was appropriate statistical analysis used?

Table 2: Methodological quality checklist for descriptive/case series

Paper

No.          Author & year                                      1              2              3              4              5              6              7              8              9              MQS                Ag

1. Opris, Ionescu, Costinas, &

Nistoran, 2020                                     N             Y             N             Y             N             N             Y             Y             Y             8                100%

2. Arif & Shafiullah, 2022 N             N             Y             N             Y             Y             N             N             Y             9                100%
3. Bdair, 2021 Y             Y             Y             N             N             N             N             Y             N             8                90%
4. Yasmin, 2022 Y             N             N             Y             Y             Y             Y             N             Y             7                100%
5. Khan & Abid, 2021 N             N             Y             Y             Y             N             N             N             Y             8                90%
6. Suryaman & Mubarok, 2020 Y             N             Y             Y             N             N             Y             Y             N             6                80%
7. Cant & Cooper, 2017                 N             Y             N             N             Y             Y             Y             Y             Y             5                95%
8. Hassan, Mirza, & Hussain, 2020 Y             Y             Y             N             N             Y             Y             N             Y             9                80%
9. Mukhtar, Javed, Arooj, & Sethi,

2020                                                       N             Y             N             N             Y             N             N             Y             N             7                100%

10. Sarpong, Dwomoh, Boakye, &

Ofosua-Adjei, 2022                            Y             Y             N             Y             N             Y             Y             N             Y             9                80%

11. Chan, Bista, & Allen, 2022 N             N             Y             Y             Y             N             N             Y             Y                6              90%
12. Hameed, Husain, Jain, Singh, &

Sabina, 2020                                        N             N             Y             N             N             Y             Y             Y             Y             6                90%

13. Djidu et al., 2021 Y             Y             Y             N             Y             Y             Y             N             N             8                85%
14. Angel-Urdinola, Castillo-Castro

& Hoyos, 2021                                    N             Y             Y             Y             N             N             Y             Y             Y             6                100%

15. Agrawal & Pillai, 2020 Y             Y             Y             Y             Y             Y             Y             Y             Y             7                75%
16. Cen, Ruta, Al Qassem, & Ng, 2019 Y             Y             Y             N             N             N             N             Y             Y             8                90%
17. Chung, Tung, & Lou, 2020 N             N             N             Y             Y             N             Y             N             N             7                80%
18. E Armas, Tori, & Netto, 2020 Y             Y             Y             N             Y             N             Y             N             Y             9                80%
19. Prokić-Cvetković, 2020 N             Y             Y             Y             N             Y             Y             Y             Y             6                100%
20. Karstensen & Lier, 2020 Y             N             N             Y             N             N             Y             N             Y             8                90%

*Papers with positive scores                      10           12           13           11           10           9              14           11           15

(% of Papers)                                                50%        60%        65%        55%        50%        45%        70%        55%        75%

*Control trials with scores > 5   = 4

*Randomized controlled trials with scores > 6 = 9

MQS: Methodology quality score, Ag: Agreement

1. Was the study based on a random or pseudo-random sample? 2. Were the criteria for inclusion in the sample clearly defined? 3. Were confounding factors identified and strategies to deal with them stated? 4. Were outcomes assessed using objective criteria? 5. If comparisons were being made, were there sufficient descriptions of the groups? 6. Was follow-up carried out over a significant time? 7. Were the outcomes of people who withdrew described and included in the analysis? 8. Was outcomes measured reliably? 9. Was appropriate statistical analysis used? at 0% of I², which is between 0% and 100%. It was required to test the differences between groups according to various variables because the I² value determined for the current research revealed high heterogeneity (I² = 94%). As a result, moderator analyses as proposed by [82] were carried out in this study.

Moderator analysis

The analysis of the literature revealed that many factors were taken into account in the meta-analyses of online learning. In their meta-analysis of online learning, [83] investigated how the type of online learning technology and the user’s characteristics affected the extent of the impacts of the relevant application (student, teacher, professor, etc.). The heterogeneity test results in the current study revealed that the I² value was 86.84; in light of this result, it is clear that further research into the moderating effects of numerous variables has to be done. In order to thoroughly assess the analyses, we further investigated in this study what educational levels, academic disciplines, and intervention times result in a more successful application of online learning.

Intervention process

The final year technical and vocational education students participated in the experimental intervention by taking the “welding and fabrication” course in the academic year 2020-2021. The researchers designed an accomplishment exam and questions within the parameters of the course before the intervention to assess the student’s academic performance. The relevant accomplishment test was used to determine the experimental group’s pre-test results before the utilization of online learning. Due to the pandemic process, all course material in the area of welding and fabrication has been presented through a distance education system that follows the online learning paradigm. The course material was covered over the course of six weeks, and after that time, the posttest scores were collected using the same success test.

Table 3: Meta-analysis result

Study or                 Experimental                        Control                                                  Std. Mean difference          Mean diff

Subgroup               Mean     SD           Total      Mean     SD           Total      Weight   IV, Random, 95%CI   IV, Random, 95%CI

Opris, et al 20       3.22        1.28        10           3.10        2.01        10           03.5%    2.81[1.28, 2.01]

Arif et al 22           2.98        1.09        05           3.18        1.98        08           07.5%    1.27[1.00, 1.62]

Bdair, 21                               3.80        1.22        15           3.11        0.98        05           07.6%    0.11[0.21, 3.20]

Yasmin, 22           4.10        2.00        10           2.89        1.33        18           06.2%    0.81[1.10, 4.21]

Khan et al 21        3.67        1.09        10           3.71        2.10        19           04.5%    1.03[0.51, 1.09]

Hassan, et al 20   2.88        2.15        16           2.01        1.88        13           09.3%    -0.12[1.38, 2.62]

Mukhtar, et al 20 3.67        1.99        15           2.60        2.01        17           07.2%    -0.10[2.06, 1.36]

Sarpong, et al 22  2.78        1.22        19           3.07        2.01        11           02.4%    1.31[2.38, 2.39]

Chan, et al 22       4.06        2.02        11           3.78        2.06        22           08.1%    1.22[1.36, 2.90]

Hameed, et al 20 3.26        1.88        18           2.89        1.90        06           06.8%    0.41[2.10, 2.01]

Djidu et al 21        2.00        1.00        12           3.01        1.62        11           05.3%    -1.06[1.54, 1.99]

Cen, et al 19         3.48        2.10        11           2.19        1.88        10           10.2%    2.03[3.00, 2.11]

Chung, et al 20     3.18        1.88        10           3.02        1.92        14           09.2%    0.40[1.22, 1.67]

Agrawal, et al 20  2.54        1.02        19           3.22        1.06        12           05.1%    1.00[1.09, 3.64]      -2    -1      0     1     2

Karstensen            3.88        1.36        08           2.11        1.00        09           02.1%    0.37[2.10, 0.78]

Et al 20                  2.91        1.05        11           3.07        1.22        08           05.0%    1.06[0.30, 3.78]

Total (95% CI)                                   200                                         180         100.0% 0.84 [1.51, 2.49]                                 

Heterogeneity: Tau² = 0.03; Chi² = 09.01, df = 14, (p < .00009), I² = 94%

Test for overall effect: Z = 1.33 (p < .00099)

Favors (Experimental) Favours (Control)

Table 4: Pooled effect size of the scales measured

Sample size           Analysis                                                 P-value  Heterogeneity

Activity  Exp/control           mode                      WMD 95% CI                      effect                     x²     df   p

GMAW  200/180 Random                2.10[1.02, 2.35]   <.00001 3.01 14  .11

SMAW   200/180 Random                1.22[3.82, 2.86]   <.00001 2.81 14  .09

GTAW                   200/180 Fixed                      1.46[2.11, 4.21]   <.00001 2.10 14  .18

FCAW                    200/180 Fixed                      1.88[1.63, 2.61]   <.00001 1.96 14  .23

Gas Metal Arc Welding (GMAW), Shielded Metal Arc Welding (SMAW), Gas Tungsten Arc Welding (GTAW), Flux-Cored Arc Welding (FCAW)

To ascertain the combined impact sizes for each category, we looked at the sub-item achievement test scores of students in welding. The outcome indicates that all examined categories have a significantly positive difference in favor of the experimental (treatment) group compared to the control for GMAW, SMAW, GTAW and FCAW welding activities using an oxyacetylene flame with (WMD 2.10 @ 95% CI 1.02 – 2.35), (WMD 1.22 @ 95%, CI 3.82- 2.86), (WMD 1.46 @ 95% CI 2.11 – 4.21) and (WMD 1.88 @ 95%, CI 1.63 – 2.61) at p < 0.00001). There are specific welding practices and methodologies for various welding processes. Arc welding techniques, such as gas metal arc welding (GMAW), shielded metal arc welding (SMAW), gas tungsten arc welding (GTAW), and flux-cored arc welding, were the focus of the VR and AR welding training sessions (FCAW). Other names for GMAW include metal active gas (MAG) welding and metal inert gas (MIG) welding. Stick welding or manual metal arc (MMA) welding are other names for SMAW, which also uses an electrode. Last but not least, TIG (tungsten inert gas) welding is another name for GTAW. GMAW was the most widely used welding technique followed by SMAW and GTAW before FCAW which received the least amount of attention from the schools.

(See in PDF FILE)

Fig. 2: Ease of VR and AR teaching for welding practice during Covid

Sensitivity analysis

We employed the fixed-effect statistical model, which treats qualities as though they were being handled as non-random, as opposed to the random-effects model, which interprets exploratory variables as arising from random causes. Due to the variety in studies reporting test scores, we conducted a sensitivity test on individual research by dividing 10 articles from 15 studies. The development of students’ VR and AR practical abilities is significantly impacted by instructional models, as shown by the pooled effect size of recorded SMD = 2.18 @ 95% with CI = 1..02 – 1.09 @ p =.00001. To determine whether there was any publication bias, we utilized a symmetrical funnel plot shape, as seen in Fig. 3, to report the analysis of the test scores of the ten publications included in the meta-analysis. The outcome (SMD = 4.49, 95%, CI = 2.37 – 3.63 @ p .00001) demonstrates that the treatment had no negative effects on the achievement scores of the kids. This suggests that the experimental group is favored by the pooled effect using Egger’s bias indicator statistical scale (t = 1.74, p = 0.157). The primary analysis’s impact was unaffected by the outcome when it was put through the Begg-Mazumdar bias indicator test (Z = 1.48, p = 0.230).

DISCUSSION

The literature on online learning in technology education quickly developed with the increase in online learning during the Covid. With a very small number of empirical research, the majority of the literature on online education that is currently available is descriptive in character. Education in the arts and social sciences is where you’ll find the majority of the empirical literature on online teaching. Nonetheless, there is increasing evidence that online education methods are used in the fields of technology and engineering. Despite the growing use of online teaching methods in schools, little study has been done to determine how beneficial they are for teaching technology. Also, there isn’t many quantitative research that examines how well online instruction contributes to the development of psychomotor skills.

This review has been to synthesize learning outcomes in the existing literature and explore the impact of this educational intervention.

The findings of this comprehensive review on the efficiency of online pedagogy in teaching welding skills indicate some influence on students’ knowledge and abilities. However, the improvement in knowledge and skills was often rather small as significantly higher scores on welding skills implementation were reported by various welding activities. This was not self-perceived because the impact of online teaching on the practical skill acquisition of students was known. Adequate knowledge and skills are indispensable for the successful implementation of online pedagogy. Most studies show challenges to the successful implementation of online teachings like technical glitches, management support and the willingness to change current practice models were some barriers to online teaching.

Regarding the effects of online pedagogy, there were some contradictory findings. While some studies reported improvement in online pedagogy, others did not. The study of [84] reveals variable results. Online instruction has improved statistically significantly in some trials, but not in others. It was unclear whether the studies evaluated in this review will be comparably effective in teaching skills across countries. Several systematic reviews have also come to the conclusion that interventions in online teaching may have a favorable impact on students’ knowledge and expertise in subject areas relating to technology.

CONCLUSION

Covid-19’s development has forced lengthy lockdowns throughout the world, forcing some nations to complete their educational requirements via online learning. The findings of this systematic review are unambiguous in terms of the value of online instruction in teaching welding techniques to undergraduate technology students. None of the studies chosen for this review’s inclusion were created expressly to measure how well online teaching works when evaluating undergraduate technology students’ technical skills. There is proof, though, that the use of online pedagogy during COVID-19 significantly improves two of the three outcomes essential to technical reasoning: student knowledge acquisition and creative thinking. The ability of online pedagogy to use VR and AR welding training workshops and incorporate a pedagogical approach that will teachers in designing, implementing, and improving the online teaching system was reported to have increased in one study [85]. Studies like [86, 87] and [88] evaluated creative thinking and skill acquisition, respectively. Although creative and critical thinking are essential to skill development, these studies’ findings are clear that using online pedagogy in the current digital age is effective.

REFERENCES

1. Rani, K.J. and M. Roshan, Impact of COVID-19 on education. Covid 19 and Its Impact by Lockdown, 2020: p. 1.
2. Khlaif, Z.N., et al., The Covid-19 epidemic: teachers’ responses to school closure in developing countries. Technology, Pedagogy and Education, 2021. 30(1): p. 95-109.
3. Kneale, D., et al., School closure in response to epidemic outbreaks: Systems-based logic model of downstream impacts. F1000Research, 2020. 9.
4. Obiakor, T. and A.P. Adeniran, COVID-19: Impending situation threatens to deepen Nigeria’s education crisis. 2020.
5. Huang, R., et al., Disrupted classes, undisrupted learning during COVID-19 outbreak in China: application of open educational practices and resources. Smart Learning Environments, 2020. 7: p. 1-15.
6. Huang, R., et al., Guidance on open educational practices during school closures: Utilizing OER under COVID-19 pandemic in line with UNESCO OER recommendation. Beijing: Smart Learning Institute of Beijing Normal University, 2020.
7. Hajar, A. and S.A. Manan, Emergency remote English language teaching and learning: Voices of primary school students and teachers in Kazakhstan. Review of Education, 2022. 10(2): p. e3358.
8. Bozkurt, A. and R.C. Sharma, Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian journal of distance education, 2020. 15(1): p. i-vi.
9. Veletsianos, G. and S. Houlden, Radical flexibility and relationality as responses to education in times of crisis. Postdigital Science and Education, 2020. 2: p. 849-862.
10. Harahap, S.D., et al., Trends in technological innovation in education during a time of pandemic disruption: Examining the perspective of teaching digital pedagogy. Nazhruna: Jurnal Pendidikan Islam, 2022. 5(1): p. 37-50.
11. Rudhumbu, N., et al., Insight into online teaching behavior of lecturers in Zimbabwean universities during the COVID-19 era and beyond: issues and challenges. The International Journal of Information and Learning Technology, 2021.
12. Asarta, C.J. and J.R. Schmidt, The effects of online and blended experience on outcomes in a blended learning environment. The Internet and Higher Education, 2020. 44: p. 100708.
13. Puskom, U., Panduan Elearning Universitas Negeri Padang (kedua). UPT Puskom Universitas Negeri Padang, 2013.
14. Kneebone, R., et al., An innovative model for teaching and learning clinical procedures. Medical education, 2002. 36(7): p. 628-634.
15. Richmond, A.S. and R. Cummings, Implementing Kolb’s learning styles into online distance education. International journal of Technology in Teaching and Learning, 2005. 1(1): p. 45-54.
16. Brundiers, K., A. Wiek, and C.L. Redman, Real‐world learning opportunities in sustainability: from classroom into the real world. International Journal of Sustainability in Higher Education, 2010.
17. Guillot, A. and C. Collet, Construction of the motor imagery integrative model in sport: a review and theoretical investigation of motor imagery use. International Review of Sport and Exercise Psychology, 2008. 1(1): p. 31-44.
18. Kind, V., A conflict in your head: An exploration of trainee science teachers’ subject matter knowledge development and its impact on teacher self‐confidence. International Journal of Science Education, 2009. 31(11): p. 1529-1562.
19. Weiss, M.R., V. Ebbeck, and D.M. Wiese-Bjornstal, Developmental and psychological factors related to children’s observational learning of physical skills. Pediatric Exercise Science, 1993. 5(4): p. 301-317.
20. Wilford, A. and T.J. Doyle, Integrating simulation training into the nursing curriculum. British journal of nursing, 2006. 15(17): p. 926-930.
21. Ferguson, R.F. and C. Danielson, How framework for teaching and tripod 7Cs evidence distinguish key components of effective teaching. Designing teacher evaluation systems: New guidance from the measures of effective teaching project, 2015: p. 98-143.
22. Seidel, T. and R.J. Shavelson, Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of educational research, 2007. 77(4): p. 454-499.
23. Harman, S., The Bill and Melinda Gates Foundation and legitimacy in global health governance. Global Governance, 2016: p. 349-368.
24. Chan, V.S., et al., VR and AR virtual welding for psychomotor skills: a systematic review. Multimedia Tools and Applications, 2022: p. 1-35.
25. Bickerstaff, G.F., The use of welding simulators improve proficiency in entry-level welding students. Paper in Lieu of Thesis. Northwestern State University of Louisiana, 2015.
26. Ipsita, A., et al. VRFromX: from scanned reality to interactive virtual experience with human-in-the-loop. in Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021.
27. Prokić-Cvetković, R. Possibilities of the Application of ghe Software for Welding in Virtual Reality on the Welders Training. in Sinteza 2020-International Scientific Conference on Information Technology and Data Related Research. 2020. Singidunum University.
28. Angel-Urdinola, D.F., C. Castillo-Castro, and A. Hoyos, Meta-analysis assessing the effects of virtual reality training on student learning and skills development. 2021.
29. Stone, R., et al., Full virtual reality vs. integrated virtual reality training in welding. 2013.
30. Liyang, H. Application of welding simulation training system. in IOP Conference Series: Materials Science and Engineering. 2018. IOP Publishing.
31. Byrd, A., R. Stone, and R. Anderson, The use of virtual welding simulators to evaluate experienced welders. 2015.
32. Nguyen, T., The effectiveness of online learning: Beyond no significant difference and future horizons. MERLOT Journal of online learning and teaching, 2015. 11(2): p. 309-319.
33. Bahasoan, A.N., et al., Effectiveness of online learning in pandemic COVID-19. International journal of science, technology & management, 2020. 1(2): p. 100-106.
34. Ali, B.M., A critical study of effectiveness of online learning on students’ achievement. Baig, MA (2011). RESEARCH PAPERS A CRITICAL STUDY OF EFFECTIVENESS OF ONLINE LEARNING ON STUDENTS’ACHIEVEMENT. I-Manager’s Journal of Educational Technology, 2011. 7(4): p. 28-34.
35. Srivastava, P., Advantages & disadvantages of e-education & e-learning. Journal of Retail Marketing & Distribution Management, 2019. 2(3): p. 22-27.
36. Zheng, B., et al., Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of educational research, 2016. 86(4): p. 1052-1084.
37. Demirci, A., Internet ile tüketici eğitimi. Ankara: Gazi Üniversitesi Eğitim Bilimleri Enstitüsü, 2006.
38. Emine, A. and Y.E. KARAKAYA, SPOR EĞİTİMİ KURUMLARINDA GÖREV YAPAN AKADEMİK PERSONELİN UZAKTAN EĞİTİME YÖNELİK GÖRÜŞLERİ: NİTEL BİR ÇALIŞMA. SPORMETRE Beden Eğitimi ve Spor Bilimleri Dergisi. 18(2): p. 1-12.
39. ÜÇÜNCÜ, G. and M. YILMAZ, Ekolojik ayak izi kavramının kullanılmasının 7. sınıf öğrencilerinin çevreye yönelik tutumları ve tüketim tercihleri üzerine etkisi. Gazi Eğitim Bilimleri Dergisi, 2019. 5(2): p. 81-94.
40. Caskurlu, S., et al., The qualitative evidence behind the factors impacting online learning experiences as informed by the community of inquiry framework: A thematic synthesis. Computers & Education, 2021. 165: p. 104111.
41. Batdı, V., Y. Doğan, and T. Talan, Effectiveness of online learning: a multi-complementary approach research with responses from the COVID-19 pandemic period. Interactive Learning Environments, 2021: p. 1-34.
42. Satyawan, I.M., W. Wahjoedi, and I.K.I. Swadesi, The effectiveness of online learning through Undiksha e-learning during the covid-19 pandemic. Journal of Education Technology, 2021. 5(2): p. 191-199.
43. Butnaru, G.I., et al., The effectiveness of online education during covid 19 pandemic—a comparative analysis between the perceptions of academic students and high school students from romania. Sustainability, 2021. 13(9): p. 5311.
44. Viner, R.M., et al., School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review. The Lancet Child & Adolescent Health, 2020. 4(5): p. 397-404.
45. Huang, R., et al., Handbook on facilitating flexible learning during educational disruption: The Chinese experience in maintaining undisrupted learning in COVID-19 outbreak. Beijing: Smart Learning Institute of Beijing Normal University, 2020. 46.
46. Malkawi, E., A.K. Bawaneh, and M. Bawa’aneh, Campus off, education on: UAEU Students’ satisfaction and attitudes towards e-learning and virtual classes during COVID-19 pandemic. Contemporary Educational Technology, 2020. 13(1): p. ep283.
47. Zhou, L., et al., ‘School’s out, but class’ on’, the largest online education in the world today: Taking China’s practical exploration during The COVID-19 epidemic prevention and control as an example. Best evid chin edu, 2020. 4(2): p. 501-519.
48. Sullins, D.P., Absence of Behavioral Harm Following Non-efficacious Sexual Orientation Change Efforts: A Retrospective Study of United States Sexual Minority Adults, 2016–2018. Frontiers in Psychology, 2022. 13.
49. Mulyanti, B., W. Purnama, and R.E. Pawinanto, Distance learning in vocational high schools during the covid-19 pandemic in West Java province, Indonesia. Indonesian Journal of Science and Technology, 2020: p. 271-282.
50. Tzivinikou, S., G. Charitaki, and D. Kagkara, Distance education attitudes (DEAS) during Covid-19 crisis: factor structure, reliability and construct validity of the brief DEA Scale in Greek-Speaking SEND teachers. Technology, Knowledge and Learning, 2021. 26(3): p. 461-479.
51. Coman, C., Ţ îru, LG, Mese ş an-Schmitz, L., Stanciu, C., & Bularca, MC (2020). Online Teaching and Learning in Higher Education during the Coronavirus Pandemic: Students’ Perspective. Sustainability, 12 (24), 10367.
52. Muthuprasad, T., S. Aiswarya, and K. Aditya, Girish., K.(2021). Students’ perception and preference for online education in India duringCOVID-19 pandemic. Social Sciences & Humanities. https://doi. org/10.1016/j. ssaho, 2020.
53. Paudel, P., Educación en línea: Beneficios, desafíos y estrategias durante y después del COVID-19 en la educación superior. Revista Internacional de Estudios en Educación, 2021. 3(2): p. 70-85.
54. Baticulon, R.E., et al., Barriers to online learning in the time of COVID-19: A national survey of medical students in the Philippines. Medical science educator, 2021. 31(2): p. 615-626.
55. García-Peñalvo, F.J., et al., Impact of the COVID-19 on higher education: an experience-based approach, in Information technology Trends for a global and Interdisciplinary research community. 2021, IGI Global. p. 1-18.
56. Kundu, A. and T. Bej, COVID-19 response: students’ readiness for shifting classes online. Corporate Governance: The International Journal of Business in Society, 2021.
57. Singh, G. and G. Hardaker, Barriers and enablers to adoption and diffusion of eLearning: A systematic review of the literature–a need for an integrative approach. Education+ Training, 2014.
58. Al Mulhem, A., Investigating the effects of quality factors and organizational factors on university students’ satisfaction of e-learning system quality. Cogent Education, 2020. 7(1): p. 1787004.
59. Zhu, C. and K. Justice Mugenyi, A SWOT analysis of the integration of e-learning at a university in Uganda and a university in Tanzania. Technology, Pedagogy and Education, 2015. 24(5): p. 1-19.
60. Strike, T., Administrative planning, higher education institutions. Encyclopedia of International Higher Education Systems and Institutions, 2018. 6: p. 1-6.
61. Adedoja, G., The influence of age and educational qualification on stakeholders perception of integrating mobile technology into basic education in Nigeria. African Research Review, 2016. 10(3): p. 96-110.
62. Debattista, M., A comprehensive rubric for instructional design in e-learning. The International Journal of Information and Learning Technology, 2018.
63. Ashwin, P. and D. McVitty, The meanings of student engagement: Implications for policies and practices, in The European higher education area. 2015, Springer, Cham. p. 343-359.
64. Steed, A., et al., Evaluating immersive experiences during Covid-19 and beyond. interactions, 2020. 27(4): p. 62-67.
65. Garcia Estrada, J. and E. Prasolova-Førland, Running an XR lab in the context of COVID-19 pandemic: Lessons learned from a Norwegian university. Education and Information Technologies, 2022. 27(1): p. 773-789.
66. Khan, Z.H. and M.I. Abid, Distance learning in engineering education: Challenges and opportunities during COVID-19 pandemic crisis in Pakistan. The International Journal of Electrical Engineering & Education, 2021: p. 0020720920988493.
67. Marinoni, G. and H. van’t Land, The impact of COVID-19 on global higher education. International higher education, 2020(102): p. 7-9.
68. Page, M.J., et al., The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews, 2021. 10(1): p. 1-11.
69. Kirkpatrick, D. and J.D. Kirpatrick, The Kirkpatrick four levels. 2011: Kirkpatrick Partners.
70. Moher, D., et al., Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement (Chinese edition). Journal of Chinese Integrative Medicine, 2009. 7(9): p. 889-896.
71. Higgins, J., Assessing risk of bias in included studies. Cochrane handbook for systematic reviews of interventions, version 5.1. 0, 2011.
72. Schultz, T. and Z. Florence, Joanna Briggs institute meta-analysis of statistics assessment and review instrument. Adelaide: Joanna Briggs Institute, 2007.
73. Alderson, P., et al., The albumin revierwers. Human albumin solution for resuscitation and volume expansion in critically ill patients.(Cochrane Review). The Cochrane Library, 2005. 1.
74. Thomas, J., et al., Living systematic reviews: 2. Combining human and machine effort. Journal of clinical epidemiology, 2017. 91: p. 31-37.
75. Ghogomu, E.A., et al., Updated method guidelines for cochrane musculoskeletal group systematic reviews and metaanalyses. The Journal of rheumatology, 2014. 41(2): p. 194-205.
76. Wells, G.A., et al., Checklists of methodological issues for review authors to consider when including non‐randomized studies in systematic reviews. Research synthesis methods, 2013. 4(1): p. 63-77.
77. Saltelli, A., et al., Global sensitivity analysis: the primer. 2008: John Wiley & Sons.
78. Egger, M., et al., Bias in meta-analysis detected by a simple, graphical test. Bmj, 1997. 315(7109): p. 629-634.
79. Borenstein, M., et al., A basic introduction to fixed‐effect and random‐effects models for meta‐analysis. Research synthesis methods, 2010. 1(2): p. 97-111.
80. Borenstein, M., et al., Introduction to meta-analysis. 2021: John Wiley & Sons.
81. Schmidt, D.A., et al., Technological pedagogical content knowledge (TPACK) the development and validation of an assessment instrument for preservice teachers. Journal of research on Technology in Education, 2009. 42(2): p. 123-149.
82. Wood, R.E., A.J. Mento, and E.A. Locke, Task complexity as a moderator of goal effects: A meta-analysis. Journal of applied psychology, 1987. 72(3): p. 416.
83. Šumak, B., M. Heričko, and M. Pušnik, A meta-analysis of e-learning technology acceptance: The role of user types and e-learning technology types. Computers in human behavior, 2011. 27(6): p. 2067-2077.
84. Sarpong, S.A., et al., Online teaching and learning under COVID-19 pandemic; perception of university students in Ghana. European Journal of Interactive Multimedia and Education, 2021. 3(1): p. e02203.
85. Chan, V.S., et al., VR and AR virtual welding for psychomotor skills: a systematic review. Multimedia Tools and Applications, 2022. 81(9): p. 12459-12493.
86. Yasmin, F., et al., Safety and efficacy of colchicine in COVID-19 patients: A systematic review and meta-analysis of randomized control trials. PloS one, 2022. 17(4): p. e0266245.
87. Karstensen, S. and A.R. Lier, Virtual welding: A didactic perspective. Nordic Journal of Vocational Education and Training, 2020. 10(1): p. 95-107.
88. Chung, C.-c., C.-C. Tung, and S.-J. Lou, Research on optimization of VR welding course development with ANP and satisfaction evaluation. Electronics, 2020. 9(10): p. 1673.