Reliability and Validity Testing of the Teacher Digital Literacy Scale

Zhao Meiyu¹, Siti Maziha Mustapha²

^1,2 Infrastructure University Kuala Lumpur, Malaysia

ABSTRACT: The purpose of this study was to test the reliability and validity of the Teacher Digital Literacy Scale. Methods: A total of 240 respondents were sampled from 4 universities in Changchun City, Jilin Province， China, and a proportional random sampling method was used to generation sample, after screening and cleaning the data, 194 questionnaires were confirmed as valid questionnaires. The data analysis was performed with the aid of SPSS version 26.0. The Cronbach’s alpha coefficient of the scale is 0.923, and the lowest Cronbach’s alpha coefficient for the five dimensions digital awareness, digital technology knowledge and skills, digital application, digital social responsibility, and professional development is 0.797, indicating that the reliability of the scale is good. For the Exploratory Factor Analysis (EFA), all scale items performed well, and based on the analysis results, the scale is divided into 5 dimensions. The final scale is suitable for the next step of field research.

KEYWORDS: Teacher digital literacy, Exploratory factor analysis, Reliability, Validity

1. INTRODUCTION

In January 2020, the World Economic Forum released a report titled “Schools of the Future: Defining New Models of Education for the Fourth Industrial Revolution”, stating that human society is undergoing the Fourth Industrial Revolution. Educators, must continuously improve their literacy, keep up with the pace of the times, and cultivate a workforce that meets the needs of society. The development of society and the prosperity of the economy play a crucial role (Teixeira et al. 2021).

Many scholars have conducted research on teacher digital literacy and provided the connotation of digital literacy. Li et al. (2022) pointed out in their study that in the digital age, teacher digital literacy should not only be limited to knowledge and skills but should be a transformation of digital thinking and the emergence of innovative consciousness. Innovating teaching models and integrating digital technology into the teaching process are all challenges teachers face in improving their digital literacy. Similarly, Qi and Zhang (2022) believe that teachers in the digital age need to improve their Internet thinking and Internet teaching ability. They pointed out that the focus of Internet thinking is equal voice, and the status of students and teachers is equal, which requires teachers to make positive changes.

Ge and Han (2017) supplemented what digital literacy teachers should possess from the perspective of ethical cognition. They believe that in addition to the professional knowledge involved in teaching and the application of digital technology, in the digital age, teacher digital literacy should also include digital security and ethics literacy, as well as the literacy of correctly understanding virtual and reality.

In November 2022, the Chinese Ministry of Education released the education industry standard for “Teacher Digital Literacy”, and China has reached a basic consensus on the connotation of teacher digital literacy. This standard provides a definition and framework for teacher digital literacy.

2. THE FRAMEWORK OF TEACHER DIGITAL LITERACY

There are five dimensions in the framework released by the Chinese Ministry of Education, that is digital awareness, digital technology knowledge and skills, digital application, digital social responsibility, and professional development.

2.1 DIGITAL AWARENESS (DAW)

In industry standards, digital awareness has three dimensions: digital understanding, digital willingness, and digital willpower.

Digital understanding refers to teachers’ understanding of the value of digital technology in the economy society and education, as well as their understanding of potential new issues that may arise in education (Ministry of Education of the People’s Republic of China, 2022).

Digital willingness is the attitude of teachers towards digital technology resources and their applications in education and teaching (Ministry of Education of the People’s Republic of China, 2022).

Digital willpower refers to the belief that teachers have a positive attitude towards overcoming difficulties and solving problems in the face of digital education (Ministry of Education of the People’s Republic of China, 2022).

2.2 DIGITAL TECHNOLOGY KNOWLEDGE AND SKILLS (DTKS)

Digital technology knowledge is a common knowledge of digital technology that teachers should understand, such as the concept and basic principles of digital technology (Ministry of Education of the People’s Republic of China, 2022).

Digital technology skills are the skills that teachers should master in the application of digital technology resources, such as what resources to choose, what software to use, and how to use them (Ministry of Education of the People’s Republic of China, 2022).

2.3 DIGITAL APPLICATION (DAP)

There are four dimensions in digital application, which are digital instructional design, implementation of digital teaching, digital industry evaluation, and digital collaborative education.

2.4 DIGITAL SOCIAL RESPONSIBILITY (DSR)

In industry standards, legal ethics and digital security both belong to the dimension of digital social responsibility. Teachers should pay attention to protecting information security and distinguishing the authenticity of information while applying the network in accordance with the law and regulations.

2.5 PROFESSIONAL DEVELOPMENT (PD)

Within this dimension, includes the learning, research, and innovation of digital technology resources. This has both internal and external factors. The internal factor is the willingness of teachers to spontaneously improve their digital literacy, while the external factor is the management and training of teachers by universities.

3. MEASUREMENT OF TEACHER DIGITAL LITERACY

The instrument to measure teacher digital literacy was developed by Lu (2024) in the study “Current situation analysis and influencing factors research in digital literacy of secondary school mathematics teachers”. The instrument investigates teacher digital literacy from five dimensions as mentioned earlier. On the other hand, the instrument was developed by Lu in 2024, it studied teacher digital literacy under the education industry standard released by the Ministry of Education of China.

There were 33 items in this instrument, excluding questions on the respondents’ demographic profile. DAW with 6 items, DTKS with 10 items, DAP with 9 items, DSR with 5 items, and PD with 3 items.

4. METHODOLOGY

This study is a Non-experimental survey research, whereby data were collected at Changchun City, China.

A proportionate random sampling was conducted according to the proportion of teachers in four universities after getting the permission of these universities in the study area. A total of 194 questionnaires were distributed through the online survey platform. Exclude questionnaires that meet the following criteria as invalid: questionnaires filled out for too short or too long, such as less than 60 seconds or more than 20 minutes, which are significantly different from normal time; All options are the same (such as selecting all “never” options); There are a large number of unselected items in the questionnaire, accounting for more than 10% of the total number of questions. Finally, 194 questionnaires were confirmed as valid questionnaires for the next data analysis, the effective rate is 100%.

Using exploratory factor analysis to test the indicator system of teacher digital literacy. Using SPSS 26.0 software for KMO test, principal component analysis, factor extraction, etc.

5. RALIABILITY ANALYSIS

There were 33 items in the questionnaire, and teacher digital literacy was initially measured by five dimensions as mentioned earlier.

This study was conducted in Changchun City, China, in August 2024, and the demographic profile of 194 respondents can be found in Table 1.

For the reliability analysis, the reliability of each dimension was calculated. The results as shown in Table 2, indicated that the total reliability value for all 33 items in this research was 0.923, while the reliability values for each dimension were 0.876, 0.932, 0.925, 0.893, and 0.797, respectively. The result showed teacher’s digital literacy questionnaire is reliable. The reliability values for each dimension of this questionnaire survey were above 0.7, which is considered acceptable (Yockey, 2019).

          Table 2. Reliability Analysis of Teacher Digital Literacy Scale (n=194)

6 EXPLORATORY FACTOR ANALYSIS

Table 3 shows the results of the KMO and Bartlett’s test of teacher digital literacy. The value of the KMO was 0.914. This value is higher than the threshold value of 0.6 (Hair et al., 2018). Bartlett’s Test of Sphericity was also a significant result, with a Chi-Square value of 4434.166 and a p-value of 0.000.

Based on PCA with varimax rotation, 33 items were analyzed, and the results are shown in Table 4. The analysis results showed that there were 5 components with eigenvalue greater than 1.0, so 5 components were extracted and the total explained variance was 65.062%, above the minimum value of acceptable variance interpretation. Specifically, component 1 contributed 29.530% of the variance, component 2 contributed 14.034%, component 3 contributed 8.847%, component 4 contributed 7.887%, and component 5 contributed 4.763% resulting in a cumulative explanation of 65.062% of the variance.

As shown in Figure 1, the scree plot sorted the 33 items into five components. Following the fifth factor, a break was discovered on the scree plot.

Figure 1: Scree Plot of Teacher Digital Literacy

In Table 5, shows the rotated component matrix, retained items, and their factor loading. From this table, it can be found that all 33 projects were classified into 5 components, and their factor loading all greater than 0.6. The five dimensions are digital technology knowledge and skills with 10 items, digital application with 9 items, digital awareness with 6 items, digital social responsibility with 5 items, and professional development with 3 items.

7. CONCLUSION

Thus, it can be determined that the 33 items on the Teacher Digital Literacy Measurem.ent are valid with respect to content. However, at the EFA stage, five main influencing factors of teacher digital literacy were extracted, they are digital awareness (DAW), digital technology knowledge and skills (DTKS), digital application (DAP), digital social responsibility (DSR), and professional development (PD).

REFERENCES

1. Ge, W., & Han, X. (2017). Teaching competency framework for teachers in a digital age. Modern Distance Education Research, 145(1), 59-67. https://doi.org/10.3969/j.issn.1009-5195.2017.01.008
2. Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E. (2018). Multivariate Data Analysis (8th ed.). United Kingdom: Cengage Learning.
3. Li, M., Han, X., Li, M., & Zhou, Q. (2022). Digital transformation in higher education teaching and learning: Vision, challenges and responses. China Educational Technology, 426(07), 23-30.
4. Ministry of Education of the People’s Republic of China (Ed) 2022. Notice of the ministry of education on the release of the ” Digital Literacy of Teachers” education industry standard. Available online: http://www.moe.gov.cn/srcsite/A16/s3342/202302/t20230214_1044634.html (accessed on 1 February 2023)
5. Qi, H., & Zhang, J. (2022). The development of digital technology and the reform of teaching mode in higher education. China Higher Education, 2022(18), 56-58.
6. Teixeira, A. F., Gonçalves, M. J. A., & Taylor, M. de L. M. (2021). How higher education institutions are driving to digital transformation: A case study. Education Sciences, 11(10). https://doi.org/10.3390/educsci11100636

7. Yockey, R. D. (2019). SPSS Demystified (3th ed). China Renmin University Press.