Assessment and Learning Mathematics Integrated 21st Century Competency-Computational Thinking Skills in Junior High School: Partly Detailed and Completed Intervention Stage

Hisyam Ihsan¹, Muhammad Ammar Naufal²

^1,2Department of Mathematics Education, Universitas Negeri Makassar, Makassar, Indonesia

ABSTRACT: This research is intended to develop the intervention of mathematics assessment and learning in junior high school integrated 21st Century competence-computational thinking skills. The research method is Design-based Research at the partly detailed and completed intervention stage. Twofold yield of the intervention of the research design are prototypical intervention (instructional materials) and its accompanying design principles (design model of assessment and learning). The instructional materials that have been developed are (1) blueprints for assessment and learning, (2) lesson plans containing outline topics, student worksheets, project tasks and homework, and (3) LMS e-Learning. The design principle that accompanies the intervention is a design model for mathematics learning and assessment integrated computational thinking skills. The results of the formative evaluation at the partly detailed and completed intervention stage showed that the development of intervention met the quality criteria of being practical and effective.

KEYWORDS: Assessment and learning mathematics, Computational thinking, Instructional materials, Design model, formative evaluation

I. INTRODUCTION

Computational thinking has become a popular issue in mathematics education [1] in recent years. According to [2] computational thinking and mathematics are “natural allies.” In terms of logical structure and the ability to explore and model mathematical relations, [3] contends that computational thinking and mathematics are closely related. Both computational and mathematical thinking use cognition, metacognition, and disposition centers on problem-solving [4]. They identify and develop social-cultural learning opportunities that affect real-world thinking and practice.

In mathematics, critical thinking is the capability to find patterns, generalize the pattern, and construct a commonly accepted solution for all situations and identify excluded cases. Computer science advances involve the ability to recognize common patterns and special circumstances. This capability is called computational thinking. Computational thinking, also called the ‘5th C’ of 21st-century skills, has attracted the interest of educators in several topics,from science to math to social studies [5]. Critical thinking, creativity, collaboration, and communication (4C) are becoming more important in schools. This trend has encouraged K-12 schools to adopt project-based learning, inquiry learning, and deeper learning to promote Higher Order Thinking Skill (HOTS) throughout the learning process. With the expansion of technology and information in the era of Industry 4.0, experts agree that computational thinking is a key 21st-century ability for learners. This does not mean that every student should be a computer scientist, nor is basic mathematics and science literacy required to comprehend how the world works. Therefore, education should focus on developing computational skills, which are now important to every job [6].

II. METHOD

Design-based Research is used in this study. As in Figure 1, [7], [8], and [9] indicate that Design-Based Research is a systematic and flexible research method that attempts to enhance educational practice utilizing iterative analysis (cycles), design, development, and implementation, based on collaboration between researchers and practitioners in real-world settings, and contextualizing sensitive design principles and theories.

Based on this method, development of intervention, mathematics assessment and learning integrated computational thinking skills (AP-KBKM) in junior high school, obtained twofold yield [10] in the form of (a) development of prototypical intervention, i.e., instructional materials of AP-KBKM, and (b) construction of design principles, i.e., design model of AP-KBKM.

(See in PDF File)

Fig. 1 Design-based research procedures [11]

(See in PDF File)

Fig. 2 Formative evaluation of prototypical interventions [10]

As in Figure 2, with adaptation [12], the first cycle is the global design stage, prototype version I is evaluated to obtain prototype version II with two intervention quality criteria, i.e., relevance and consistency through appraisal by three experts. The results of the first cycle have been reported and disseminated [13] and [14]. In the second cycle, which is a partly detailed and completed intervention stage, the prototype version II is evaluated to obtain a prototype version III with two intervention quality criteria, i.e., practicality and effectiveness through try out, then discussed in a panel session with experts to produce a prototype version IV.

III.  RESULTS DISCUSSIONS

In this second cycle, the results of the research on the partly detailed and completed intervention stage are presented below.

1. Practical Problem Analysis

Analysis of practical problems begins by looking at various problems that occur in the first cycle, related to research objectives with a twofold yield of research design. Continuing discussions through teams and practitioners, tracing the latest references, and increasing intervention activities.

1. Solution Development (Theoretical Framework)

The results obtained from the analysis of practical problems are then rewritten in the conceptual framework of AP-KBKM, which becomes a guideline in carrying out refinement and improvement of interventions as intended in the twofold yield of research design to obtain practical and effective interventions.

1. Evaluation and Testing (in Practice)

Results of Data Analysis for Practicality

From the three classes that were sampled in junior high school. The results of the descriptive analysis show that by using criterion 3 (learning is carried out normally), the average score of the observer’s assessment of the instrument (items 1-50), all of which are above 3, which means that intervention through the prototype does not require correction.

The results of inferential analysis through generalization theory and implementation fidelity show that after the test assumptions are met, the one-way ANOVA F-test for the prototype dimensions of instructional materials shows that F-stat =0.338 and p=0.644, which means the hypothesis has no effect on the dimensions of the implementation of learning in the population can be accepted at the significance level α=0,05<p.

Furthermore, the prototype is assessed based on fidelity of implementation through intraclass correlation (ICC) using the formula by [15]. The results of data processing obtained that ICC = 0.427. Based on the category of [16], the ICC value is quite adequate. So, the enthusiasm and willingness of educators and students to carry out the learning process through a prototype (fidelity of implementation) is stated to be quite adequate. This means that the intervention through prototype II of the instructional materials is declared practical.

Results of Data Analysis for Effectiveness

For the three sample classes that were tested through pre and posttest, descriptively, the average score of all components of computational thinking is above 0, which means that students can do the tasks given. The total mean for the pretest is 15.85 and for the post test is 67.50 out of a possible score of 100. So, on average score there is a gain, with a normalized gain (N-gain) average = 61% which is classified as medium [17]. This calculation uses the following 〈cg〉 adjusted N-Gain [18].

Inferentially, by taking the expected outcome criteria that the students posttest average can exceed 60 out of maximum 100 that is possible, it can be hypothesized (Ha) that μ: the average KBKM Grade 7-9 post-test in the population is more than 60, or can be written as

Ho: μ = 60 versus alternative Ha: μ > 60.

The results from the t-test of one sample obtained a statistical value t-test = 2.482 with p = 0.010 and Cohen’s d = 0.453. Because the actual risk from the sample (p-value) is quite small, it means that the test is significant at the level α = 0,05 > p. This means that there is an effect in the population, and the magnitude of the effect is d = 0.453 ≈ 45%, which is moderate [19]. Therefore, in the population, the learning outcomes (post-test) of students in learning mathematics integrated computational thinking skills are more than 60 and are significant with a moderate significant effect contribution.

Thus, the results above, the average total achievement of the KBKM component through the N-gain is classified as medium and the effect size of learning outcome is also moderate, then it can be categorized that the level of intervention effectiveness is middling [20].

Revision and Improvement (Prot. III & IV)

Based on the results of data analysis, data distribution, and suggestions for correction and improvement obtained through a tryout with teachers educate students and also a tryout in 3 classrooms (pilot studies 1 and 2), the prototype version III was obtained.

The results of the pilot study provide a twofold yield of research design as already mentioned. Through panel sessions with experts in focused discussions with the research team in discussing the research outcomes monitored based on indicators and conceptual dimensions of weak interventions and student learning outcomes. The AP-KBKM design model framework is also considered and discussed based on (1) procedural design principles: characteristics of the design approach, (2) substantive design principles: design characteristics (intervention) itself [10]. That’s the prototype version IV.

1. Design Principles

In [21] note that, “a teaching-learning model is a generalized instructional process which may be used for many different in a variety of subjects”. Based on these quotes it can be said that a model of learning, in general, can be applied to various disciplines including mathematics. This does not mean that a learning model is suitable for every topic in a subject.

The main component of a model is the phase of learning implementation or often called the model structure or syntax. The design model of AP-KBKM consists of four phases as follows:

First phase: competency framework; Second phase: standard of skills and knowledge; Third phase: training if assessment; and Fourth phase: certification.

The syntax of AP-KBKM model is included in the CBL (Competency-Based Learning) model group ([22] and [23]). This AP-KBKM model is included (inline) in various learning models that teachers apply in the learning process in their classes.

CONCLUSION

Basic research through R & D has been carried out at the partly detailed and completed intervention stage and provides model design for assessment and learning of mathematics in junior high schools integrated 21st century competencies-computational thinking skills, supported by instructional materials prototype that are practical and effective.

ACKNOWLEDGMENTS

On this occasion, I would like to express my greetings and gratitude to all those who have helped in conducting this research, especially the team members, lecturers and teachers who participated, especially thanks to the Rector and his staff at the UNM Research Institute. Thank you also to the Ministry of Education and Technology of Indonesia who has funded the research.

REFERENCES

1. Broley, C. Buteau and E. Muller, “(Legitimate peripheral) computational thinking in mathematics,” in CERME, Dublin, Ireland, 10, Feb 2017.
2. English, “On MTL’s Second Milestone: Exploring Computational Thinking and Mathematics Learning,” Mathematical Thinking and Learnin, vol. 20, no. 1, pp. 1-2, 2018. DOI: https://doi.org/10.1080/10986065.2018.1405615.
3. Gadanidis, J. M. Hughes, L. Minniti and B. J. G. White, “Computational Thinking, Grade 1 Students and the Binomial Theorem,” Digital Experiences in Mathematics Education, vol. 3, no. 2, pp. 77-96, 2016. DOI: https://doi.org/10.1007/s40751-016-0019-3.
4. Kallia, S. P. v. Borkulo, P. Drijvers, E. Barendsen and J. Tolboom, “Characterising computational thinking in mathematics education: a literature-informed Delphi study,” Research in Mathematics Education, vol. 23, no. 2, pp. 159-187, 2021. https://doi.org/10.1080/14794802.2020.1852104.
5. Noonoo, “Computational Thinking Is Critical Thinking. And It Works in Any Subject,” 2019. https://www.edsurge.com/news/2019-05-21-computational-thinking-is-critical-thinking-and-it-works-in-any-subject
6. OECD, “PISA 2021 Mathematics Framework (second draft),” Paris: PISA, OECD Publishing, 2018.
7. A. Sandoval and P. Bell, “Design-Based Research Methods for Studying Learning in Context: Introduction,” Educational Psychologist, vol. 39, no. 4, pp. 199-201, 2004. DOI: 10.1207/s15326985ep3904_1.
8. Wang and M. J. Hannafin, “Design-based research and technology-enhanced learning environments,” Educational Technology Research and Development, vol. 54, no. 4, pp. 5-23, 2005. DOI: https://doi.org/10.1007/BF02504682.
9. Amiel and T. C. Reeves, “Design-Based Research and Educational Technology: Rethinking Technology and the Research Agenda.,” Journal of Educational Technology & Society, vol. 11, no. 4. International Forum of Educational Technology & Society, p. 29–40, 2008.

http://www.jstor.org/stable/jeductechsoci.11.4.29..

10. v. d. Akker, B. Bannan, A. Kelly, N. Nieveen and T. Plomp, Educational Design Research: Part A: An introduction., Netherland: Enschede: SLO., 2013. International – SLO p.23, 29.
11. C. Reeves, “Enhancing the Worth of Instructional Technology Research through “Design Experiments” and Other Development Research Strategies,” in International Perspectives on Instructional Technology Research for the 21st Centur, New Orleans, LA, USA., Paper presented on April 27, 2000.
12. Nieveen, “Formative Evaluation in Educational Design Research,” in Plomp, T., & Nieveen, N. M. (Eds.) (2010). An introduction to educational design research: Proceedings of the seminar conducted at the East China Normal University, Shanghai, 2007.
13. Ihsan, F. Arwadi, and Sutamrin, “The Elementary Education Assessment and Learning Integrated 21st Century-Computational Thinking Skills in Mathematics: Global Design Stage,” in Proceedings of the 1st World Conference on Social and Humanities Research (W-SHARE 2021), 2022. https://doi.org/10.2991/assehr.k.220402.002
14. Ihsan, Sutamrin, F. Arwadi, A. Zaki, & M. A. Naufal, “Keterampilan Berpikir Komputasi pada Matematika.” Makassar. Nas Media. 1st ed. 2021.
15. Tong, S. Tang, B. J. Irby, R. Lara-Alecio, C. Guerrero, and T. Lopez, “A process for establishing and maintaining inter-rater reliability for two observation instruments as a fidelity of implementation measure: A large-scale randomized controlled trial perspective,” Studies in Educational Evaluation, Vol. 62, Pages 18-29, 2019. https://doi.org/10.1016/j.stueduc.2019.04.008.
16. David, B. Elving and K.S. Roaldsen, “Intraclass correlation – A discussion and demonstration of basic features.”PloS on vol. 14,7 e0219854. 22 Jul. 2019, doi:10.1371/journal.pone.0219854
17. Hake, “Interactive-Engagement Versus Traditional Methods: A Six-Thousand-Student Survey of Mechanics Test Data for Introductory Physics Courses,” American Journal of Physics Volume 66, Issue 1, pp 64-74. 1998. https://doi.org/10.1119/1.18809.
18. P. Coletta, & J. J. Steinert, “Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories.” Physical Review Physics Education Research, 16(1), 1-7. 2020.

https://doi.org/10.1103/PhysRevPhysEducRes.16.010108

19. Cohen, “Statistical Power Analysis for the Behavioral Sciences.” New York, NY: Routledge Academic. 1988. [Google Scholar]
20. Ihsan, M. Darwis, Sutamrin, F. Arwadi, & M. A. Naufal, “Pengembangan Model Desain Asesmen dan Pembelajaran Matematika Terintegrasi Kompetensi Abad 21-Keterampilan Berpikir Komputasi.” Makassar. Chani Publishing, pp 22-23, 2021.
21. Bell, Teaching and Learning Mathematics (In Secondary Schools), Dubuque, Iowa: Wm. C. Brown Company Publisher. , 1981.
22. A. Voorhees, “Competency-Based Learning Models: A Necessary Future,” New Directions for Institutional Research, vol. 2001, no. 110, pp. 5-13, 2001. https://doi.org/10.1002/ir.7.
23. PAHOO, “Assessment strategies training on competency-based education for health professionals focused on maternal and perinatal health” Virtual campus for public health, 2021.