Measuring Mathematical Competence of Grade 3 Students in New General Curriculum: A Case of Vietnam

Thi Kieu Oanh Nguyen^1*, Xuan Cuong Dang^2*

^1,2 The Vietnam National Institute of Educational Sciences, Hanoi, Vietnam

ABSTRACT: The general education curriculum promulgated by the Vietnamese Ministry of Education and Training in 2018 has identified 10 common competencies that need to be formed and developed for students in the coming period, including computing capacity. The Math general education curriculum issued in 2018 helps students form and develop mathematical competence (the most concentrated expression of computing capacity) including the following core elements: mathematical thinking and reasoning; mathematical modeling; mathematical problem solving; mathematical communication; use mathematical learning tools and media.

The aim of this study was to design an assessment framework for mathematical competence.

From the Grade 3 Math curriculum, the research team has proposed a math competency standard with 90 items. From these 90 items, design 6 illustrative test questions. Each test consists of 30 questions, between which there are 15 fixed questions designed to measure students’ mathematical competence.

A sample of 383 students in grade 3 were recruited for the study with the participation of 3 researchers and 5 elementary school teachers. A mathematical competence framework and six tests with anchor items were designed to measure students’ mathematical competence.

The results of analysis using an approach to item response theory showed that the items had good fits with the model and they could be used to describe mathematical competence learning progression with different levels of proficiency. The tests were reliable and valid, and the anchor items were good for connecting students’ mathematical competence.

KEYWORDS: Vietnam, Grade 3 Students, New General Curriculum, Mathematical Competence, Measuring

1. INTRODUCTION

In the context of Vietnam’s fundamental and comprehensive reform of general education, Vietnam’s General Education Program was issued in 2018 with the orientation of developing students’ qualities and competence. According to this program document, Mathematics contributes to the formation and development of students’ general and mathematical competence. Vietnam’s 2018 Mathematics curriculum describes the mathematical competency requirements for each educational level (elementary school, middle school, high school). However, the math curriculum does not describe in detail the requirements for math competency in each grade. This makes it often difficult for teachers to assess students’ mathematical competence during the process of teaching mathematics in each class. Therefore, if we can build standards to assess mathematical competence in each class, it will help teachers easily design tools to assess mathematical competence during the teaching process.

The objective of this study is to identifies the steps to develop a standard for assessing mathematical competence. From that, a toolkit of 6 tests was designed to measure the mathematical competence of 3rd grade students. The toolkit will be tested on a sample of 383 3rd grade students. Analyze the test results to know the content of the assessment of mathematical competence that is appropriate to the model and the content of the assessment that is not appropriate, from It has been adjusted to have a more appropriate set of assessment tools for mathematical competence in teaching grade 3 Mathematics at the primary level.

Regarding methods, the research team used a combination of many research methods, mainly the method of analyzing and summarizing experience; Professional solution; Experimental method.

LITERATURE REVIEW

There are different definitions on mathematical competence, mathematical literacy or numeracy. The Organisation for Economic Co-operation and Development (2012) defined mathematical literacy is the “ability to access, use, interpret, and communicate mathematical information and ideas, to engage in and manage mathematical demands of a range of situations in adult life” (p. 33). Faragher and Brown (2005) stated that numeracy is the “ability and willingness to use a wide range of mathematics in the context of people’s lives” (p. 5).

In terms of measuring mathematical literacy, researchers have been interested in developing learning progressions. They have been known as a developmental scale, and in this scale, each student can be indicated with the location where the student would know and be able to do at that level, and also be indicated the next level where the student will be move in the next stages in terms of knowledge and skills (Heritage, 2008). The learning progressions will support teachers during their teaching practice as well as help them to make decision on the development of student learning. Additionally, the learning progressions help students know what they are having or aquiring at the moment as well as what they can achieve to acquire the next level in th learning progressions. Reviewing the curriculum development processes in some countries, it can be seen that the learning progressions of the mathematical literacy have been developed in those curriculum. For instance, in the Australian national curriculum, the learning progression of mathematics literacy has been developed in different stages and this learning progression can be used by both teachers and students to support their teaching and learning practices (ACARA, 2014).

In terms of measuring mathematical literacy, some researchers also develop instruments and learning progression to have better understanding about students’ development of mathematical literacy. For example, Balt et al. (2020) measured mathematics learning of first grade students to understand students’ mathematics learning. The authors developed instruments, known as tests, to measure mathematical literacy. The instruments were used repeatedly in different time. The results of those measurements were used and analysed to have understanding about students’ mathematics skills. Strickland et al. (2016) also developed instruments to measure mathematical literacy for students with additional learning needs. Item response theory was used to analyse the data, and a learning progression was proposed to explain students’ mathematical literacy. ACARA (2022) also measure mathematical literacy of Australian students in Grade 3, 5, 7, 9 in the There are ten levels of the learning progression and students’ skills in each grade level will be reported following this scale (ACARA, n.d). Some specific domain of mathematical literacy have also been measured. For instance, Kim et al. (2017) conducted research to measure geometric skills of students. Using item response theory, the authors also proposed a learning progression including five levels of geometric skills, that could be used to inform learning, teaching as well as to assess students’ geometric learning. Additionally, Callingham et al. (2019) also investigated students’ statistical reasoning skills, and proposed a learning progression including eight increasing levels.

In Vietnam’s General Education Curriculum in 2018, mathematics contributes to the formation and development of students’ mathematical competence. Mathematical competence is defined as having the following core components: mathematical thinking and reasoning competence; mathematical modeling competence; mathematical problem-solving competence; mathematical communication competence; ability to use mathematical learning tools and means.

The main purpose of the present research is to propose assessment standards for mathematical competence following by the proposed standards in new general education curriculum in Vietnam.

2. CONTENT

2.1 Theoretical framework:

The competency assessment standard according to the 2018 curriculum is considered a system of statements about the requirements/expressions of students to achieve the corresponding competencies specified in the curriculum for each grade level. The characteristic of the competency assessment standard is that it can be directly measured through different instruments, and the statements of the competency assessment standard are associated with specific educational contents. The development of competency assessment standards aims to support educational administrators, teachers and students in the process of teaching, assessing and learning in the direction of developing students’ competencies. The research proposed the process of developing standards for assessing mathematical ability as follows:

In the process of developing the proposed mathematical competency assessment standard, in each step, there will be specific proposals on methods, techniques and ways to support the development of competency assessment standards in the subject.

Step 1: Analysing Math curriculum in terms of content standards and mathematical literacy requirements

The 2018 Mathematics curriculum has defined mathematical competence as consisting of five components: mathematical thinking and reasoning competence; mathematical modeling competence; mathematical problem-solving competence; mathematical communication competence; ability to use mathematical learning tools and means.

Step 2: Develop behavioral indicators.

In this step, each defined behavioral indicator needs to be observable and measurable, as evidence of what the students can do, say, create, write. The 2018 Mathematics curriculum has requirements for each competency component, these requirements are set for students after the end of each level of education (primary, lower secondary and upper secondary school). For example, for the component of “mathematical thinking and reasoning”, the requirements to achieve at the end of primary school are as follows:

These can be seen as manifestations of the mathematical thinking and reasoning competency component for elementary school. However, the following problems can be seen:

• Descriptions of requirements for each competency component remain highly general.
• Descriptions of the requirements required for each competency component are not tied to specific mathematical contents. Here, in the case of this competency component, it can be seen that statements about the requirements to be achieved are for students at the end of grade 5 but there is almost no cohesion with the corresponding Mathematics content of grade 5.
• There are no descriptions of the requirements to meet for each competency component for students in classes other than seniors.

From the above-mentioned issues, it will be difficult to measure and evaluate the achievement of these requirements. Therefore, this is the most important step in the process of developing competency assessment standards. Because only when a system of requirements to be achieved/expressed specifically for each competency component is in place, then assessment and teaching activities towards capacity development will be effective. The approach for this step is suggested through the following model:

According to this model, based on the requirements to be met on the educational content of each respective grade level in each subject/field of study/educational activity stipulated in the curriculum and based on the requirements to be met (at the end of each education level) to make statements about the corresponding competency assessment standards for each that grade block. In each standard statement, this competency assessment will include verbs expressing the level of achievement of the requirements for each competency component and associated with the statement of the requirements to be achieved in terms of the corresponding educational content.

Step 3. Develop quality criteria

At this step, the determination of quality criteria each behavioral indicator aims to distinguish different levels of quality of actions and actions performed. Typically, when writing behavioral indicators as well as quality criteria, we use corresponding verbs that reflect different levels of attainment in terms of the specific expression of competence of interest. For each statement about the competency assessment standard in the above step, the corresponding level for that statement can be determined, and other levels such as exceeding or below that standard can be given.

Step 4. Consult an expert on the competency assessment standards

After having a draft of competency assessment standards, experts will be invited to read and comment. Typically, invited specialists will be experts in mathematics, specialists in psychology, specialists in educational measurement and evaluation. After expert comments, the standard will be finalized again before implementing the next step.

Step 5. Instrument design

This is the stage to build a system of instruments to measure the competency assessment standards. In this case, the instruments are six different tests. The test development process will be described in the next section.

Step 6. Try out in practice

Piloting can be done through testing the respective instruments. By piloting these instruments, depending on the extent and scale of the standard formulation, the test data can be used for one or more of the following purposes: (1) adapt the statements of the assessment standard; (2) identify thresholds and build capacity development paths of interest; (3) adapt the instruments for future uses.

Step 7. Revising assessment standards

On the basis of analyzing the data obtained from the test results, select the unsuitable items for adjustment in the standard set. From there, design appropriate questions for the toolkit.

2.2 Research results

2.2.1. Mathematical competence standards

Based on the proposed requirement of the mathematical competence in the new curriculum, a framework of numeracy skills was developed including five above mentioned components and four main strands of mathematics knowledge that were defined in the curriculum. Within each strand, a set of indicators were identified to measure the competency of understanding and using relevant knowledge to solve problems within that strand of mathematics as well as in the contexts of real-life situations. Especially, in each domain, based on the requirements in both current and new mathematics curriculum for each grade level, a set of detailed indicators of students’ mathematical competence was developed, each indicator measured a specific aspect of mathematical competence. This set of indicators was reviewed by experts on mathematics education and mathematics teachers. This has been a complicated process including the participation of many researchers and teachers. At the end of the process, 90 indicators of mathematical competence have been identified. Following table shows examples of standards for assessing mathematical competence in the topic ” flat shapes and cubes ” in Grade 3 Mathematics:

Table 1. standards for assessing mathematical competence in the topic ” flat shapes and cubes ” in Grade 3 Mathematics

In the table above, the first and second columns are kept the same as in the 2018 General Education Program in Mathematics. The third column is specific to the requirements to be met that the program has stated in the second column, describing clearly define the activities/manipulations that students must be able to do, the units of knowledge that students must acquire corresponding to the manifestations of mathematical competence. This column is considered a standard for assessing mathematical competency.

2.2.2. Math Item Development process

Based on 90 proposed indicators of mathematical competence, 90 items have been developed to measure these 90 indicators. At first, 90 items were carefully drafted by 2 researchers. Then, 10 mathematics teachers were invited to review and to give feedback on the quality of the items and whether each items measuring respective indicator correctly. After finalising, these items were used to design six tests. Each tests included 30 items, and each pair of tests had 15 common items. One of the reasons using common items was to put all items in a common scale so that the comparison of item difficulties can be made.

Example for matrix of Test 1

Table 2. Matrix of Test 1

Each question can assess one or more components of mathematical competence. Depending on the content and the way the question is asked, each question can help to make a clearer assessment of a certain element of mathematical competence.

Example of questions assessing elements of mathematical competence in grade 3:

– Questions about mathematical thinking and reasoning:

Sort the numbers 40 302; 40 032; 47600, 47599 in order from smallest to largest

– Math modeling assessment questions:

– Math problem solving assessment questions:

– Assessment questions about mathematical communication:

(See in PDF File)

– Evaluation questions about using tools:

Use e ke to draw a right angle and name it.

Thus, based on the requirements to be met in the curriculum to determine standards for assessing mathematical competency. From descriptions of mathematical competency assessment standards, teachers can build a system of assessment questions corresponding to each element of competency.

2.2.3 Participants

After developing the set of mathematical competency standards, experts who are working on the field of Mathematics Education and ten mathematics teachers in Primary schools were invited to review the content of Mathematical competency standards. Designing an instrument of six tests from 90 items of Grade 3 Math competency standards. In the phase of piloting, there were 383 Grade 3 students who participated in taking the tests. The convenient method of sampling was used in this study.

2.2.4. Pilot Data Analysis

The data of 383 Grade 3 students was used for data analysis item response theory through R (R Core Team, 2021) software with TAM package was used for data analysis process. Rasch model with dichotomous score items was used to analyse the pilot data (Masters, 1982) to examine the instruments’ reliability and validity. Data for all six tests was combined. Using the method of concurent equating (Von Davier, 2011), the data provided the information on indices such as fit statistics  and item difficulty to examine the quality of the standards as well as the test items.

The results show that there are 16 out of 90 items with poor fit indices, as shown in Table 2.

Table 3. Items with poor fit indices.

The questions that create irrationality can be caused by the following reasons: Many questions in the above table ask about measurement such as: temperature (item 23), watch time on a clock, number of days in a month (Item 45; item 80), relationship between measurement units, measurement conversion (item 27, Item 56; item 25). These contents are quite familiar, students often already know them in everyday life, so they can easily do them. Or there are questions that students do not follow correctly and still find the correct results (item 9: apply the relationship between addition and subtraction in calculation practice); …

In terms of item difficulties, the range of item difficulties varies from -4.36 to 2.20. The results show that there are some easy items in the test. Table 3 shows the information on the easy items.

Table 4. Information about easy items

The item-person mapping also provide the information relating to the relationships between students’ abilities and item difficulties. It can be seen that many items were easy to students. The above information helps researchers to examine the quality of indicators of mathematical literacy standards and to improve the quality of the set of items.

Figure 1. Item-person variable map (See in PDF File)

According to the map above, some questions that are considered to have a very low level of difficulty are Item3, Item47, Item61, Item63, Item65. These questions are described in the standard set as follows:

         Table 5. The standard for assessing math competency of some questions is too easy

From the table above, it can be seen that the above questions are all about numbers and calculations. These are types of exercises that students are very familiar with and know how to do in lower grades.

2.2.5. Adjust and complete:

Research shows that the reason why students can easily answer the above questions is because the Math curriculum is designed in a concentric direction, developing according to number cycles. In 1st and 2nd grade students have learned about numbers and operations within 1000. By 3rd grade, students learn numbers and operations within 10 000 and then continue to learn numbers and operations in range 100 000. Students are therefore very familiar with these types of questions in the lower grades. Furthermore, many students in real life have known or used numbers in the range of 100 000 before learning about this content in 3rd grade.

With the reasons analyzed above, we can adjust as follows: For questions that are too easy like item 3 and item 47, it may not be necessary to check; with easy questions, but to test the knowledge that first appeared in grade 3 (such as items 61, 63), it is still necessary to test because this is the minimum requirement in the program; For questions where the test skill is so familiar that students know how to do it, such as a test that compares two numbers in the range of 100 000 (item65), it is still necessary to test but lower the difficulty rating of the ability. Mathematical skills from level 2 to level 1.

Thus, the initial test results show that the assessment tool set is based on the standard descriptions of assessing the mathematical ability of 3rd grade students with a fairly appropriate level of assessment. Some questions are too easy compared to the expected level of difficulty. These questions need to be adjusted to an appropriate level of difficulty. The 2018 Primary School Math program has arranged the learning of natural numbers in many number cycles, specifically numbers in the range 10, 20, 100, 1000; 10 000; 100 000, numbers up to class million. Arranging in such number cycles may help students use numbers and perform four operations better, but there may also be skills that students have mastered that still require too much practice. This can be boring for students and a waste of time.

3. CONCLUSION

On the basis of learning experiences on standards of competency assessment in teaching Mathematics of some countries and Vietnam’s Primary Mathematics Curriculum issued in 2018, this study has presented the standard development process. assessment of Mathematical competence for students in grade 3. From that set of standards, illustrative design of a tool for assessing mathematical competence for students. Experiments show that the design and use of the 3rd grade math competency assessment toolkit according to the research process is a feasible initial step. Designing standards for assessing mathematical competence in Math helps teachers organize Math teaching in a more convenient way to develop competency. Teachers can use this assessment standard as a basis to build tools to assess students’ mathematical ability in the teaching process. Moreover, the math competency assessment standard is also the fulcrum for compiling and evaluating textbooks and other materials in Mathematics.

4. ACKNOWLEDGEMENT

This research was funded by the Vietnam Ministry of Education and Training under grant number CT2022.10.VKG.04.

REFERENCES

1. Australian Curriculum Assessment and Reporting Authority. (2016). The National Assessment Program – Literacy and Numeracy (NAPLAN) Assessments. https://www.acara.edu.au/assessment/naplan.
2. Australian Curriculum Assessment and Reporting Authority. (2014). Australian Curriculum. Australian Curriculum, Assessment and Reporting Authority,.
3. Balt, M., Fritz, A., & Ehlert, A. (2020). Insights Into First Grade Students’ Development of Conceptual Numerical Understanding as Drawn From Progression-Based Assessments. Frontiers in Education, 5(80). https://doi.org/10.3389/feduc.2020.00080
4. Callingham, R., Watson, J., & Oates, G. (2019). Statistics and probability: From research to the classroom. In D. Siemon, T. Barkatsas, & R. Seah (Eds.), Researching and Using Progressions (Trajectories) in Mathematics Education (pp. 181–204). Brill Sense.
5. Faragher, R., & Brown, R. I. (2005). Numeracy for Adults with Down Syndrome: It’s a Matter of Quality of Life. Journal of Intellectual Disability Research, 49(10), 761–765. https://doi.org/10.1111/j.1365-2788.2005.00747.x 
6. Griffin, P., & Care, E. (2015). Assessment and Teaching of 21st Century Skills: Methods and Approach. Springer.
7. Heritage, M. (2008). Learning Progressions: Supporting Instruction and Formative Assessment. National Center for Research on Evaluation, Standards and Student Testing.
8. Kim, E. M., Haberstroh, J., Peters, S., Howell, H., & Nabors Ol´ah, L. (2017). A Learning progression for geometrical measurement in one, two, and three dimensions. ETS Research Report Series, 2017(1), 1–26. https://doi.org/10.1002/ets2.12189
9. Organisation for Economic Co-operation and Development. (2012). Literacy, Numeracy, and Problem Solving in Technology-Rich Environments: Framework for the OECD Survey of Adult Skills. OECD Publishing. http://www.oecd-ilibrary.org/education/literacy-numeracyand-problem-solving-in-technology-rich-environments_9789264128859-en.
10. Strickland, J., Woods, K., & Pavlovic, M. (2016). Assessing and Understanding Early Numeracy for Students with Additional Learning Needs [Paper presentation]. AARE Conference 2016.
11. Vietnam Ministry of Education and Training. (2014). Vietnam National Assessment Report for Grade 5. Vietnam Ministry of Education and Training.
12. Vietnam Ministry of Education and Training. (2018). Chương trình Giáo dục Phổ thông môn Toán [General Education Curriculum – Mathematics]. Vietnam Ministry of Education and Training.