8. CONCLUSION
5.5 RESEARCH SUB-QUESTION 3: WHAT IS THE IMPACT OF THE ASSESSMENT FEEDBACK
5.5.1 F EEDBACK AS AN INTEGRAL PART OF TVET C URRICULUM A SSESSMENT
180
Combined Staff and Student ratings are higher than student self –ratings. Student self-rating Minimum score is 77/128 – 60.1% and therefore 8.9% lower than the combined score. The student maximum self-rating at 105.8/128-82.6% is 15.6% lower than the combined rating.
This can be indicative of students with a well-developed self-awareness portrayed in the highest social competence dimension average measured in this survey – 14/16 (87.5%). The lowest combined score was achieved in the Social Competence Dimension, Self-regulating Conduct – 12.9/16 - (80.6%); Student self-rating score for this dimension is 12.3/16 – 76.8%.
This is a high-risk area in terms of social competence required in the workplace given the increased importance of interconnectedness, co-existence and teamwork as critical 21st century competences as indicated in Chapter Two-2.2.
5.5 Research Sub-Question 3: What is the impact of the assessment Feedback structure
181
Tab. 29: Feedback criteria and graphic illustration of responses for TVET Curriculum Assessment
Assessment Feedback criteria
Current TVET Curriculum Rater responses graphically illustrated
1. Assessment results are thoroughly discussed with students regarding assessment tasks done at the College/Institution.
2. Results of practical tasks are thoroughly discussed with students regarding assessment tasks done in the workplace centre/workshop.
3. Challenges are discussed and intervention strategies are provided after every assessment.
0 0
67%
33%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 6%
61%
33%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 5%
78%
17%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
182 4. Students’ strengths are acknowledged and discussed.
5. Students are given the opportunity to share their point of view with regard to the assessment results.
6. Students receive feedback less than one week
after the assessment. 0 22%
61%
17%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 5%
78%
17%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 11%
67%
22%
Strongly Disagree
Disagree Agree Stronly Agree Rater Responses
183
0
33%
50%
17%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
7. Students receive feedback more than one month after the assessment.
8. Students receive specific help (developmental programmes) with the work they struggled with in the assessment tasks.
9. The current assessment and feedback that students receive, stimulates creative and innovative thinking strategies towards finding effective and efficient solutions to address socio-economic challenges.
0
50%
39%
11%
Strongly
Disagree Disagree Agree Strongly Agree Rater Repsonses
44%
28% 28%
0 Strongly
Disagree
Disagree Agree Strongly Agree Rater Responses
184 10. Students are thoroughly prepared and equipped to perform well in their occupation in the workplace as a result of effective feedback, which helped them improve their competence in finding better solutions.
11. Students will be able to apply the theory of their course with ease in the practical part of their job in the real workplace, as a result of feedback with regard to the transfer of theory to practice.
12. Students experience the feedback environment as positive: a good learning and development opportunity to help them take
charge of their development and learning. 0 11%
78%
11%
Stronly
Disagree Disagree Agree Strongly Agree Rater Responses
6% 17%
72%
5%
Strongly Disagree
Disagree Agree Stronly Agree Rater Responses
0% 12%
82%
6%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
185 13. Assessment and feedback processes help me as a Lecturer/Supervisor with the ongoing improvement of the quality of my training/service delivery to students.
14. Students clearly understand the feedback and specific suggestions they receive from me, as a Lecturer/Instructor, and know which steps to take to improve their performance by finding new ways of thinking about problem solving and finding holistic solutions.
15. The current assessment system does not allow for quality, detailed feedback to students and feedback occurs predominantly in the form of a static statement of results (report/test scores).
The responses on the Curriculum Feedback provided by TVET staff involved in COMET assessment, rating and development processes is indicative of their perception of Feedback.
The predominantly positive responses regarding Feedback do not correlate with the overall
0 6%
55%
39%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
0 0
94%
6%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
5.50%
50%
39%
5.50%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
186
performance of students in the COMET Large-Scale assessment. There are various indicators of confusion and contradiction:
Challenges are discussed and intervention strategies are provided after every assessment. - 78% - Agree and 17% Strongly Agree versus Students receive specific help (developmental programmes) with the work they struggled with in the assessment tasks – 50% indicated Disagree.
The current assessment system does not allow for quality, detailed feedback to students and feedback occurs predominantly in the form of a static statement of results (report/test scores) – 39 % Agree and Strongly Agree, 5.5%.
The current assessment and feedback that students receive, stimulates creative and innovative thinking strategies towards finding effective and efficient solutions to address socio-economic challenges – 50% Agree; 17% Strongly Agree and 33% Disagree. The occupational competence levels achieved by students in the COMET Large Scale assessment based on their problem solving skills differ significantly – Nominal – 60.4 %; Functional - 14, 9%;
Processual- 13, 1% and Holistic Occupational Competence- 11, 4%.
Very few Strongly Agree nor Strongly Disagree responses were provided which allude to not completely adhering to the Feedback criteria but only to a certain extent.
These discrepancies can be an indication of a poor perception of the science of Feedback as an integral part of Assessment and Development of Occupational competence and commitment.
The second section of the Feedback questionnaire focus specifically on feedback related to COMET. COMET Raters have been exposed to the COMET feedback process.
\
187 Tab. 30: Rater responses to COMET assessment criteria
COMET Assessment Feedback Criteria
Rater Responses
1. The open test tasks assessed according to COMET criteria allows for more specific feedback actions.
2. Inter-rater reliability as a first step in preparation for COMET assessment equips staff with improved competence to provide quality feedback to students.
3. The rating scale, using the eight COMET competence criteria assists Staff with the self-evaluation of their work, hence strengthening their professional development to provide more accurate feedback to students.
6% 6%
66%
22%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
0 6%
47% 47%
Strongly
Disagree Disagree Agree Stronly Agree Rater Responses
6% 6%
59%
29%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
188 4. The typical character of COMET
where a solution space is provided, describing possible solutions and variants associated with the open test tasks improve Staff confidence and competence for the feedback process.
5. A high quality feedback process will assist in preparing students to be able to solve complex problems holistically in the place of work.
6. Occupational identity is interrelated to occupational commitment and can be shaped by a specific, directive and empowering reflection on assessment results.
7. A Dual education and training programme, where students learn theory (academic learning) and workplace practice (vocational learning) is the best solution for developing self-evaluation and reflective feedback among students.
6% 0
76%
18%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 6% 35%
59%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 0
69%
31%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 12% 12%
76%
Strongly Disagree Disagree Agree Strongly Agree Rater Responses
189 8. Current Government TVET
policies and strategies place sufficient emphasis on feedback as a process to inform effective and efficient, industry related education and training.
9. Work Process Knowledge – know that (guide action), know how (interpretation of action) and know why (reflective action) are valuable levels to interactively discuss assessment outcomes with students with regard to their levels of occupational competence as well as development needs.
10. Decisions on how to improve teaching and learning rely heavily on previous assessment outcomes and feedback reports.
11. Feedback is often seen as an add-on to assessment and not an integral part of assessment.
0
33%
50%
17%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
0 6%
59%
35%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
6% 6%
59%
29%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0
47%
35%
18%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
190 12. COMET feedback reports
provide a clear, graphic illustration to students on their performance, assisting in self-evaluation and reflection.
13. A TVET qualification, as outlined in the current curriculum, is sufficient to prepare students to solve problems holistically and therefore competent to succeed in the world of work.
14. Feedback results are captured and well documented for future reference.
15. COMET feedback can assist the students to be actively involved in shaping their progress from being a novice (beginner) to an expert (skilled) in their respective occupations.
6% 12%
53%
29%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
12%
47%
23% 18%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
12%
29%
35%
24%
Strongly Disagree
Disagree Agree Strongly Agree Rater Responses
0 6%
59%
35%
Strongly
Disagree Disagree Agree Strongly Agree Rater Responses
191
The possibility of COMET having a positive impact on Feedback is clearly illustrated by the responses of the Raters. There is a significant higher level of decisiveness in comparison to the first section on Feedback within the current Curriculum.
The impact of the COMET Open-ended Test Tasks to allow for specific feedback action is valued at 66%-Agree and 22% Strongly Agree level. This rating signals the need for assessment based on diverse, holistic problem solving skills.
High quality Staff training such as inter –rater reliability training provided to COMET Raters seems indispensable for quality, meaningful Assessment feedback to students according to 47% -Agree and 47% - Strongly Agree ratings.
76% of Raters Strongly Agree to a Dual Education and Training programme as essential for self-evaluation and reflective feedback. 59% indicated that the TVET qualification as outlined in the current curriculum does not adequately prepare students for the world of work. 33%
alluded to Government TVET policies and strategies not emphasising Feedback sufficiently to an extent where it will inform industry related education and training while 67% reported that it is sufficiently emphasised. The impact of the positive ratings regarding Government Assessment policies is however not reflected in the student occupational competence levels measured in this study.
94 % of the Raters see the solution space provided with COMET open-ended test tasks as a method to improve Staff confidence and competence. This is a principle feature in the development of effective and efficient Assessment Feedback strategies, procedures and principles.
Raters (94%) acknowledge that the COMET Feedback process can assist students to be actively involved in their journey from being a Novice (Beginner) to an Expert (Skilled worker) in their occupations - Agree – 59% as well as Strongly Agree-35%.
The results yielded by the Feedback questionnaire affirm the position of Feedback as an integral part of Assessment and Development of TVET Students. It is however clear that whilst the acknowledgement of the importance of Feedback, the process need to be established and situated in TVET.
192
5.5.2 SEMI-STRUCTURED FOCUS GROUP INTERVIEWS - FEEDBACK AND