MONITORING THE CODE

This is an area about which there is also some uncertainty, but it is important enough to warrant a separate section. While Section 6.6 looks at grievance

procedures and complaints, the present section draws on interview and other data to evaluate what is happening, and what needs to happen at the level of the

institution.

6.4.1 Internal Monitoring

First, it is important to recall that the Code requires a system of internal monitoring (section 7.1 relating to information, and section 23.3 relating to compliance with

When asked about this area, all institutions were very comfortable with the notion of internal monitoring on an annual basis. There was, however, an interesting range of responses about institutional practice.

• PTEs generally noted that they needed to review all of their programmes and systems annually, given a similar review process by NZQA. They saw the internal monitoring as something which would be (or already was), part of this review process.

• Similarly, several state schools and tertiary institutions noted that they also had a systematic annual review process, and that the monitoring of the Code and information for IS would again be a natural extension of this process.

• Several other state schools and two state tertiary providers were aware that a process needed to be put in place, but had not yet made any decisions.

One provider stated that with changes likely to the Code, they wanted to wait and see what might be required.

• Two providers had already put in place a special review process which was reportedly quite different from other review and/or quality assurance

systems. In each of these cases, a special committee had been

established, involving either Board of Trustee representation (for a school), or student representation (for a tertiary institution).

The Evaluation Team see considerable merit in a special process being established. For secondary and tertiary institutions, a review panel might well consist of the following:

• The international director/manager;

• A Board or Council representative;

• At least one senior IS student representative;

• Where appropriate, a homestay or boarding establishment representative;

• A member of the senior management team (principal, international dean);

• A teacher thoroughly familiar with IS needs.

This is not to deny what a number of those interviewed strongly asserted, that internal monitoring is "constant", or "on-going". Even when this is the case, the Evaluation Team believes that an annual stand-alone process is to be

commended.

6.4.2 External Monitoring

This area was one of the most interesting in the whole data collection process.

Perhaps because the Code Administrator, responsible under the Code for setting up a system of external monitoring (section 23.1), has been seriously occupied with an on-going stream of Code applications, or perhaps because individual institutions are not yet aware of systems already being put in place, the response to questions about such monitoring again offered a wide range of responses, with very few clear patterns.

First, institutions were asked how often they should be externally monitored, and by whom.

The PTE sector were almost unanimous here, supporting the notion that the monitoring be conducted as part of the annual NZQA process. Most did note that they would also expect that NZQA would need people who were sufficiently experienced to know what questions to ask.

The latter point was a unanimous response - in fact the only unanimous response - in the state sector. All commented in some form or another that they wanted

"specialists". Comments like, "They need to be up-to-date", "They need to really know what questions to ask", or "They should get people who have extensive and recent experience" were common (cf. Peddie et al., 2003, on regional advisers).

What was definitely not unanimous among state (and one or two private) providers was the frequency of external monitoring. Responses of between two and five

changes were needed; and, in just one case, "If we get it wrong, the students won't come".

Finally, the 'agency' to undertake the external monitoring was again an area where there were no clear patterns (apart from the PTEs - see above). Some felt that

"The Ministry gets the levy, so they should do it". Others were closer to the private sector. As one school interviewee commented, "ERO review us now. Why add anything different? Just make sure they have a specialist in the field". A third group believed it should be either "A specialist team", or "A person who is currently working in the field and who therefore has the specialist knowledge required".

Whenever the idea of a specialist regional adviser was suggested, there was always a caveat; "As long as they really know what happens in [schools]".

The Evaluation Team believes that external monitoring systems need to be put in place as soon as possible. Furthermore, the agency is arguably less important than the system. All institutions need to be provided with a list of the kinds of information that need to be supplied to the external reviewers. The proposals of the Code review that internal monitoring data be in a form available to the Code Administrator and to external reviewers is thus to be applauded. The strong desire for 'expertise' is also an important issue. As argued elsewhere (Peddie et al., 2003), experience and expertise are important, but the notion that an external reviewer needs to be someone who has current IS programme experience - as in, 'I was there yesterday' - is not supported.

Overall, however, the Evaluation Team concludes that external monitoring needs to meet (at least) the following:

• It should be rigorous, with well-publicised criteria, standards and data collection procedures, developed with full consultation with the industry, but these criteria may well vary among different sectors;

• It should be conducted by people who are thoroughly familiar with the Code and with the needs of IS, desirably but not necessarily by those who have extensive practical IS experience;

• The monitors may be part of a regular review team, or specially constituted - there is no apparent need for a unitary system;

• External monitoring should take place at least every two years, but should be immediate when a signatory is subject to more than, say, two

substantiated complaints to the IEAA within a period of six months.