Our Ref: 9604.00
QUALITY IMPROVEMENT?
AN EVALUATION OF THE IMPLEMENTATION OF THE CODE OF PRACTICE FOR THE PASTORAL
CARE OF INTERNATIONAL STUDENTS (2002-2003)
FINAL REPORT
July 2003
AUCKLAND UNISERVICES LIMITED A wholly owned company of THE UNIVERSITY OF AUCKLAND
Prepared for
International Policy Development Unit The (NZ) Ministry of Education
PO Box 1666 Wellington
Consultants Roger Peddie Marilyn Lewis Gary Barkhuizen C/- School of Education The University of Auckland
Reports from Auckland UniServices Limited should only be used for the purposes for which they were commissioned. If it is proposed to use a report prepared by Auckland UniServices Limited for a different purpose or in a different context from that intended at the time of commissioning the work, then UniServices should be consulted to verify whether the report is being correctly interpreted. In particular it is requested that, where quoted, conclusions given in UniServices reports should be stated in full.
ACKNOWLEDGEMENTS
A large number of people made this evaluation possible. The Evaluation Team would like to offer special thanks to Christine Scott and Jan Hollway, each of whom was interviewed on several occasions, and who speedily responded to phone messages and e-mails. Second, our thanks to the regional facilitators, who welcomed us to the professional development sessions we observed, and in most cases gave us useful information and ideas in interviews. Third, our thanks to the Advisory Group who made several very useful suggestions in their meetings.
The Ministry of Education staff involved in the two evaluations have offered us information, reports, advance warning of changes in policy, and have been particularly open in sharing their views and thoughts. We have very much appreciated this, and would like to thank Neil Scotts and Emily Fabling from the International Policy Unit, and Jim Sinclair, David Beer and Sarah Hart from the Code Administrator's office. Thanks also to Vanessa James, who was contracted to be centrally involved in several aspects of the Code development.
Nick Lewis and Andrew Butcher offered some very thoughtful readings and ideas.
A number of staff in the International Office of the University of Auckland gave us valuable information in the early stages of the evaluation. Other academic
colleagues proved to be a useful sounding board as the project developed, particularly Roger Dale.
Last, but by no means least, we wish to thank all of those who freely gave of their time and expertise in the interviews. The typical response was, "Let us know which day you want to come, and we'll organise the right people for you". The friendly help of a number of administrative staff needs to be added to this list; they were often the ones who organised times, downloaded documentation, and told us where to park…
To all of the above, we have appreciated your help; without you, this report would not exist.
ABSTRACT
This final report summarises the results of a13-month review of the
implementation of the (2002) mandatory Code of Practice relating to full fee-paying international students (IS) in New Zealand institutions. The report covers a
number of areas common to a parallel report evaluating two professional
development (PD) initiatives relating to IS programmes which is reporting on the same date (Peddie et al., 2003).
After clarifying the 'events' of the past 18 months, and the methodology used for the evaluation, the report focuses first on the application procedures to become a signatory to the Code. This section reports on some issues which are closely linked to the PD report. It then considers a number of issues which were either the direct result of the implementation of the Code, are still unresolved. The
evaluation also looks at internal and external monitoring procedures and at the early outcomes of grievance procedures, especially as they have impacted on the International Education Appeal Authority (IEAA).
Overall, the evaluation provides data which show that the introduction of the mandatory Code has, to date, had a positive effect on the export education industry in New Zealand. This conclusion must, however, be regarded as
tentative, because new legislation passed in December, 2002 (Education (Tertiary Reform) Amendment Act, 2002), and the subsequent very significant Code review process from March to July, 2003 have meant that what is reported on in this evaluation cannot offer a totally firm blueprint for future developments.
The report nevertheless concludes with a number of recommendations relating to future Code needs and possibilities, and to future evaluation and research.
TABLE OF CONTENTS
Acknowledgements i
Abstract ii
Table of Contents iii
List of Abbreviations iv
Executive Summary v
List of Recommendations xi
Section One: Introduction 1
Section Two Timeline of Events 6
Section Three Methodology 13
Section Four Analysis: Content, Context and Mechanisms 23 Section Five The Code: Becoming a Signatory 29 Section Six The Code: Effects on Institutions and Students 36 Section Seven Conclusions and Recommendations 53 References
Appendices:
Appendix One Samples of Data Collection Material Appendix Two Reports Submitted to the Ministry
Appendix Three E-mailed Submission on Primary Students Appendix Four Key Initiatives (from Export Education) Appendix Five Advisory Group
61 64
LIST OF ABBREVIATIONS/SHORT VERSIONS
APPEL ATSA
Association of Private Providers of English Language Aotearoa Tertiary Students' Association
(the) Code The mandatory Code of Practice for the Pastoral Care of International Students. Ministry of Education, 2002.
ERO Education Review Office Export
Education Export Education in New Zealand: A Strategic Approach to Developing the Sector. Ministry of Education, 2001.
IEAA International Education Appeal Authority IS Full fee-paying international student
ISANA ISANA International Education Association Ministry Ministry of Education
NZAPEP NZIS
New Zealand Association of Private Education Providers New Zealand Immigration Service
NZQA New Zealand Qualifications Authority
PD Professional Development
PRC PTE RFP TDO Tertiary
Peoples' Republic of China Private training establishment
Request for Proposal (Ministry of Education document) Tertiary Development Officer
Refers to all post-school institutions (degree-granting or not) (T)ESOL (Teaching) English to speakers of other languages
EXECUTIVE SUMMARY
Introduction
In 2001 the Government released Export Education: A Strategic Approach (Ministry of Education, 2001a, 2001b). One of the "Key Initiatives" in this document was the development of a mandatory code of practice aimed at institutions with full fee-paying international students (IS). The Code of Practice for the Pastoral Care of International Students was subsequently developed, Gazetted and then published (Ministry of Education, 2002a). In mid-2002 a team from the University of Auckland was contracted through UniServices Ltd to
evaluate the implementation of the Code to mid-2003. The team was already evaluating two professional development (PD) initiatives relating to IS
programmes. While the two evaluations were 'aligned' in late 2002, mainly because the focus of the PD initiatives was increasingly on aspects of the Code, two separate reports were requested, and have been submitted (cf. Peddie et al., 2003).
Goals of the Evaluation
It is recognised by both the Ministry and the Evaluation Team that the original reporting date of June 30, 2003 meant that some aspects of the implementation of the Code would not have had sufficient time to allow for valid and reliable
conclusions – especially in terms of the appeals procedures.
Nevertheless, there were some important issues which can be (at least initially) evaluated before the end of June, 2003.
The focus of this evaluation was to assess whether the introduction of a mandatory Code was making a positive contribution towards the quality of the educational experience of IS in New Zealand educational institutions. Also, is the Code making a positive contribution to the government policy set out in Export Education?
Primarily, the evaluation aimed to determine:
• What (if any) issues arose from the process undergone by all types of educational institutions applying to be signatories to the Code;
• What positive changes have so far arisen from the introduction of the Code in terms of the broad areas of:
1. Marketing
2. Orientation programmes
3. Programming/teaching-learning 4. Pastoral care
5. Homestay/accommodation provisions
6. Other factors associated with the internationalisation of institutions and their communities
There were a number of more specific evaluation questions related to these broad aims. Section One of the full report details these and offers other introductory comments.
Timeline of Events
The evaluation period began about half-way through the six months allowed (under the Code's transitional arrangements), for institutions to become
signatories. Section Two of the report details a number of 'events' affecting the two linked evaluations. Of critical importance to this evaluation were:
• The date by which institutions had to be signatories to the mandatory Code (30September, 2002);
• New legislation enabling the payment of a new "Levy" for each IS, affecting the definition of an "international student" and as a consequence requiring the registration of (mainly) language schools offering courses of less than three months (December, 2002);
• A subsequent process to revise the Code, both to ensure that it conformed with the new legislation, and to reconsider a number of areas covered by the version operating in 2002 (March to July, 2003).
Methodology
The evaluation used a form of "realist evaluation", which is outlined in Section Three. In brief, this includes consideration of:
"Content", what the intervention - the Code - entails;
"Context", the (political, economic, and other factors affecting the intervention;
"Mechanisms", the attitudes and beliefs of those participating in the intervention;
"Outcomes", but the discussion of outcomes intrinsically includes the processes used.
The procedures used were almost completely linked to the PD evaluation. Even before the Code evaluation contract was signed, much of the data-gathering had related to the Code, while subsequent interviews and other data-gathering 'visits' always covered both evaluations.
In all, over 80 data-gathering visits were made. These included:
• Thirty-eight visits to institutions across sectors and across New Zealand, in which 68 individuals supplied data face-to-face (or in a few cases, by questionnaire);
• Attendance at 13 PD seminars, in most of which material related to the Code figured prominently, and a good deal of informal discussion was also possible;
• A number of other interviews with specialists in various areas, and attendance at several seminars/other events relating to IS.
The vast majority of the institutions visited were purposively selected as representing good practice, either because their Code applications were considered "exemplary", or because regional PD facilitators identified them as worthy of a visit.
Content, Context and Mechanisms
Section Four analyses the key aspects of the intervention content, the context in which the intervention of the Code implementation took place, and the enabling and disabling 'mechanisms' (attitudes and beliefs) held by those in institutions.
By way of summary, the content was the introduction of a mandatory Code, with a fixed time frame, associated costs, and a substantial application process. The context included the policy contained in Export Education, the recent rapid rise in IS numbers against a background of global trends in this and related areas, the neo-liberal approach adopted by successive governments in recent years, funding issues, and factors such as the SARS virus and the strong Kiwi dollar.
The mechanisms were a mix of positive (enabling) and negative (disabling). The former emerged in terms of the professional and genuine desire of almost all institutions to ensure quality programmes. The latter included resentment over a centrally-imposed Code with considerable time and money compliance
consequences, and some more naïve beliefs about the future - which could be characterised as, "There's no problem, they'll just keep coming".
Becoming a Signatory
The evaluation found that the process of becoming a signatory was a largely, but definitely not always, positive experience. Some primary schools with very few IS in particular resented the need to formalise in policies and other paper
requirements what they thought were already good internal processes for IS. But, while the process did tend to be time-consuming, close to half of the respondents stated that being required both to review their current policies and to develop what they recognised as necessary new ones was a valuable exercise. But another large group believed that approved templates of the more "technical" policies relating to such matters as fees remission, indemnity and grievance policies
should have been available on the Ministry's website from the start of the process.
The evaluation found that both the PD programmes and the work of the Code Administrator's office made very positive contributions to the process by which 940 Code signatory applications had been processed by the 'closing date' of 30
September, 2002. It might be noted that by the end of August, 2003, this number will have exceeded 1300.
The Code: Effects on Institutions and Students
Section Six covers quite a wide range of issues emerging from the evaluation.
While the full report contains the details, the following represents the key features of this section of the evaluation.
First, interview questions asked about each of the major areas of potential change as a result of the introduction of the Code (cf. Goals of the Evaluation, above).
The data suggest that 'good' institutions were affirmed in the policies they already had, while others made positive changes. Not unexpectedly, the area of teaching and learning was least affected, as the Code does not specifically address
standards in this area.
Next, some "unresolved issues" are noted and briefly discussed. These were:
• An on-going concern that one Code cannot cover the differential needs of so many different sectors and programmes (a view the evaluators reject);
• A number of specific issues addressed to the Code Administrator - but many of these should be resolved by the revised Code and the new Guidelines being developed;
• The roles of other Government agencies, especially that of the Immigration Services;
• The mixed impact of the PD and the Code on regional organizations - some responded very positively, a few reportedly did not;
• The relatively frequent lack of a strategic or business plan relating to IS, notably in the school sector, but also found elsewhere;
• The need for Code provisions to be better understood by non-specialist staff, both teachers and general staff;
• Clearer information about what needs to be provided to the Code Administrator if policy and/or other changes occur within a signatory institution.
There were, next a variety of responses about how the Code should be monitored, both internally on an annual basis (as the Code requires) and externally (through a system to be developed by the Code Administrator). Very few clear patterns
emerged, especially in terms of external monitoring - though the private sector saw it as something that could easily be accommodated through their current annual NZQA reviews, provided specialist staff were involved.
The effect of the Code on complaints to the International Education Appeal Authority (IEAA) was also investigated. While the number of complaints since October has more than trebled over the number received in the year 2001-2002, the evaluators agree with the IEAA and the Code Administrator that this is very likely to be because of the vastly increased publicity effected by the Code about grievance procedures. But the evaluators warn that such a trend cannot be allowed to continue, and that appropriate changes need to be made when there are several complaints about an institution.
Finally, the Ministry's handling of the Code review process to date is praised, for the 'extra' (to Code requirements) consultation offered, and the responsiveness to particular sectoral concerns.
Conclusion
Section Seven summarises some of the key issues noted here, and offers a series of recommendations. These are summarised in the next section of the report.
Overall, however, the evaluators consider that the introduction of the mandatory code has had a positive effect on programmes for IS in all sectors and throughout New Zealand.
SUMMARY OF RECOMMENDATIONS1
Recommendation One: That the introduction of a mandatory Code of Practice by the Ministry is to be commended as a very positive move in the light of the current state of IS programmes in New Zealand.
Recommendation Two: That the Code be regarded as a 'living document', and that the Code Administrator (currently, the Ministry), make it clear that necessary and/or desirable improvements/changes are likely to continue to occur; but that such changes will typically not require significant/new
application procedures on the part of signatories.
Recommendation Three: That the Code Administrator urgently develop a mandatory set of procedures relating to the requirement of signatories to notify the Administrator of changes to the original Code application, and/or conditions under which a new application must be submitted; and that a new owner/CEO of a PTE be required to accept the status of the previous Code application, or lodge a new one.
Recommendation Four: That the Ministry is to be commended for the decision to develop a comprehensive set of 'guidelines' to accompany the revised Code of Practice, and is encouraged to make this guide a 'living document', with best practice ideas inserted as they are provided.
Recommendation Five: That (currently) a specific site on the Ministry's
International website be devoted to a regularly (e.g. monthly) updated listing of 'best practice' suggestions relating to IS programmes; and that relevant organisations, institutions and industry groups be encouraged to support such a site.
1 Explanations and discussion of these recommendations is found in Section 7.3.
Recommendation Six: That the Ministry develop a website protocol which clearly signals a 'best practice' link; and that the Ministry investigate ways of encouraging providers to visit their website on a regular basis (at least once a month) to see what new best practice and other IS-related ideas are available.
Recommendation Seven: That the Ministry investigate the possibility of adding to their International website relevant information from other Ministries, so that providers could access such information without searching through other sites.
Recommendation Eight: That the Ministry continue to manage the office of the Code Administrator for at least the next two to three years.
Recommendation Nine: That the Ministry urgently commission an evaluation of the implementation period and (at least) the first year of operation of the revised Code.
Recommendation Ten: That a comprehensive plan of research relevant to the quality provision of programmes for IS be urgently developed, and that priority research be commissioned as soon as possible.2
SECTION ONE INTRODUCTION
1.1 BACKGROUND
In mid-April 2002, the Ministry of Education (henceforth, the Ministry), called for tenders for an evaluation of the implementation of the mandatory Code of Practice for the Pastoral Care of International Students (the Code) (Ministry of Education, 2002). As noted in Section Two, the Code was published in January, 2002 and was due to come into effect in March, but transition arrangements included an allowance six months before the provisions actually became mandatory.
The focus of the evaluation was primarily to look at the implementation and the effect of the Code on the provision of programmes for international students (IS), but also to comment on the contribution of the Code to the broader policy of Export Education in New Zealand. A Strategic Approach to Developing the Sector
(Ministry of Education, 2001a, 2001b).
A team from The University of Auckland, through UniServices Ltd, was contracted in mid-2002 to carry out an independent external evaluation of the Code
implementation, and to report back to the Ministry, initially by June 30, 2003. The same team had previously won a contract to evaluate two professional
development (PD) initiatives carried out in 2002 (and later slightly extended into 2003). In late 2002, the two projects were 'aligned' with data collection and the reporting dates made identical. Finally, the Evaluation Team successfully
requested a further month, with both reports to be delivered by July 31, 2003. This was due mainly to the occurrence of several unanticipated events in the last few months of the Evaluations (see Section Two).
Members of the Evaluation team are: Roger Peddie (Team Leader), Marilyn Lewis and Gary Barkhuizen, with occasional help from a series of Research Assistants.
1.2 OVERVIEW OF THIS REPORT
This final report is titled "Quality Improvement?" as the introduction of a mandatory Code is noted under that heading in the Key Initiatives section of Export Education (Ministry of Education 2001a, 2001b). A central issue, therefore, is whether the introduction and implementation of the Code has indeed improved the quality of programmes for IS in New Zealand.
The report is divided into eight sections. The introduction gives a general background to and overview of the report, and then lists the goals and research questions of the evaluation. Section Two presents a timeline of the 'events' relating to the two aligned evaluations - the Code, and the two PD initiatives.
Section Three introduces the conceptual model used in the Evaluations ("Realist Evaluation") and then details the procedures used to collect the data. Section Four gives an analysis, in terms of that model, of what may have affected the success or otherwise of the Code implementation, noting as appropriate any strengths and/or limitations in the 'content' of the Code, or the 'context' and 'mechanisms' which inevitably interacted with certain aspects of the Code, especially given the review process in 2003.3
Sections Five offers a closer look at the data relating to the transition period and the Code application process, while Section Six looks at the effect the Code had on providers who successfully became signatories. It also looks at the effect on students in terms of grievance procedures and the impact on International
Education Appeal Authority (IEAA), and several 'unresolved issues'. It concludes with a brief comment on the Code review.
Section Seven summarises the main findings of the Evaluation, then presents a series of recommendations relating to the Code. The report offers some
suggestions for future research, and ends with a few closing remarks.
Following the text of the report, the Appendices contain the following.
• A list of instruments used to gather data for the two linked evaluations, and copies of a selection of these instruments.4
• A list of reports submitted during the two linked evaluations.
• An originally emailed submission (June, 2003) from the Team Leader on IS in the primary sector.
• The Key Initiatives for 2001-2002 from Export Education in New Zealand (Ministry of Education, 2001a, 2001b). This is taken from the Ministry's website and is a slightly reformatted version of the section in the Overview document.
• Information about the Advisory Group for the two evaluations described in Section 1.1.
1.3 EVALUATION AIMS AND RESEARCH QUESTIONS
The RFP identified major aims and associated research questions about which the evaluation should seek informative data. These aims and questions were as follows.
The Code and international students
1. Do international students know about and understand the Code? How do they know about it? Have they referenced the Code or used the IEAA or Review Panel for any reason?
2. Has the Code provided satisfactory means of redress against providers by
international students (from both a student and provider perspective)? Have there been examples of complaints that could not be adequately dealt with by the IEAA/Review Panel? Are the sanctions available under the Code adequate?
4 Only a selection is provided, as there is a good deal of overlap among the forms used.
3. Has the implementation of the Code changed the standard of care for international students?
The Code and signatories
4. What has been the practical impact of the Code on the organisation of international student programmes by providers (including costs)?
4.1. Over a range of institution types, from primary/secondary schools, public tertiary institutions and private training establishments;
4.2. By size (i.e. international student population);
4.3. Across institutions of different socio-economic status and ethnic make-up.
5. How effectively have providers implemented the Code, on a Part by Part basis?
6. Have there been any changes to participation by institutions that can be linked to the Code? (ie. has the Code enabled or encouraged smaller and/or regional institutions to develop or expand their international student programmes or, conversely, has it discouraged those institutions)?
7. How helpful to the sector is the support offered by the Ministry? (e.g. workshops to assist signatories with implementation of the Code and the attestation process) 8. In light of the evaluation, what recommendations can be made in relation to the Code itself, to processes surrounding the Code and its implementation and for further policy and guidelines development in areas affecting the pastoral care of international students?
The Code and accommodation provision
9. What has been the impact on availability and other aspects of homestay accommodation as a result of more stringent procedures introduced under the Code, including the associated Homestay Guidelines?
(homestay, boarding establishment, living with a designated caregiver, and living with parents)?
The Code, the IEAA and the Review Panel
11. In light of the information on complaints to the IEAA and Review Panel and the feedback from students on accessibility and knowledge of the IEAA and Review Panel, what recommendations (if any) can be made in relation to the IEAA and Review Panel structure and processes?
(Request for Proposal) As is commonly the case, the proposal accepted by the Ministry warned that in some cases only indicative answers might be able to be supplied. When data collection was completed, it was found that the answers to most of the research questions could be provided, but that the advance warning was a wise one.
Unanswered questions emerged because of the 'context' limitations (cf. Section Two), the introduction of a substantial Code review process, and/or because the timeline for data collection sometimes did not permit a reliable answer. For these reasons, and unlike the PD report (Peddie et al., 2003), no question by question summary of answers is provided in Section Seven.
SECTION TWO TIMELINE OF EVENTS
2.1 INTRODUCTION
Before looking at either the methodology or the outcomes of the evaluation, it is useful to include a short summary of the 'events' of the past 19-20 months.
Precise dates and details are not always included in this summary. Rather, this section is provided to describe and in part explain why some things happened the way they did, and why it became increasingly obvious that the two previously separate evaluations (of PD initiatives and of the implementation of the Code), would benefit from being 'aligned'. The alignment resulted in common data- gathering procedures, and a common reporting date. In a number of cases, a commentary on or discussion of these 'events' will be covered in later chapters of this report, or in the parallel chapters of the final report on the PD evaluation (Peddie et al., 2003).
2.2 2001
In 2001, Export Education in New Zealand: A Strategic Approach to Developing the Sector, was released as a full report and in 'overview' format (Ministry of Education, 2001a, 2001b), and money was made available through Vote
Education to implement a number of the strategies this document outlined. By late 2001, Scott Strategic had completed a contract which involved a needs analysis for PD in the (at that stage secondary) schools, and the development of a set of materials which could be used to help deliver such PD. When a 'Request for Proposal' (RFP) to deliver PD for schools was duly sent out by the Ministry, Scott Strategic successfully bid for the contract. At a meeting in December, 2001, Scott Strategic discovered that the contracting agency also required PD for primary
(mainly) regional offices and associations to identify at least two well-qualified and suitable people who could act as facilitators for the delivery of the PD. In the case of the vast region covered by Education Wellington International (EWI), six
facilitators were contracted.5
At about the same time, a Request for Proposal was sent out for the conduct of a needs analysis for PD in the tertiary sector. This contract was won by Education New Zealand, who subsequently contracted Jan Hollway as Tertiary Development Officer (TDO) to carry out the work. Jan had a strong managerial background both academically and in practice. She had developed and managed a sizeable and successful private training establishment (PTE) in Hawkes Bay. She then moved to Auckland (while retaining ownership of the PTE), was completing an MBA, and was interested to take further a new role as an education consultant.
Also late in 2001, a Request for Proposal was sent out for an evaluation of these two PD initiatives. This, as noted earlier, was contracted to the University of Auckland team through UniServices Ltd. Further work was also begun/carried out by other contractors. Professor Colleen Ward of Victoria University of Wellington had completed a very useful literature review on the impact of IS on domestic students (Ward, 2001), and in 2002 she was contracted to develop a wide-ranging questionnaire to be administered to IS at all levels except primary, originally in the second half of the year. That questionnaire was delayed, but a large sample of students have been surveyed in mid-2003, and analysis of the results is currently proceeding.
2.3 2002
In January, 2002, the new (mandatory) Code of Practice for the Pastoral Care of International Students (the Code) was published, after it had been initially
Gazetted in December, 2001. The Code was to come into effect on 31 March, 2002. However, section 24 of the Code (Transition Arrangements) allowed
5 The region included Wellington, Manawatu-Wanganui, Taranaki, Wairarapa-Bush and Hawkes Bay.
providers six months, to the end of September, 2002, to become signatories to the Code before its provisions did in fact become mandatory. The 2002 Code
replaced the previous voluntary Code of Practice for the Recruitment, Welfare and Support of International Students, which had been introduced in 1996.
In February, 2002, the (PD) Evaluation Team began its work with observation of most of the initial training of the facilitators. The Team subsequently negotiated with Scott Strategic to observe a range of the first and second round (follow-up) seminars held around the country over the next few months (cf. Section 3.2.3). It rapidly became apparent that most of the attendees at these seminars increasingly focused on the requirements which would allow them to become signatories to the Code (cf. Section Five).
Over a period of three months in mid-2002, the Ministry arranged for the TDO to offer a series of PD seminars aimed at new providers. This was in part to ensure that such providers were fully aware of the requirements of the Code, whether they were or were not already signatories (cf. Section Five; also Peddie et al., 2003:
Section 6.3).
In mid-2002, a further Request for Proposal was sent out by the Ministry, this time for an evaluation of the implementation of the Code. The RFP did not mention a fixed time frame, given the differing dates of the 'implementation', but it did note that the final report (originally) due on June 30, 2003, would in fact be an "interim"
report in terms of some aspects of the proposal. This gave a clear - and in the view of the Evaluation Team very sensible - indication that an evaluation process would need to continue in some form. The same University of Auckland-based Evaluation Team was successful in winning this contract.
From this point forward, it became increasingly obvious that any data collection for the PD contract also involved the Code contract, and vice versa. This was
accepted in late 2002 both by the Ministry and by the University of Auckland
what was in fact one interview about a common area of interest and allowed such documents to be redeveloped to cover both contracts (cf. Appendix One).
In mid-December, 2002, a further event impacted on the evaluations. The Tertiary Education Reform Bill finally became law (Education (Tertiary Reform)
Amendment Act, 2002). This had three major effects. First, the definition of an
"international student" was altered by the removal of the necessity for such students to hold a current student visa or permit. This in turn meant that an overseas person on a visitor's permit/visa, who was studying at an educational institution offering courses of less than three months, was now classified as an
"international student". Second, the provisions of the Act meant that a number of such institutions who had previously not required New Zealand Qualification Authority (NZQA) registration and/or accreditation for relevant courses now had to fall in line. Third, groups visiting educational institutions of all kinds for 'short visits' were now potentially covered by the Code.
It was clear that, as well as providing a significant challenge to NZQA, this Act necessitated modification of the Code. Not unexpectedly, the Ministry decided to use this opportunity to consider changes in other aspects of the Code at the same time.
Late in 2002, the Code Administrator contracted Scott Strategic to offer a further (limited) series of PD seminars. These were originally targeted at primary schools with IS who had not attended earlier PD, new primary school providers, and teachers of English for speakers of other languages (ESOL). The last category proved to be unworkable, in the main because many ESOL teachers were preoccupied by several major controversies about their roles, practices and conditions. Secondary school-based or linked homestay personnel were then added to the list as a replacement group. At the same time, Scott Strategic were contracted to make any necessary changes or additions and then to put all of their PD materials onto the Ministry website. The time frame for all of these additional tasks to be completed was early 2003.
Also in late 2002, the final reporting on the tertiary needs analysis was
unexpectedly held up. While some comments on this process are presented in the companion report (Peddie et al, 2003: Section Six), here it may simply be reported that the full report prepared by the TDO was not initially accepted by her
Reference Group, and that a subsequent rewrite has to date (July, 2003) not been released. Instead an (extended) Executive Summary was called for and
eventually released in February, 2003 for a limited period of time through the Education New Zealand website.6 In practice, this would have meant that a final report on the PD contract would have been impossible until some time in March, 2003.
2.4 EARLY/MID 2003
By the end of March, 2003, the additional PD seminars facilitated by Scott
Strategic had been completed, and the Executive Summary of the Hollway Report had been released. The next three months were very important on two counts.
First, the Ministry consulted widely regarding an arguably major review of the Code. Second, the Evaluation Team carried out a large number of case study visits and interviews.
The visits and interviews form part of the data reported on in Sections Five to Seven. On the other hand, the Code review process is the focus of what follows.
As noted earlier, following the passing of the new legislation in December, 2002, the Code had to be revised. The Ministry took this opportunity to include both the mandatory changes and to reconsider several other major features. The former included consideration of both courses currently offered by unregistered providers and a wide range of educational and other institutions offering 'courses' to people from overseas. While the legislation appeared to be quite clear-cut, the
interpretations were not. 'Courses' involving a half-day introduction to kayaking for
tourists needed to be differentiated from 'short courses' in ESOL lasting 12 weeks and leading to other 'courses' offered immediately afterwards.
Moreover, short courses offered by signatory providers also included a very wide range of situations. Short visits of an educational nature to a school or college of education, for example, are very different from those of visiting sporting groups who are home-stayed for two or three nights. These issues were looked at by the Ministry and (especially) by NZQA, and also formed part of the interviewing
process for this evaluation.
The Ministry had also embarked on a commendable process. While the 2002 Code (Section 26) required 20 working days of notification to providers about Code changes, the Ministry planned and effectuated two rounds of consultation in the period April-June, 2003. This process began with the release of a Discussion Document (Ministry of Education, 2003b), developed during March and
promulgated in early April. This resulted in 312 submissions, including 93 from the primary sector. An analysis of these submissions was released in early May, (Ministry of Education, 2003a), followed by the promulgation of a Draft revised Code in late May (Ministry of Education, 2003c). This included both a general introductory comment, and annotations about the proposed changes.
Approximately 60 further submissions were made, and the final version of the revised Code is expected in late July. This is linked to the fact that the new definitions of an IS come into force in July. It should be added that from early 2003, thanks to close cooperation with staff in the Ministry, possible key factors emerging in the Code revision process had already been incorporated into interview and meeting schedules taking place well before the release of the Discussion Document.
A significant number of the submissions to the Discussion Document related to proposals to make major changes to provisions for students under 13 years of age, namely that such students be living with a parent or court-appointed guardian. These proposals arose from concerns expressed to the Minister of Education in 2002, and from a contracted study subsequently carried out in mid- 2002 on a small sample of primary schools with IS. While the previewed possible
changes relating to students under 13 did not appear in the first draft of the revised Code, it should be noted that the Ministry has since carried out further
investigations in this area, and a Government decision is expected in late July or early August.
It should also be noted that, among others (TESOL, Quality Assurance, Research, Schools and Tertiary PD), an Advisory Group has been established by the Ministry to help develop a comprehensive set of (revised) Code Guidelines. As part of that process, the Evaluation Team were asked to - and did - provide a short paper listing and explaining recommendations relating to the proposals of the revised Code and the Team Leader supplied an extended e-mail submission relating to primary-aged IS (cf. Appendix Three). The Evaluation Team also supplied a report on possible future national and other provisions of PD.
Finally, the Education Review Office (ERO) had been requested in 2002 by the Minister to investigate a number of aspects of the impact of IS in schools. ERO gathered data from 94 schools in Term Four, 2002 and Term One, 2003. They released their report on the investigation in June, 2003 (Education Review Office, 2003). While a few points from this report are incorporated in the current
evaluations, these have been limited to issues directly related to the research questions. Nevertheless, relevant sections of the ERO report should later be carefully examined in the light of the results of the Ward student survey when it is eventually released.
SECTION THREE METHODOLOGY
3.1 OVERVIEW: 'REALIST EVALUATION'
The methodology for this evaluation, and the companion evaluation of the Professional Development initiatives (Peddie et al., 2003), is adapted from the
‘realist evaluation for evidence-based practice’ developed by British evaluators whose initial focus areas were often new programmes in social work (Pawson &
Tilley, 1997; Kazi, 2000; Kazi & Spurling, 2000). This approach draws from the earlier work of Michael Scriven (1994), with his notions of the three ‘boxes’, black, grey and white.
Scriven argued that evaluations which dealt only with outcomes treated the
processes which led to those outcomes as a 'black box', not open to scrutiny, and presumed to be unimportant. While this allows policy makers to assess success or failure, it offers very little by way of constructive suggestions in the latter case.
The “grey box” approach attempts to examine process, at least in part. A “white box” approach was seen as the ideal. This would be an evaluation which
holistically examines process and outcomes, and the relationships between them (Scriven, 1994).
Realist evaluation builds from this last approach, adding the notion of “contexts in practice” (Kazi & Spurling, 2000). In terms of social work it has been defined as addressing “all significant variables involved in…practice, through a realist
effectiveness cycle which links the models of intervention with the circumstances in which practice takes place.” (Kazi, 2000). While the “cycle” of evaluation was possible only to a limited extent in these current evaluations, this was the reason, for example, for the evolving pattern of interview questions as the evaluations proceeded.
One of the most useful features of realist evaluation is the categorisation of data collection methods into content, context, mechanisms and outcomes (Kazi &
Spurling, 2000: 8). For the current evaluations, these may be defined as follows.
• Content: the intervention or programme; here, it was the introduction of a mandatory Code, together with the requirements to become a signatory and later, the December 2002 legislation which changed the definition of an
“international student” (Education (Tertiary Reform) Amendment Act, 2002).
In the PD evaluation it was both the programme provided by Scott Strategic, and the needs analysis by Jan Hollway for Education New Zealand. For the purposes of both evaluations it is considered essential that both the time frame for the contracts, and finance available be seen as important aspects of the 'content'.
• Context: the situational factors which both pre-exist at the time of
intervention, and which develop during the intervention process. These are factors affecting the people and the institutions participating in the PD initiatives, and not those affecting the contractors and facilitators.
• Mechanisms: these are the motivations, practices (and sometimes goals), of the individuals and institutions who are the recipients of the intervention and/or the programme implementation. These mechanisms can be
enabling (of the intervention or programme), or disabling, and can arise out of reactions to the content or context, or from an interaction between both.
• Outcomes: the results of the intervention or programme. This is often seen as the major focus for funding agencies, but that is not the case when further funding also depends on how successful the particular processes have been in achieving these outcomes.
3.2 PROCEDURES
In what follows, the term 'visits' is used as a neutral term to include a wide range of activities: attendance at professional development (PD) seminars, interviews in educational institutions, a range of other interviews, and several further types of (face-to-face) data collection.
3.2.1 Activities / Visits
The original proposals accepted by the Ministry included 20 visits in each contract.
When the two contracts were 'combined' late in 2002, one of the arguments for this was that the combination would allow for more extensive data-gathering. A further argument was that the third round of PD, which took place in late 2002 and early 2003, could be a free add-on to the PD contract.
Both of these arguments were accepted, and both have been vindicated in practice. While a total of 40 visits were proposed (20 for each contract), by the end of June 2003 more than 80 visits constitute the key data base for the final reports. This was in part made possible following a decision not to survey
institutions by questionnaire, especially as the schools' PD had been surveyed by an evaluator working with Scott Strategic.
The sample has been a mixture of 'opportunistic', as in the unexpected chance to gather information at a recent two-day seminar in Nelson, and purposive, to
ensure a suitable range of both institutions, and meetings with key personnel. The Evaluation Team has also taken very seriously the following selection criteria for institutional visits:
• A wide geographical coverage;
• Coverage of all types/sectors involved;
• Coverage of institutions which are at very different levels of experience;
• The need to identify best practice in a wide range of areas.
This led to an initial purposive selection, initiated by the Team Leader with the help of the Code Administrator's office. A request was made for the Office to select two or three 'exemplary' Code applications for each sector. This was done on the basis that subsequent visits would be able to verify whether institutional practice was indeed that which was represented by the Code application. Despite
concerns expressed by some members of the Advisory Group about the reliability of such an approach, the Evaluation Team believed - and still believes - that it was a useful starting point.
After subsequently receiving 14 exemplary applications, it was found that these gave a fairly representative geographical sample, with Auckland figuring strongly, but with a number of other regions also appearing.
The procedure then followed was to contact one of the regional facilitators who had provided PD and to ask them for recommendations of highly regarded programmes in institutions other than the 'exemplary' sector. So, for example, a visit was to a location where the exemplary application was a secondary school, this might result in a primary school and a tertiary (public or private) institution also being visited.
This has clearly produced a biased sample, but this was a deliberate strategy.
The Evaluation Team saw little merit in visiting 'bad' institutions when the clear goal of the PD, the Code and the evaluations was to help to raise the standards.
On the other hand, it must be strongly emphasised that the findings of this part of the evaluation cannot be assumed to be 'typical' of institutions with IS. This was in part countered for the school sector, however, by a number of informal discussions with a wide range of experience and school background at the PD seminars
attended by members of the Evaluation Team.
The summary which follows indicates the nature of visits undertaken for the two linked evaluations.
3.2.2 Summary of Visits: Institutions
It was determined early in the evaluations to identify the industry as comprising seven main 'categories' of institution with IS programmes. These were primary and intermediate schools, secondary schools (including those with primary classes), polytechnics, universities, language schools (both private and state), private training establishments (PTEs) other than language schools (including both degree-granting and others, and state colleges of education.
Table 3.1 presents a summary of institutional visits for the two Evaluations.
Table 3.1 Visits to Institutions
Type No. of Visits North Is. South Is.
Primary schools 7 5 2
Secondary schools 10 4 6
Polytechnics 5 3 2 Universities 4 3 1
Language schools 6 5 1
PTEs 5 5 0
Colleges of Educn 1 1 0
Totals 38 26 12
As indicated earlier, a starting-point for these visits was the selection by the Code Administrator's office of 14 'exemplary' Code applications, comprising two or three for each sector. Twelve of these were visited. One refused on the grounds that they were "too busy", and in the other both the principal and the international director were new, and the school is currently preparing a completely revised Code application. They were happy to be seen later in the year if required.
The number of people interviewed, and the number of interviews varied from institution to institution. Typically, in primary schools the principal was interviewed, with on three occasions a second person present (in all three cases, the person responsible for ESOL). In secondary schools, the principal was normally
interviewed briefly, and the international director/manager interviewed separately.
In tertiary institutions, there was no clear pattern. In several cases two people were interviewed simultaneously. In four cases only one person was interviewed, but another completed a short questionnaire. In one language school three separate interviews were held, as well as a telephone discussion with the absent owner; in another, only the CEO was interviewed. Overall, 64 people were
interviewed and four senior staff sent in 'context' or other questionnaires. Thus, 68 staff provided data relating to the 38 institutions visited.7
The primary schools visited covered an interesting range of experience and numbers of full fee-paying international students (IS). They included two intermediate schools.
The secondary schools include two private schools, and represent schools with from fewer than 10 IS up to over 100.
The polytechnics include two major urban and three from secondary/rural areas.
One is a very significant player, with a long history of IS programmes The language schools comprise four private institutions and two University- affiliated schools.
The private training establishments include two degree-granting institutions, and three offering other forms of qualification. One currently has no full fee-paying IS, but is a signatory and planning to market for 2004.
The college of education has both IS students in 'standard' programmes, and visiting groups, often senior educators coming for short-term tailored programmes.
3.2.3 Summary of Visits: Professional Development
Table 3.2 presents an overview of professional development 'events' attended by one of the Evaluation Team in 2002 and early 2003.8
7 These figures do not include 'non-standard' interviews with five international staff
Table 3.2
Professional Development Events Attended
'Round' No. of Visits North Is. South Is.
Training 1 (3 days) 1 N/A
First Round 4 3 1
Second 4 2 2
Tertiary 1 0 1
Third 3 2 1
Total 13 8 5
The Training of the facilitators for the schools' PD took place over four days in Wellington, early in 2002. The Team Leader of the Evaluation Team attended three of these days.
The First Round was the series of one-day seminars for schools, covering a wide range of policy and other advice relating to IS programmes.
The Second Round comprised a range of regionally-determined follow-up
initiatives for schools, based on what the (regional) facilitators concluded from the First Round evaluations. In a large number of cases, there was considerable interest in the Code application process.
The Tertiary round was a limited series of eight PD seminars aimed at new providers in the tertiary sector. This programme was delivered by the TDO.
As noted earlier, the Third Round comprised seminars targeted at three distinct groups: primary schools showing as having IS, but who were not yet signatories;
primary schools who had not attended the earlier PD; and homestay coordinators based in or associated with secondary schools. In fact, this did not happen in any 'clean' way. A number of schools simply took this new opportunity to get further information and ideas about the provision of quality programmes for IS.
3.2.4 Other Interviews
Interviews were held with a number of people involved in the PD initiatives, although Code issues were regularly part of the conversation. The 'principal PD actors' - Jan Hollway (the TDO), Lyn Scott and Christine Scott (schools' PD contractors) - were all interviewed. Jan and Christine were interviewed four times each, and there was also a number of e-mail and telephone links with these two people. Lyn Scott was interviewed once. An interview was also held with the evaluator contracted by Scott Strategic to monitor their PD work in 2002, and e- mail contacts were also made with that evaluator. It should be added that all of the raw data collected for the schools' PD (seminar evaluations and
questionnaires), has been made available to the Evaluation Team, along with summary data of evaluations carried out by the TDO.
Interviews of various kinds were also held with nine of the fourteen regional facilitators for the schools' PD. This involved just five extra visits, as two of the interviews involved two regional facilitators participating at the same time, and two have already been counted in the list of institutional visits. Informal discussions with several other facilitators, however, took place in the course of the
observations of schools' PD sessions.
Other interviews which have been held are described below.
• Lester Taylor of Education New Zealand, which was contracted to the Ministry to investigate PD needs in the tertiary sector (Jan Hollway was then contracted to Education New Zealand to be the TDO and to carry out the needs analysis).
• A short-term contract staff member of the Ministry who was directly involved in the PD contracts.
• A brief and early interview with a contract staff member who assisted with the development and writing of the Code of practice, but was also involved
• Professor Colleen Ward, Victoria University of Wellington; a lengthy phone discussion in 2002 and a face-to-face meeting in 2003.
• Four face-to-face interviews, and e-mail/telephone contacts with the Code Administrator, along with phone conversations with Code support staff David Beer (who attended one face-to-face meeting), and Sarah Hart.
• Two meetings with Dr Nick Lewis, who is researching the development of three new industries in New Zealand (wine, education, fashion).
3.2.5 Other Visits
Five other data-gathering visits have been undertaken.
• Attendance at an ISANA regional seminar on Chinese IS, held in Auckland (August, 2002).
• Attendance at (and participation in) a seminar on Asians in Education, held in Auckland (October, 2002).
• Attendance by a contracted agent at the ATSA conference on the needs of IS, held in Nelson in May, 2003
• Informal and on-going discussions with a former9 Chinese doctoral IS, whose thesis is looking at intercultural communication in a period of increasing globalisation and a growing international knowledge economy.
• Participation in a project seeking to analyse IS student movements from Asia to New Zealand in the period 1945 to the present (project under the auspices of the New Zealand Asia Institute).
3.3 CONCLUDING COMMENT
Along with all of the above, and not counting as a 'visit', some direct
correspondence has been conducted by e-mail with Dr John Pickering of ISANA, and by phone and e-mail with Andrew Butcher (now based at the University of Auckland). Informal discussions have also been held with several colleagues at the University of Auckland, notably Professor Roger Dale, who has a keen interest
9 "Former", because she has now gained permanent resident status.
in globalisation and internationalisation. Several very informative meetings of varying length have also occurred between the Evaluation Team Leader and staff of the Ministry's International Policy Unit. Leaving out all of these 'other'
exchanges, a total of more than 80 data-gathering visits were made during the Evaluations. This has led to considerable confidence as to both the validity and reliability of the data presented in this report.
SECTION FOUR
ANALYSIS: CONTEXT, CONTENT AND MECHANISMS
4.1 INTRODUCTION
This section analyses the context, content and mechanisms of the implementation of the Code in terms of the theoretical framework outlined in Section Three.
However it does not detail the processes and outcomes of this initiative. They are presented and discussed in the sections which follow.
Given the closely linked nature of the two Evaluations, much of what follows is common to both, especially in the ‘context ‘ section
4.2 CONTENT
The ‘intervention’, i.e. the introduction and implementation of the mandatory Code can be seen to have the following characteristics.
• It went well beyond what had been established several years earlier in the voluntary Code of 1996.
• It was developed over quite a lengthy period of time, with a good deal of industry consultation and input.
• It offered mainly ‘free’ access to regional assistance, in that the PD
seminars and activities offered by Scott Strategic were free of charge, and both travel and teaching relief were provided when requested; while those offered by the TDO had only a small charge which fell short of covering costs.
• It offered further substantial assistance to institutions through the office of the Code Administrator and the Ministry website.
• Becoming a signatory was bounded by the six months time frame set out in the Code and the money to be paid to the Administrator.
• It was compulsory for all institutions who wished to retain and/or offer places to IS (as defined by the pre-December, 2002 legislation);
• This meant that it required all institutions to make a conscious decision to 'participate' or not.
• The application process was backed up by (initially) a sizeable printed guide, and later, by templates and samples of forms, contracts, etc., on the Ministry’s International website.
• The Code itself was not backed up by any general guidelines, with the exception of a set of guidelines relating to accommodation (Ministry of Education, 2002b).
In terms of the realist evaluation approach, the Code can definitely be seen as a form of programmed intervention. It sought to guarantee a minimum level of quality by a check on providers who already had good systems, by requiring those who did not to develop them, and by seeking to ensure that new providers had all necessary systems in place before taking on IS programmes.
4.3 CONTEXT
The context for all of the IS initiatives is extremely complex. While some aspects of the context are ‘universal’ to all participants - such as the political climate and the ‘international scene’ - other aspects are more sector or institution-specific.
This contextual complexity can be seen as affecting at least two key areas of the two linked evaluations: the mechanisms operating (see below), and the outcomes achieved.
The context of both evaluations involves a number of interacting factors. First and foremost, the publication of the government policy statement Export Education (2001a, 2001b), is a major influence on what is currently being evaluated. But, in another sense, the importance of the government policy may have much less
realisation of its economic importance, and attempted to develop, improve and regulate what was already happening.
This was partly also because of the second factor, the recent very rapid and enormous expansion in the numbers of IS coming to study in New Zealand. This increase was the result of a number of reasons, some of which were on-going, such as increasing internationalisation, the growth of the knowledge economy, and the wider effects of globalisation. Others were 'accidental', or derived from these larger developments, notably the changing political situation in the Peoples' Republic of China (PRC) which led to large numbers of PRC students being able to look for education overseas.
A significant internal factor was the previous legislation which allowed both an institution offering courses of less than three months to be free from NZQA requirements for registration and accreditation, and the opportunity for IS on visitors permits to enrol and study at such institutions.
A third factor is the neo-liberal political climate in New Zealand, which has a central tenet that government should minimise its involvement where industry and the market can (and should) be involved and that, while accountability should be a key requirement, responsibility should rest with providers (Olssen, 2000). The situation in the Ministry at the start of the Evaluations is an intriguing example of this approach. In early 2002, after the economic impact of IS in New Zealand in 2001 had been calculated to be in excess of one billion dollars in overseas currency earnings, the International Policy Unit of the Ministry comprised fewer than three permanent staff.
A fourth and linked factor is that the education sector has for some years been claiming that central funding is increasingly inadequate. Particularly in the public tertiary sector, there has been a growing need for institutions to generate funding through external sources, including contract research, bequests and donations, and through the much higher fees paid by international students. Estimates
involving three universities, for example, suggest that at least 10-15% of their total income is generated from IS fees.
In the course of the Evaluations, a fifth set of factors emerged as a major contextual influence. The outbreak of SARS, and subsequent restrictions on travel, particularly from PRC has had an important impact on New Zealand. This country has a much higher dependency on PRC as a source of students than others (Asia 2000, 2002, Education Review Office, 2003), and so a sharp decline from that country potentially affects New Zealand much more than it does others.
Again, while to date nothing as drastic has yet emerged, the very large
dependence on Korea as a source of IS in primary schools (more than 80% of all students), is a very important contextual factor. Along with these considerations, the New Zealand dollar has recently risen quite sharply against key international currencies, making New Zealand fees potentially less attractive than previously.
4.4 MECHANISMS
The contextual factors are of course closely linked to the mechanisms in the various IS initiatives evaluated in 2002-03. These mechanisms may be summed up as follows.
4.4.1 Enabling Mechanisms
Enabling mechanisms are those which support or bring about positive responses and/or actions relating to the interventions/programmes being evaluated. As noted above, these can be either individual or institutional responses, or both. The enabling mechanisms noted in data collection for the Code evaluation were as follows.10
• A genuine desire to learn more about quality provision for IS programmes and care on the part of a number of school personnel.
• A growing realisation that the Code and the process of becoming a
signatory could provide a number of benchmarks for such quality provision.
• An acceptance by many that the responsible individual and/or the institution had insufficient knowledge about IS students and their personal,
accommodation and programme needs.
• A realisation that the financial benefits of IS fees would be at risk unless quality programmes and care were being offered.
• A realisation that the 'industry' can stand or fall to a marked extent on the performance of the least competent providers.
• (In a few cases) a desire to use the Code as a way of persuading the institution as a whole to improve provisions for IS, particularly in terms of improving the skills and abilities of non-specialist staff.
4.4.2 Disabling Mechanisms
Disabling mechanisms are beliefs and/or policies currently held by individuals and/or institutions which support or bring about negative consequences and/or actions relating to the interventions/programmes being evaluated. The disabling mechanisms noted in the Code evaluation initiatives were as follows.
• An occasional belief on the part of an individual being interviewed that they 'knew it all' and that (sometimes), if only sufficient money were provided from the Levy, there would be no Code problems of any kind for 'their' institution.
• The apparent ignorance of a few providers as to the contextual factors which might drastically affect their programme. These providers seemed to assume that more and more IS would continue to come to New Zealand, and that all they needed to do was to expand their current IS capacity.11
• An anti-Ministry/central-control attitude, which was either connected to individual experience, or to a more general belief that the industry could do it, and why were government interfering by imposing a mandatory Code?
• Annoyance at financial factors linked to the Code implementation process, including both the Levy and various compliance costs.
11 This was certainly much less evident as tertiary institutions realised that new student numbers in 2003 were dropping, reportedly due to the SARS virus, the greater strength of the Kiwi dollar and greater competitiveness from other countries.
• An (assumed) attitude on the part of some primary school principals with only one or two IS a year, that giving up a full day or more for PD just to find out about the Code (especially when travel was involved), was not
sufficiently important for them to attend.
4.4 PROCESS AND OUTCOMES
These are detailed and discussed in subsequent sections.
SECTION FIVE
THE CODE: BECOMING A SIGNATORY
5.1 INTRODUCTION
As noted in Section Two, the 'initial' process of current IS providers becoming signatories to the (mandatory) Code took place in the period from late March to the end of September, 2002. Since that time, a number of late applications were received, and now there is a steady trickle of new providers, that is, institutions which are only now offering programmes to IS.
This section evaluates the process of becoming a signatory through interviews with the Code Administrator, and data from interviews in institutions. It does not, however, seek to replicate the detailed reports supplied by the Code Administrator to the Operations Division of the Ministry.
5.2 THE APPLICATION PROCESS: INSTITUTIONAL PERSPECTIVES
Early indications, as reported in Milestone One (pp4-5), were that there were two quite polar positions regarding the application process. First, that some
institutions believed that a full set of templates should have been available at the same time as the application material, to avoid what one principal noted as "an inordinate amount of time" in formulating from what she believed were already very good practices a set of documents that would satisfy the Code Administrator.
Second, other institutions believed that, while the application process did take quite a lot of time, it was a valuable exercise, ensuring that policies and
formulations of practice were both Code-compliant and appropriate for the institution.
When the institutional interviews and observations of PD seminars had been completed, the analysis of comments about the process did not in fact support the
