Evaluating Computer-Supported Social Learning: An Integrative Study of the Development of Soft Skills in a University-Based Mediation Training Programme

Evaluating Computer-Supported Social Learning:

An Integrative Study of the Development of Soft Skills

in a University-Based Mediation Training Programme

Doctoral Dissertation

submitted in fulfilment of the requirements for the degree

„Doktor der Philosophie“ (Dr. phil. / Ph.D.)

to the

Universität Hamburg / University of Hamburg, Germany,

Fakultät für Erziehungswissenschaft, Psychologie und Bewegungswissenschaft

(Faculty of Education, Psychology and Human Movement),

Fachbereich Psychologie / Department of Psychology

presented by Jens Joachim Rogmann

Halstenbek, Germany 2008

In accordance with §§8 and 9 of the Promotionsordnung (Doctoral Degree Regulations) of August 20, 2003, this document had been submitted for review and approval by the

Department of Psychology’s appointed

Promotionsprüfungsausschuss / Doctoral Degree Examination Committee

Prof. Dr. Reinhold Schwab

(Prüfungsausschussvorsitzender / Head of the Examination Committee)

Prof. Dr. R. Alexander Redlich

(Betreuer und Dissertationsgutachter / Advisor and Thesis Reviewer)

Prof. Dr. Friedemann Schulz von Thun (Dissertationsgutachter / Thesis Reviewer)

Prof. Dr. Matthias Burisch

(Disputationsgutachter / Disputation Examiner)

Prof. Dr. Torsten Meyer

Fb Erziehungswissenschaften, Juniorprofessor für Erziehungswissenschaft unter besonderer Berücksichtigung der Forschung und Lehre im Bereich Multimedia

(Disputationsgutachter / Disputation Examiner)

Disputationsdatum/Date of Disputation: 31-03-2008

© Jens Joachim Rogmann Hamburg, Germany, 2008

3

Evaluating Computer-Supported Social Learning:

An Integrative Study of the Development of Soft Skills

in a University-Based Mediation Training Programme

5

Acknowledgements

As happy as I am now, just having finished the concluding sentences of this dissertation, I also feel grateful to the many without whom this thesis would have never seen the light of day.

First and foremost, I would like to thank Prof. Dr. R. Alexander Redlich for giving me the opportunity to evaluate and contribute to the Department of Psychology's PROCON curriculum. He has supported and encouraged and educated me in many more ways than I could possibly state here. Yet, if, humbly, I had to single out one of these many ways for all of which I am most grateful, it would be the opportunity to repeatedly experience how he, based on compassionate and humane principles, proactively and frankly approached and re-conceptualized difficult situations while involving his counterparts (or, if put in a more spiritual term, his neighbours) in an honest and trustworthy dialogue. It has taught me again that, and the end of the day, what really counts is Menschlichkeit.

In a similar way, I owe much gratitude to my family for their love and understanding which have proved so essential over all these years. Katja, my beloved wife, and my mother Helga have both encouraged me and have provided unstinting, generous support in both word and deed. Thank you so much for your empathy and all tangible help (which does mean to include child-minding, financial contributions, and also my gracious brother Jan's seemingly unwearied acceptance of extra duties in our family business.) And, Lina, my beloved daughter, thank you for the joyfulness and happiness you have brought to our lives.

Furthermore, I would like to express my thankfulness to my colleague and friend Sascha Kilburg for most enlightening discussions and his help in advancing CSSL theory, coursewares and practice, to Alexandra Bielecke for all her kind contributions and skilful assistance in running the PROCON curriculum, and to the other members of the Beratung und

Training work group for their supportive supervision and making me feel part of a team. I am

also grateful to the members of my Doctoral Degree Examination Committee, Prof. Dr. Friedemann Schulz von Thun, Prof. Dr. Matthias Burisch, Prof. Dr. Torsten Meyer, and Prof. Dr. Reinhold Schwab from the University of Hamburg whom I had the honour to learn from and work with in various fields, ways and contexts.

peer-tutors for their time, thoughts, and encouragement. Without your remarkable interest in the subject matter, your cooperativeness in contributing, and your honest feed-backs, this project would lack its empirical grounding.

7

Abstract

There is a growing recognition that graduates from institutions of higher education need to be equipped with higher levels of certified social skills. Recent innovations in advanced learning technologies have provided opportunities for enhancing traditional delivery modes of planned social skill development. While current mainstream research and practice in this area explores approaches to computer-supported collaborative learning (CSCL), an additional framework is put forward in this study.

Computer Supported Social Learning (CSSL) draws on Social Learning Theory and various models for social competence, its development and assessment. CSSL essentially focuses on individual learners and aims at scaffolding situated conceptual knowledge to improve socio-communicative competence. For this purpose, computer technologies provide individual learners with video-based scenes and interactive tasks pertaining to complex and critical inter-personal situations.

In this study, the impact of the introduction and use of tailored CSSL-based coursewares for mediation was evaluated in a university-based setting. Blending traditionally delivered simulation training with preparational use of the coursewares was expected to positively effect learning and resulting levels of socio-communicative competence required by mediating third parties in group-based conflict-resolution.

Across 195 participating students from four successive yearly cohorts, courseware use was found to be associated with superior post-curricular situational judgement. No consistent significant associations were found between CSSL courseware use and post-test levels of conceptual knowledge, interest in subject matter, or self-efficacy.

Potential problems of the statistical, theoretical, and internal validity as well as the generality of the findings are discussed, and ideas for theory, future research and practice are explored.

9

Table of Contents

List of TablesList of Tables

List of Tables ...151515 15 List of Figures

List of FiguresList of Figures

List of Figures...17...1717 17 1.

1.1.

1. IntroductionIntroductionIntroductionIntroduction...21...2121 21

1.1 1.1 1.1

1.1 Mediation Training as Soft Skills Development in Higher EducationMediation Training as Soft Skills Development in Higher Education ...Mediation Training as Soft Skills Development in Higher EducationMediation Training as Soft Skills Development in Higher Education... 23..232323

Methods employed in social skills training... 24

Mediation training as “soft skills” development. ... 24

The need for evaluative research. ... 25

No evaluation tradition... 26

Vocational assessment of practitioners... 26

Foci of mediation research. ... 26

Funding of mediation research. ... 27

1.2 1.2 1.2 1.2 Social Skills Training: ‘FaceSocial Skills Training: ‘Face----toSocial Skills Training: ‘FaceSocial Skills Training: ‘Facetoto----face’ and ‘Blended’ Deliverytoface’ and ‘Blended’ Delivery ...face’ and ‘Blended’ Deliveryface’ and ‘Blended’ Delivery... 28282828 Potential advantages of “blending”. ... 29

Social skill development may require situated, direct interaction... 30

“Blending” as an emerging trend. ... 31

Preliminary Research Outcome... 33

2. 2.2. 2. Theoretical BackgroundTheoretical BackgroundTheoretical BackgroundTheoretical Background...37...3737 37 2.1 2.1 2.1 2.1 The Nature of SocioThe Nature of Socio----Communicative CompetenceThe Nature of SocioThe Nature of SocioCommunicative CompetenceCommunicative Competence ...Communicative Competence... 39393939 Social competence as Socio-communicative competence. ... 39

Multidimensionality in individual pre-requisites... 40

Direct and Effective Interaction. ... 43

Situational Appraisal. ... 44

Situation and Contextual Conditions. ... 45

Intrapersonal Dynamic Processes... 46

Handling and Behavioural Response. ... 48

Socio-communicative competence: A working definition. ... 49

Socio-communicative competence in Mediation... 50

Socio-communicative Competence and Mediation... 54

Everyday Life... 56

Social and Emotional Intelligence Testing. ... 56

Situational Judgement Testing... 58

Performance-Based Assessment. ... 60

Tapping Internal Contingencies... 62

Portfolio Assessment... 63

Assessment Purposes and Applicability... 64

Approaches to Technology-Supported or “Blended” Assessment. ... 65

2.3 2.32.3 2.3 Perspectives on the Development of SocioPerspectives on the Development of SocioPerspectives on the Development of SocioPerspectives on the Development of Socio----Communicative CompetenceCommunicative CompetenceCommunicative Competence...Communicative Competence... 70...707070 Major Perspectives on Learning and Instruction. ... 70

Behaviourist-Empiricist Conceptualisation... 71

Cognitivist-Rationalist Conceptualization. ... 72

Excursus: Transmission Models... 73

Constructivist Conceptualisations... 76

Situative Conceptualizations... 83

Conclusions. ... 86

2.4 2.42.4 2.4 A Framework Model of SoA Framework Model of SoA Framework Model of SocioA Framework Model of Sociociocio----Communicative Competence DevelopmentCommunicative Competence Development ...Communicative Competence DevelopmentCommunicative Competence Development... 87878787 Components of an Integrative Theoretical Framework... 87

Enactive Learning... 88

Observational Learning. ... 90

SCT/SLT as a Meta-Theoretical Base... 93

Framework Integration and Links to the Praxis of Training... 94

2.5 2.52.5 2.5 ComputerComputerComputerComputer----Supported Social Learning (CSSL)Supported Social Learning (CSSL)Supported Social Learning (CSSL) ...Supported Social Learning (CSSL)... 97...979797 Defining Computer-Supported Learning. ... 97

The 3C Model. ... 98

Computer-Supported Social Learning (CSSL). ... 99

2.6 2.62.6 2.6 Potential Outcomes of ComputerPotential Outcomes of ComputerPotential Outcomes of Computer----Supported Social LearningPotential Outcomes of ComputerSupported Social LearningSupported Social Learning...Supported Social Learning... 106...106106106 Performance. ... 106

Post-Training Attitudes. ... 107

Further Outcome Variables. ... 109

2.7 2.72.7 2.7 Evaluating ComputerEvaluating ComputerEvaluating Computer----Supported Social LearningEvaluating ComputerSupported Social LearningSupported Social LearningSupported Social Learning... 110110110110 Programme, not Product. ... 110

Evaluation, not Research... 111

Type of Evaluation. ... 113

Problems of Goal-Based Approaches to Evaluation... 114

Alternatives. ... 116

The Role of Computer-Supported Learning. ... 117

“Integrative Evaluation”. ... 121

Summary. ... 123 2.8.

2.8.2.8.

11

3. Rationale and Method 3. Rationale and Method3. Rationale and Method

3. Rationale and Method...129...129129 129

3.1 3.1 3.1

3.1 SettingSetting ...SettingSetting... 131...131131131

The PROCON Curriculum. ... 131

3.2 3.2 3.2 3.2 Instructional Approach and PedagogyInstructional Approach and Pedagogy ...Instructional Approach and PedagogyInstructional Approach and Pedagogy... 136...136136136 BuT - Philosophy. ... 136

Degree of Autonomy in Participation and Learning... 136

Enactive Learning. ... 137

Modelling and Observational Learning. ... 137

Resource-based Learning and Error-Friendliness... 138

Coherence (Stimmigkeit) and Personal Development... 139

‘Instruction’ versus ‘Learning’ in the PROCON Curriculum. ... 140

3.3 3.3 3.3 3.3 ComputerComputer----Supported Learning in thComputerComputerSupported Learning in thSupported Learning in thSupported Learning in the e e PROCONe PROCON curriculumPROCONPROCON curriculum curriculum curriculum ... 143...143143143 Demand and Initial Survey... 143

Development and Implementation Process. ... 144

Approach to Blended Delivery... 145

Courseware Design and Pedagogy. ... 146

Content. ... 151

3.4 3.4 3.4 3.4 Evaluation Research DesignEvaluation Research Design ...Evaluation Research DesignEvaluation Research Design... 153...153153153 Integrative Evaluation Strategy. ... 153

Design... 155

3.5 3.5 3.5 3.5 ParticipantsParticipants ...ParticipantsParticipants... 157...157157157 3.6 3.6 3.6 3.6 Principles of Instrument Design and ImplementationPrinciples of Instrument Design and Implementation...Principles of Instrument Design and ImplementationPrinciples of Instrument Design and Implementation... 161161161161 Anonymity and Efficiency... 161

Micro-cultural Awareness and Change Processes. ... 162

Integrative Evaluation Approach. ... 162

3.7 3.7 3.7 3.7 Data Sources: Instruments and ProceduresData Sources: Instruments and Procedures ...Data Sources: Instruments and ProceduresData Sources: Instruments and Procedures... 164...164164164 Pre-test... 164

Online Surveys for PROCON Core Modules. ... 165

Courseware Log Files... 166

Post-test. ... 167

3.8 3.8 3.8 3.8 Data Analysis, Statistical Procedures, and Effect Size CalculationData Analysis, Statistical Procedures, and Effect Size Calculation...Data Analysis, Statistical Procedures, and Effect Size CalculationData Analysis, Statistical Procedures, and Effect Size Calculation... 172172172172 Principles of Data Analysis... 172

Parametric Effect Size Estimates... 174

Effect Size Estimates based on Non-Parametric Statistics... 175

Interpretations of Effect Size... 177

4.1 4.14.1

4.1 Indicators of Learning GainsIndicators of Learning GainsIndicators of Learning Gains ...Indicators of Learning Gains... 183183183183

Self-Reported Learning Gains... 183

Changes in Self-Efficacy Levels... 186

Knowledge and SJT Test Scores. ... 191

4.2 4.24.2 4.2 Indicators of Module PreparationIndicators of Module PreparationIndicators of Module Preparation ...Indicators of Module Preparation... 194194194194 Indicators of CSSL Courseware Use... 194

Validity of Courseware Use Measures... 195

Courseware Use Indices for Impact Analysis. ... 196

4.3 4.34.3 4.3 CCCSSL and LearningCSSL and LearningSSL and Learning ...SSL and Learning... 206206206206 H1: CSSL and Situational Judgement... 206

H2: Preparation and Situational Judgement. ... 214

H3: CSSL and Knowledge. ... 217

H4: CSSL and Post-Curricular Level of Interest. ... 219

H5: CSSL and Learning Gains. ... 221

4.4 4.44.4 4.4 CSSL and SelfCSSL and SelfCSSL and Self----EfficacyCSSL and SelfEfficacyEfficacy...Efficacy... 226...226226226 H6: CSSL and Self-Efficacy Levels. ... 226

H7: Preparation and Self-Efficacy Levels. ... 234

H8: CSSL and 'Accuracy' of Self-Efficacy Levels... 236

5. 5. 5. 5. DiscussionDiscussionDiscussionDiscussion... 239...239239239 5.1 5.15.1 5.1 ConclusionsConclusionsConclusionsConclusions ... 241241241241 Situational Judgement... 241

Attitudes: 'Self-efficacy' and Interest... 241

Conceptual Knowledge... 242

5.2 5.25.2 5.2 The Validity of Statistical ConclusionsThe Validity of Statistical ConclusionsThe Validity of Statistical Conclusions...The Validity of Statistical Conclusions... 243243243243 5.3 5.35.3 5.3 The Validity of Instruments and InferencesThe Validity of Instruments and InferencesThe Validity of Instruments and Inferences...The Validity of Instruments and Inferences... 247247247247 Instrumentation. ... 247

Design Considerations. ... 252

Selection and Attrition Biases... 252

Blended Learning Package... 255

Hawthorne Effects and Generalizability. ... 257

5.4 5.45.4 5.4 Further Implications for Theory, Research and PracticeFurther Implications for Theory, Research and PracticeFurther Implications for Theory, Research and PracticeFurther Implications for Theory, Research and Practice... 262262262262 CSSL Design and Integration... 262

CSSL in Postsecondary Education. ... 264

Assessment of Socio-communicative Competence... 266

Theory-based Research Agendas. ... 267 5.5

5.55.5

13 6. Appendix 6. Appendix6. Appendix 6. Appendix ...275275275 275 A. A. A.

A. PrePre----test Instruments ("Vorcheck", [V])PrePretest Instruments ("Vorcheck", [V])test Instruments ("Vorcheck", [V]) ...test Instruments ("Vorcheck", [V])... 277...277277277

Section I... 277

Section II... 283

Section III (Multimedia-based Tests)... 303

B. B. B. B. Module Online SurveysModule Online Surveys...Module Online SurveysModule Online Surveys... 309309309309 B.1 Moderation · Facilitation [F]... 309

B.2 Problem Solving II · Probleme Lösen II [P] ... 317

B.3 Konfliktmediation · Mediation Skills [M] ... 325

B.4 Konfliktmoderation · Conflict Resolution in Groups [K]... 333

C. C. C. C. PostPost----test Instruments ("Nachcheck" [N])PostPosttest Instruments ("Nachcheck" [N])test Instruments ("Nachcheck" [N])test Instruments ("Nachcheck" [N]) ... 347347347347 Section I (Evaluation) ... 347

Section II (Knowledge Tests) ... 355

Section III... 363

D. D. D. D. ExEx----Post facto Groups: PreExExPost facto Groups: PrePost facto Groups: PrePost facto Groups: Pre----test Equivalencestest Equivalencestest Equivalencestest Equivalences ... 387387387387 D.1 Sample Base vs. Excluded Participants (by Cohort) ... 387

D.2 KB2 Courseware Use Groups (Sample Base only)... 389

E. E. E. E. SPSS Extensions used for DomSPSS Extensions used for Dominance AnalysesSPSS Extensions used for DomSPSS Extensions used for Dominance Analysesinance Analysesinance Analyses... 391391391391 F. F. F. F. CR Self Efficacy Levels and Preparational MaterialsCR Self Efficacy Levels and Preparational Materials ...CR Self Efficacy Levels and Preparational MaterialsCR Self Efficacy Levels and Preparational Materials... 397397397397 F.1 Effects of Combined CR Courseware Use... 397

F.2 Courseware Use, Self-Efficacy, & Post-Test Scores... 398

F.3 Self-Efficacy & Module Preparation... 399

Bibliography BibliographyBibliography

Bibliography ...403403403 403

15

List of Tables

Table 1 Table 1Table 1

Table 1 Dimensions of Social CompetenceDimensions of Social CompetenceDimensions of Social CompetenceDimensions of Social Competence Table 2

Table 2Table 2

Table 2 Types of CognitiveTypes of CognitiveTypes of CognitiveTypes of Cognitive----Affective UnitsAffective UnitsAffective UnitsAffective Units Table 3

Table 3Table 3

Table 3 General Skills Necessary for Competent General Skills Necessary for Competent General Skills Necessary for Competent General Skills Necessary for Competent PerformancePerformancePerformancePerformance as a Neutra as a Neutra as a Neutrallll as a Neutra Table 4

Table 4Table 4

Table 4 Early Framework of Early Framework of Early Framework of Early Framework of ParametersParametersParametersParameters of Skills and Socio of Skills and Socio of Skills and Socio of Skills and Socio----Communicative Effectiveness Communicative Effectiveness Communicative Effectiveness Communicative Effectiveness in Mediation in Mediation in Mediation in Mediation Table 5 Table 5Table 5

Table 5 Lists of Situational Demand Characteristics (Tasks) and KSAOs in MediationLists of Situational Demand Characteristics (Tasks) and KSAOs in MediationLists of Situational Demand Characteristics (Tasks) and KSAOs in MediationLists of Situational Demand Characteristics (Tasks) and KSAOs in Mediation Table 6

Table 6Table 6

Table 6 EmpiricallyEmpiricallyEmpiricallyEmpirically grounded Knowledge and Skill Areas of the Mediator S grounded Knowledge and Skill Areas of the Mediator S grounded Knowledge and Skill Areas of the Mediator S grounded Knowledge and Skill Areas of the Mediator Skills Projectkills Projectkills Project kills Project Table 7

Table 7Table 7

Table 7 Differences between CognitiveDifferences between CognitiveDifferences between CognitiveDifferences between Cognitive----Constructivist and SocialConstructivist and SocialConstructivist and SocialConstructivist and Social----Constructivist Constructivist Constructivist Constructivist Perspectives Perspectives Perspectives Perspectives Table 8 Table 8Table 8

Table 8 Comparison of Conceptual Delivery, CSSL Courseware, and FaceComparison of Conceptual Delivery, CSSL Courseware, and FaceComparison of Conceptual Delivery, CSSL Courseware, and FaceComparison of Conceptual Delivery, CSSL Courseware, and Face----totototo----Face TrainingFace TrainingFace Training Face Training Table

Table Table

Table 99 99 Evaluation Outcome Parameters in Studies of TechnologyEvaluation Outcome Parameters in Studies of TechnologyEvaluation Outcome Parameters in Studies of TechnologyEvaluation Outcome Parameters in Studies of Technology----bbased/bbased/ased/ased/----supported supported supported supported Learning for Socio

Learning for Socio Learning for Socio

Learning for Socio----Communicative SkillsCommunicative SkillsCommunicative SkillsCommunicative Skills Table

Table Table

Table 101010 10 Groups of Evaluation ObjectsGroups of Evaluation ObjectsGroups of Evaluation ObjectsGroups of Evaluation Objects Table 1

Table 1Table 1

Table 11111 Suggested Improvements of the 'Conflict Resolution in Groups' ModuleSuggested Improvements of the 'Conflict Resolution in Groups' ModuleSuggested Improvements of the 'Conflict Resolution in Groups' ModuleSuggested Improvements of the 'Conflict Resolution in Groups' Module Table 1

Table 1Table 1

Table 1222 2 Paradigmatic Learning Sequence Paradigmatic Learning Sequence Paradigmatic Learning Sequence Paradigmatic Learning Sequence ((((Template used in Template used in Template used in Template used in BuTBuT ComputerBuTBuT Computer Computer Computer----BBBased Based ased ased Prepara

Prepara Prepara

Preparation Modules)tion Modules)tion Modules)tion Modules) Table 1

Table 1Table 1

Table 13333 Other Task Templates Used in Other Task Templates Used in Other Task Templates Used in Other Task Templates Used in BuTBuTBuT CoursewaresBuT Coursewares Coursewares Coursewares Table 1

Table 1Table 1

Table 14444 PrePrePrePre----testtesttesttest Equivalence of Sample Base and Excluded Participants: Equivalence of Sample Base and Excluded Participants: Equivalence of Sample Base and Excluded Participants: Equivalence of Sample Base and Excluded Participants: An Ex

An Ex An Ex

An Ex----Post Facto AnalysisPost Facto AnalysisPost Facto Analysis Post Facto Analysis Table 1

Table 1Table 1

Table 15555 Equivalence of Participant Cohorts (Sample Base Comparison)Equivalence of Participant Cohorts (Sample Base Comparison)Equivalence of Participant Cohorts (Sample Base Comparison)Equivalence of Participant Cohorts (Sample Base Comparison) Table 1

Table 1Table 1

Table 16666 Indicators Indicators Indicators Indicators of 'Conflict Resolution in Groups' Courseware Useof 'Conflict Resolution in Groups' Courseware Useof 'Conflict Resolution in Groups' Courseware Useof 'Conflict Resolution in Groups' Courseware Use Table 1

Table 1Table 1

Table 17777 Item Equivalence of Parallel Situational Judgment Test VersionsItem Equivalence of Parallel Situational Judgment Test VersionsItem Equivalence of Parallel Situational Judgment Test VersionsItem Equivalence of Parallel Situational Judgment Test Versions Table 1

Table 1Table 1

Table 18888 Appraisal of Effect Size EstimatorsAppraisal of Effect Size EstimatorsAppraisal of Effect Size EstimatorsAppraisal of Effect Size Estimators Table 1

Table 1Table 1

Table 19999 Sample Base Self Efficacy Gains in Sample Base Self Efficacy Gains in Sample Base Self Efficacy Gains in Sample Base Self Efficacy Gains in PROCONPROCON Modules with PROCONPROCON Modules with Modules with Modules with CorrespondingCorrespondingCorrespondingCorresponding Subject Subject Subject Subject Ma

Ma Ma Mattertterttertter Table

Table Table

Table 2020 2020 Knowledge and Situational Judgement Tests: Differences between Knowledge and Situational Judgement Tests: Differences between Knowledge and Situational Judgement Tests: Differences between Knowledge and Situational Judgement Tests: Differences between PrePre----testPrePretesttesttest and and and and Post

Post Post Post----testtesttest test Table 2

Table 2Table 2

Table 21111 Indicators of Courseware Use in ComputerIndicators of Courseware Use in ComputerIndicators of Courseware Use in ComputerIndicators of Courseware Use in Computer----Supported Social Learning ModulesSupported Social Learning ModulesSupported Social Learning Modules Supported Social Learning Modules Table 2

Table 2Table 2

Table 22222 Correspondence between SelfCorrespondence between SelfCorrespondence between SelfCorrespondence between Self----reported and Empirical Indicators oreported and Empirical Indicators oreported and Empirical Indicators oreported and Empirical Indicators of Courseware f Courseware f Courseware f Courseware Use Use Use Use Table 2 Table 2Table 2

Table 2333 3 SelfSelfSelfSelf----Reported Reported Conflict Resolution in GroupsReported Reported Conflict Resolution in GroupsConflict Resolution in GroupsConflict Resolution in Groups Courseware Use In Cohorts Courseware Use In Cohorts Courseware Use In Cohorts Courseware Use In Cohorts Table 2

Table 2Table 2

Table 24444 Courseware Usage Index (CUICourseware Usage Index (CUICourseware Usage Index (CUICourseware Usage Index (CUIKKKK) for Blended ) for Blended Conflict Resolution in Groups) for Blended ) for Blended Conflict Resolution in GroupsConflict Resolution in GroupsConflict Resolution in Groups

Modules Modules Modules Modules Table 2 Table 2Table 2

Table 2 Table 2 Table 2

Table 27777 Ex Post Facto Group Alternatives: TauEx Post Facto Group Alternatives: TauEx Post Facto Group Alternatives: Tau----bEx Post Facto Group Alternatives: Taubbb----Correlation with Courseware Use IndicesCorrelation with Courseware Use IndicesCorrelation with Courseware Use Indices Correlation with Courseware Use Indices Table 2

Table 2 Table 2

Table 28888 Characteristics of the Textbook/Literature Preparation Characteristics of the Textbook/Literature Preparation Characteristics of the Textbook/Literature Preparation IndicesCharacteristics of the Textbook/Literature Preparation IndicesIndicesIndices Table 2

Table 2 Table 2

Table 29999 AssociationAssociationAssociation between Courseware Usage and PostAssociation between Courseware Usage and Post between Courseware Usage and Post between Courseware Usage and Post----Test Situational Judgement Sum Test Situational Judgement Sum Test Situational Judgement Sum Test Situational Judgement Sum Scores Scores Scores Scores Table Table Table

Table 30303030 Repeated Measures MANOVA for PreRepeated Measures MANOVA for PreRepeated Measures MANOVA for Pre----Post SJT Results and ExRepeated Measures MANOVA for PrePost SJT Results and ExPost SJT Results and ExPost SJT Results and Ex----Post Facto Groups Post Facto Groups Post Facto Groups Post Facto Groups 2005/06 2005/062005/06 2005/06 Table 3 Table 3 Table 3

Table 3111 1 Prediction of Prediction of Prediction of PostPrediction of Post----testPostPosttesttest Situational Judgement Scores by Preparation Indices for test Situational Judgement Scores by Preparation Indices for Situational Judgement Scores by Preparation Indices for Situational Judgement Scores by Preparation Indices for Sa

Sa Sa

Sample Base (Multiple Regression Analysis)mple Base (Multiple Regression Analysis)mple Base (Multiple Regression Analysis)mple Base (Multiple Regression Analysis) Table 3

Table 3 Table 3

Table 32222 Association (rAssociation (rAssociation (rAssociation (rssss) between Use of Learning Materials and Mediation Knowledge ) between Use of Learning Materials and Mediation Knowledge ) between Use of Learning Materials and Mediation Knowledge ) between Use of Learning Materials and Mediation Knowledge

Post PostPost

Post----Test ScoresTest ScoresTest ScoresTest Scores Table 3

Table 3 Table 3

Table 33333 Association between Courseware Use and Association between Courseware Use and Association between Courseware Use and PostAssociation between Courseware Use and Post----testPostPosttesttest Interest (Controlling for test Interest (Controlling for Interest (Controlling for Interest (Controlling for PrePrePre Pre----test

testtest

test Preparation) Preparation) Preparation) Preparation) Table 3

Table 3 Table 3

Table 34444 CrossCrossCross----Cohort Prediction of Learning Gains in the CrossCohort Prediction of Learning Gains in the Cohort Prediction of Learning Gains in the Conflict Resolution in GroupsCohort Prediction of Learning Gains in the Conflict Resolution in GroupsConflict Resolution in GroupsConflict Resolution in Groups Module (Multiple Regression Analysis)

Module (Multiple Regression Analysis)Module (Multiple Regression Analysis) Module (Multiple Regression Analysis) Table 3

Table 3 Table 3

Table 35555 Association between Courseware Usage and Self Efficacy Raw GainsAssociation between Courseware Usage and Self Efficacy Raw Gains Association between Courseware Usage and Self Efficacy Raw GainsAssociation between Courseware Usage and Self Efficacy Raw Gains Table 3

Table 3 Table 3

Table 36666 Association between Courseware Usage and PreAssociation between Courseware Usage and Pre---- to Association between Courseware Usage and PreAssociation between Courseware Usage and Pre to Post to to PostPost----ttttestPost estest Raw Gains in Self est Raw Gains in Self Raw Gains in Self Raw Gains in Self Efficacy Efficacy Efficacy Efficacy Table 3 Table 3 Table 3

Table 37777 Repeated Measures MANOVA for SelfRepeated Measures MANOVA for Self----Efficacy Changes in MediationRepeated Measures MANOVA for SelfRepeated Measures MANOVA for SelfEfficacy Changes in MediationEfficacy Changes in MediationEfficacy Changes in Mediation----Related Related Related Related Modules and Mediation

Modules and MediationModules and Mediation

Modules and Mediation----Related Courseware ExRelated Courseware ExRelated Courseware Ex----Post Facto Groups for Cohort Related Courseware ExPost Facto Groups for Cohort Post Facto Groups for Cohort Post Facto Groups for Cohort 2004/05 2004/052004/05 2004/05 Table 3 Table 3 Table 3

Table 38888 CrossCrossCross----cohort Associations between SelfCrosscohort Associations between Self----Reported Selfcohort Associations between Selfcohort Associations between SelfReported SelfReported Self----EReported SelfEEEfficacy Levels prior to fficacy Levels prior to fficacy Levels prior to fficacy Levels prior to PROCON

PROCONPROCON

PROCON Modules with Corresponding Subject Matter (Item Reliability Modules with Corresponding Subject Matter (Item Reliability Modules with Corresponding Subject Matter (Item Reliability Modules with Corresponding Subject Matter (Item Reliability Estimation) Estimation)Estimation) Estimation) Table 3 Table 3 Table 3

Table 3999 9 CrossCross----cohort Ordinal Dominance SelfCrossCrosscohort Ordinal Dominance Self----Reported Selfcohort Ordinal Dominance Selfcohort Ordinal Dominance SelfReported SelfReported SelfReported Self----Efficacy Levels prior to Efficacy Levels prior to Efficacy Levels prior to Efficacy Levels prior to PROCON

PROCONPROCON

PROCON Modules with Corresponding Subject Matter (Distributional Stab Modules with Corresponding Subject Matter (Distributional Stab Modules with Corresponding Subject Matter (Distributional Stability Modules with Corresponding Subject Matter (Distributional Stability ility ility Estimation) Estimation)Estimation) Estimation) Table Table Table

Table 40404040 Associations between Potential Confounders and Situational Judgment Test ScoresAssociations between Potential Confounders and Situational Judgment Test Scores Associations between Potential Confounders and Situational Judgment Test ScoresAssociations between Potential Confounders and Situational Judgment Test Scores Table 4

Table 4 Table 4

Table 41111 Associations between Training Quality, Courseware Use, and Situational JudgmentAssociations between Training Quality, Courseware Use, and Situational Judgment Associations between Training Quality, Courseware Use, and Situational JudgmentAssociations between Training Quality, Courseware Use, and Situational Judgment Table 4

Table 4 Table 4

Table 42222 CSSL Courseware Use and Most Preferred Style of LeaCSSL Courseware Use and Most Preferred Style of Learning CSSL Courseware Use and Most Preferred Style of LeaCSSL Courseware Use and Most Preferred Style of Learning rning rning (2005/06 Sample Base)

(2005/06 Sample Base)(2005/06 Sample Base) (2005/06 Sample Base)

17

List of Figures

Figure 1 Figure 1Figure 1

Figure 1 Relationship between SocioRelationship between SocioRelationship between SocioRelationship between Socio----communicative Competence and Social communicative Competence and Social communicative Competence and Social communicative Competence and Social Communication Communication Communication Communication Figure 2 Figure 2Figure 2

Figure 2 An illustrative CAPS personality system with a network of CognitiveAn illustrative CAPS personality system with a network of CognitiveAn illustrative CAPS personality system with a network of CognitiveAn illustrative CAPS personality system with a network of Cognitive----Affective Affective Affective Affective Processing Units being activated by si

Processing Units being activated by si Processing Units being activated by si

Processing Units being activated by situational feature detection and associated tuational feature detection and associated tuational feature detection and associated tuational feature detection and associated processing units processing units processing units processing units Figure 3 Figure 3Figure 3

Figure 3 Five elements of the working definition of socioFive elements of the working definition of socioFive elements of the working definition of socioFive elements of the working definition of socio----communicative competencecommunicative competencecommunicative competence communicative competence Figure 4

Figure 4Figure 4

Figure 4 Four Four Four Four approaches to the assessment of socioapproaches to the assessment of socioapproaches to the assessment of socioapproaches to the assessment of socio----communicative competencecommunicative competencecommunicative competencecommunicative competence Figure 5

Figure 5Figure 5

Figure 5 Processes of enactive learnProcesses of enactive learnProcesses of enactive learnProcesses of enactive learning of socioing of socioing of socio----communicative competenceing of sociocommunicative competencecommunicative competence communicative competence Figure 6

Figure 6Figure 6

Figure 6 Cyclical pattern of learningCyclical pattern of learningCyclical pattern of learningCyclical pattern of learning Figure 7

Figure 7Figure 7

Figure 7 Processes of observational learningProcesses of observational learningProcesses of observational learningProcesses of observational learning Figure 8

Figure 8Figure 8

Figure 8 Four paths to socioFour paths to socioFour paths to socioFour paths to socio----communicative competence developmentcommunicative competence developmentcommunicative competence development communicative competence development Figure 9

Figure 9Figure 9

Figure 9 Extended Extended Extended Extended 3C 3C 3C model3C modelmodel model Figure 10

Figure 10Figure 10

Figure 10 Paradigm for Evaluating MultiParadigm for Evaluating MultiParadigm for Evaluating MultiParadigm for Evaluating Multi----MeMeMedia Learning EnvironmentsMedia Learning Environmentsdia Learning Environments dia Learning Environments Figure 11

Figure 11Figure 11

Figure 11 Evaluation CubusEvaluation CubusEvaluation CubusEvaluation Cubus Figure 12

Figure 12Figure 12

Figure 12 PROCONPROCONPROCONPROCON Core Curriculum Core Curriculum Core Curriculum Core Curriculum Figure 13

Figure 13Figure 13

Figure 13 Implementation of technologyImplementation of technologyImplementation of technologyImplementation of technology----based preparation modules into the based preparation modules into the based preparation modules into the based preparation modules into the PROCONPROCONPROCONPROCON curriculum curriculum curriculum curriculum Figure 14 Figure 14Figure 14

Figure 14 Extended Extended Extended Extended 3C Model3C Model3C Model conceptualisation of the typical delivery mix in P3C Model conceptualisation of the typical delivery mix in P conceptualisation of the typical delivery mix in P conceptualisation of the typical delivery mix in PROCON core ROCON core ROCON core ROCON core modules modules modules modules Figure 15 Figure 15Figure 15

Figure 15 Typical navigation and toolbars in Typical navigation and toolbars in Typical navigation and toolbars in Typical navigation and toolbars in BuTBuTBuT coursewaresBuT coursewares coursewares coursewares Figure 16

Figure 16Figure 16

Figure 16 Temporary Module and Evaluation Design PatternTemporary Module and Evaluation Design PatternTemporary Module and Evaluation Design PatternTemporary Module and Evaluation Design Pattern Figure 17

Figure 17Figure 17

Figure 17 Cohort Cohort Cohort Cohort ssssample ample ample ample ssssizesizesizesizes, attrition, and migration rates., attrition, and migration rates., attrition, and migration rates. , attrition, and migration rates. Figure 18

Figure 18Figure 18

Figure 18 SelfSelfSelfSelf----rated skills and knowledge gains in Facilrated skills and knowledge gains in Facilrated skills and knowledge gains in Facilrated skills and knowledge gains in Facilitation / Problem Solving Module itation / Problem Solving Module itation / Problem Solving Module itation / Problem Solving Module Online Surveys Online Surveys Online Surveys Online Surveys Figure 1 Figure 1Figure 1

Figure 1999 9 SelfSelfSelfSelf----rated skills and knowledge gains in rated skills and knowledge gains in Mediation Skills rated skills and knowledge gains in rated skills and knowledge gains in Mediation Skills Mediation Skills Mediation Skills //// Conflict Resolution in Conflict Resolution in Conflict Resolution in Conflict Resolution in Groups

Groups Groups

Groups Module Online Surveys Module Online Surveys Module Online Surveys Module Online Surveys Figures 20a

Figures 20aFigures 20a

Figures 20a----dddd Mean selfMean selfMean selfMean self----efficacy levels across time for sample base cohortsefficacy levels across time for sample base cohortsefficacy levels across time for sample base cohortsefficacy levels across time for sample base cohorts Figure 21

Figure 21Figure 21

Figure 21 Hypothesized links between Hypothesized links between Hypothesized links between Hypothesized links between PROCONPROCONPROCON modules and gains in selfPROCON modules and gains in self modules and gains in self----efficacy as modules and gains in selfefficacy as efficacy as efficacy as reported

reported reported reported Figures 21a

Figures 21aFigures 21a

Figures 21a----dddd Estimated magnitudes of change in selfEstimated magnitudes of change in selfEstimated magnitudes of change in selfEstimated magnitudes of change in self----efficacy levelsefficacy levelsefficacy levels efficacy levels Figures 22a

Figures 22aFigures 22a

Figures 22a----cccc Distributions of the Courseware Usage IndicesDistributions of the Courseware Usage IndicesDistributions of the Courseware Usage IndicesDistributions of the Courseware Usage Indices Figure 23

Figure 23Figure 23

Figure 23 Variations in selfVariations in selfVariations in selfVariations in self----reported degrees reported degrees reported degrees of text book penetration in mediationreported degrees of text book penetration in mediationof text book penetration in mediationof text book penetration in mediation----related related related related modules

modules modules modules

Groups Groups Groups

Groups Module by use of textbooks/literature (by cohort) Module by use of textbooks/literature (by cohort) Module by use of textbooks/literature (by cohort) Module by use of textbooks/literature (by cohort) Figure 25

Figure 25 Figure 25

Figure 25 Intensity and duration of preparation for the Intensity and duration of preparation for the PROIntensity and duration of preparation for the Intensity and duration of preparation for the PROPROPROCON Mediation SkillsCON Mediation SkillsCON Mediation SkillsCON Mediation Skills Module by Module by Module by Module by use of textbooks/literature (by cohort)

use of textbooks/literature (by cohort)use of textbooks/literature (by cohort) use of textbooks/literature (by cohort) Figure 26

Figure 26 Figure 26

Figure 26 Association between courseware use and situational judgement test scores in Association between courseware use and situational judgement test scores in Association between courseware use and situational judgement test scores in Association between courseware use and situational judgement test scores in cohort 2005/06 cohort 2005/06cohort 2005/06 cohort 2005/06 Figure 27 Figure 27 Figure 27

Figure 27 Between exBetween ex----post facto group comparison (courseware use and situational Between exBetween expost facto group comparison (courseware use and situational post facto group comparison (courseware use and situational post facto group comparison (courseware use and situational jud

jud jud

judgement test sum score)gement test sum score)gement test sum score) gement test sum score) Figure 28

Figure 28 Figure 28

Figure 28 Between exBetween ex----post facto group comparison (situational judgement test sum score Between exBetween expost facto group comparison (situational judgement test sum score post facto group comparison (situational judgement test sum score post facto group comparison (situational judgement test sum score and composite Courseware Usage Index CUI

and composite Courseware Usage Index CUIand composite Courseware Usage Index CUI and composite Courseware Usage Index CUIBBBB))))

Figure 29 Figure 29 Figure 29

Figure 29 PrePrePre----post analysis of SJT results for medianPrepost analysis of SJT results for medianpost analysis of SJT results for median----split courseware use expost analysis of SJT results for mediansplit courseware use exsplit courseware use exsplit courseware use ex----post facto post facto post facto post facto groups 2004/05 groups 2004/05groups 2004/05 groups 2004/05 Figure 30 Figure 30 Figure 30

Figure 30 PrePrePre----Post analysis of situational judgement test results for medianPrePost analysis of situational judgement test results for medianPost analysis of situational judgement test results for median----split courseware Post analysis of situational judgement test results for mediansplit courseware split courseware split courseware use ex

use exuse ex

use ex----post facto groups 2005/06post facto groups 2005/06post facto groups 2005/06post facto groups 2005/06 Figure 31

Figure 31 Figure 31

Figure 31 Situational judgement test scores for ExSituational judgement test scores for Ex----post Facto Groups basedSituational judgement test scores for ExSituational judgement test scores for Expost Facto Groups basedpost Facto Groups based on Courseware post Facto Groups based on Courseware on Courseware on Courseware and Textbook Use

and Textbook Useand Textbook Use and Textbook Use Figure 32

Figure 32 Figure 32

Figure 32 ExEx----post factExExpost factpost factpost facto group comparison (courseware use and chanceo group comparison (courseware use and chanceo group comparison (courseware use and chanceo group comparison (courseware use and chance----corrected mediation corrected mediation corrected mediation corrected mediation knowledge test sum score)

knowledge test sum score)knowledge test sum score) knowledge test sum score) Figure 33

Figure 33 Figure 33

Figure 33 PostPost----test level of interest in subject matter: Comparison between exPostPosttest level of interest in subject matter: Comparison between extest level of interest in subject matter: Comparison between extest level of interest in subject matter: Comparison between ex----post facto post facto post facto post facto courseware use groups

courseware use groupscourseware use groups courseware use groups Figure 34.

Figure 34. Figure 34.

Figure 34. CrossCross----blended learning cohorts learning gCrossCrossblended learning cohorts learning gblended learning cohorts learning gain score averages (K09) and ordinal blended learning cohorts learning gain score averages (K09) and ordinal ain score averages (K09) and ordinal ain score averages (K09) and ordinal dominance statistics for between ex

dominance statistics for between exdominance statistics for between ex

dominance statistics for between ex----post facto courseware use groups (tercilepost facto courseware use groups (tercilepost facto courseware use groups (tercile----post facto courseware use groups (tercile split/K split/Ksplit/K split/KB2B2B2B2)))) Figure 3 Figure 3 Figure 3

Figure 355 55 Changes in mediation strategy selfChanges in mediation strategy self----efficacy levels [SE10] for medianChanges in mediation strategy selfChanges in mediation strategy selfefficacy levels [SE10] for median----split exefficacy levels [SE10] for medianefficacy levels [SE10] for mediansplit exsplit exsplit ex----post post post post facto courseware use groups K

facto courseware use groups Kfacto courseware use groups K facto courseware use groups Kww ww

Figure Figure Figure

Figure 3 3 3 3666 6 Changes in conflict moderation process self----efficacy levels [SE11] for medianChanges in conflict moderation process selfChanges in conflict moderation process selfChanges in conflict moderation process selfefficacy levels [SE11] for medianefficacy levels [SE11] for medianefficacy levels [SE11] for median----split split split split ex

exex

ex----post facto courseware use groups Kpost facto courseware use groups Kpost facto courseware use groups Kpost facto courseware use groups Kwwww

Figure 3 Figure 3 Figure 3

Figure 3777 7 Changes in conflict resolution in groups applied skills selfChanges in conflict resolution in groups applied skills self----efficacy levels [SE13] Changes in conflict resolution in groups applied skills selfChanges in conflict resolution in groups applied skills selfefficacy levels [SE13] efficacy levels [SE13] efficacy levels [SE13] for median

for median for median

for median----split exsplit exsplit ex----post facto coursplit expost facto courpost facto courpost facto courseware use groups Kseware use groups Kseware use groups Kseware use groups Kwwww

Figure 3 Figure 3 Figure 3

Figure 3888 8 Changes in selfChanges in self----efficacy levels for mediation competencies (SE07), mediation Changes in selfChanges in selfefficacy levels for mediation competencies (SE07), mediation efficacy levels for mediation competencies (SE07), mediation efficacy levels for mediation competencies (SE07), mediation strategy knowledge (SE12) and applied two

strategy knowledge (SE12) and applied twostrategy knowledge (SE12) and applied two

strategy knowledge (SE12) and applied two----party mediation skills (SE13) in the party mediation skills (SE13) in the party mediation skills (SE13) in the party mediation skills (SE13) in the 2004/05 blended learning Mediation Skills Module for median

2004/05 blended learning Mediation Skills Module for median2004/05 blended learning Mediation Skills Module for median

2004/05 blended learning Mediation Skills Module for median----split exsplit exsplit exsplit ex----post facto post facto post facto post facto Mediation Skills courseware use Groups M

Mediation Skills courseware use Groups MMediation Skills courseware use Groups M Mediation Skills courseware use Groups Mwwww

Figures 3 Figures 3 Figures 3

Figures 3999----4944411 Changes in self11 Changes in selfChanges in self----efficacy levels for mediationChanges in selfefficacy levels for mediationefficacy levels for mediation----related courseware use group efficacy levels for mediationrelated courseware use group related courseware use group related courseware use group combinations

combinationscombinations combinations (I (I (I (I----III)III)III) III) Figure 4

Figure 4 Figure 4

Figure 4222 2 Preparational modes and changes in selfPreparational modes and changes in self----efficacyPreparational modes and changes in selfPreparational modes and changes in selfefficacyefficacy level of mediation process efficacy level of mediation process level of mediation process level of mediation process knowled

knowledknowled

knowledge (SE10)ge (SE10)ge (SE10)ge (SE10) Figure 43

Figure 43 Figure 43

Figure 43 Courseware use, learning gains, and situational judgement postCourseware use, learning gains, and situational judgement post----test scores Courseware use, learning gains, and situational judgement postCourseware use, learning gains, and situational judgement posttest scores test scores test scores [MM_SRE] for two groups with different situational judgement pre [MM_SRE] for two groups with different situational judgement pre[MM_SRE] for two groups with different situational judgement pre

[MM_SRE] for two groups with different situational judgement pre----test versions in test versions in test versions in test versions in the 2004/05 cohort

the 2004/05 cohortthe 2004/05 cohort the 2004/05 cohort Figure 4

Figure 4 Figure 4

Figure 444 44 Relationship between Relationship between aRelationship between Relationship between aaage, ge, ccccourseware ge, ge, ourseware ourseware ourseware uuuuse, and se, and ssssituse, and se, and ituituational ituational ational ational jjjjudgement udgement pudgement udgement pppostostostost----ttttest est est est scores scoresscores scores Figure 4 Figure 4 Figure 4

Figure 4555 5 Relationship between Relationship between oRelationship between Relationship between ooccupational occupational ccupational eccupational experience, eexperience, xperience, xperience, BIPBIPBIPBIP ssssensitivity ensitivity ensitivity ensitivity sssscore, core, core, ccccourseware core, ourseware ourseware ourseware u

uu

use, and se, and se, and ssssituational se, and ituational ituational jjjjudgement ituational udgement udgement udgement ppostppostostost----ttttest est est est scoresscoresscoresscores Figure 4

Figure 4 Figure 4

Figure 466 66 Rated Rated qRated Rated qquality of quality of uality of uality of cccclassroomlassroomlassroomlassroom----based based ttttraining, based based raining, raining, ccccourseware raining, ourseware ourseware uourseware uuse and use and se and ssssituationalse and ituationalituational ituational jjjjudgement udgement udgement udgement pppostpostostost----ttttest est est sssscoresest corescorescores

19

Evaluating Computer

Evaluating Computer

Evaluating Computer

Evaluating Computer----Supported Social Learning:

Supported Social Learning:

Supported Social Learning:

Supported Social Learning:

An Integrative Study of the Development of Soft Skills

An Integrative Study of the Development of Soft Skills

An Integrative Study of the Development of Soft Skills

An Integrative Study of the Development of Soft Skills

in a University

in a University

in a University

21

1. Introduction

It is a unique and original feature of this study to attempt to systematically evaluate the conditions of computer-supported mediation training in a progressive higher learning setting. In this introductory chapter, the target problems to be studied are outlined. Given that mediation training can be understood as a special form of interpersonal or socio-communicative skills training, this study seeks to explore the question how new learning technologies in the form of preparatory video-based, interactive media can support learning.

In the following, interpersonal skills as needed for third-party roles in dyadic and group conflict are briefly reviewed, and a structure of the skills is formulated. Subsequently, the present state of research dealing with the use of instructional and learning technologies to support the development of the skills needed is summarized.

It is contended that - in view of the scarcity of research investigating the effects of blending traditional with technology-based soft skills training delivery formats -, it seems a worthwhile effort to put forward and test a model for evaluating computer-supported soft skills training.

1.1 Mediation Training as Soft Skills Development in Higher Education Social skills training methods

1.1 Mediation Training as Soft Skills Development in Higher Education

Today, “soft skills” or, more specifically, interpersonal or socio-communicative competencies are recognized as being of high relevance in many applied contexts such as psychotherapy (e.g. Beutler, Machado, & Neufeldt, 1994) or management (e.g. Penley, Alexander, Jernigan, & Henwood, 1991; Cameron & Whetten, 1983). Social competent behaviour is also regarded as a keystone for successful interpersonal conflict management (Bowling & Hoffman, 2000, 1989; Ogilvie & Carsky, 2002; Schreier, 2002; The Test Design Project, 1995). It is therefore not surprising that, researchers, representatives of industry and commerce, and educational policymakers alike have repeatedly argued for the implementation of competency-based training and assessment into existing curricula in higher and continuing education settings (e.g. Chur, 2004; DIHK, 2004; Eurydice, 2002; Agiunis & Kraiger 1997).

This need has been described as a shortage of key skills, and, more specifically, as a mismatch between required and actual communication and cooperation skills of higher education graduates (Schaeper & Briedis, 2004; García-Aracil, Mora, & Vila, 2003). This common, transnational problem is also reflected in the European higher education area’s adoption of a system of convergence in higher education to be implemented by 2010, often referred to as the Bologna process. To promote graduates’ employability, accreditation provisions for new curricula in higher education require a significant percentage of learning credits to be earned through key and soft skills training (cf. e.g. FIBAA, 2000; ZevA, 2000).

This poses a number of challenges to staff, curriculum managers, and policymakers in institutions of higher education (Redlich & Rogmann, 2007). It is a quantitative challenge in that, on the one hand, a considerable number of additional courses focusing on key qualifications will have to be added to extant curricula. On the other hand, qualified staff is needed for implementation meaning that either current staff will have to be developed or new teaching staff is to be found. Moreover, it is also a qualitative challenge in that the existing and modules and courses of study will have to be developed in terms of content to concurrently provide opportunities for development of key skills. And, finally, it is also a challenge in terms of didactics and methods. For centuries, institutions of higher education have focused on refining knowledge development, knowledge acquisition and the passing-on of knowledge. By contrast, fostering – and, subsequently, testing (McClelland, 1973) and credentialing – skills and competencies rather than knowledge has tended to be sidelined in most disciplines, and it

Evaluating CSSL in mediation training 1. Introduction

is probably better explored in vocational and professional educational contexts such as training and assessment for management, sales, or customer service.

Social skills training methods

Methods employed in social skills training.

Chiefly conducted in “classroom” or “face-to-face” settings (Henninger, Hörfurter, & Mandl, 2001), social and communication skills training programmes employ interactive exercises and group discussions (Bedell & Lennox, 1997; Bishop & Taylor, 1992; Fittkau & Schulz von Thun, 1994; Günther & Sperber, 1993; Schulz von Thun, 1984, 2004a). Role-play simulations are also regularly made use of, intended to promote valid behavioural and emotional development and a foundation on which learning through modelling and observation as well as reflection can be built (e.g. Davis & Corley, 1996; Mock, 1997; Redlich & Elling, 2000). Often, simulations are video- and/or audio-recorded to allow for a more in-depth instructor- and/or peer-based feed-back, collective reflection of practice (cf. Schön, 1983), or micro-training (Daniels, Rigazio DiGilio, & Ivey, 1997). Methods of in-class communication training also include other experiential learning exercises (D. A. Kolb, 1984) and practice techniques designed to promote self-awareness and self-enquiry (Bedell & Lennox, 1997). The latter include, for example, approaches based on the “Inner Team” metaphor (Schulz von Thun & Bossemeyer, 1993; Ulrichs, 2004; cf. also Redlich, 2004b, for the mediation context), or the use of self-evaluative questionnaires and journal writing (LeBaron Duryea & Robinson, 1994).

Mediation training is soft skills development

Mediation training as “soft skills” development.

One special form of social competence or “soft-skills” development for applied professions such as Psychology or Law is mediation training (cf. Falk, 2000). Around the globe, mediation and Alternative Dispute Resolution (ADR) training programmes are offered by a growing number of higher education institutions (Botes, 2004; Polkinghorn & Chenall, 2000, Warters, 2000) as well as in private agencies or corporate training institutes.

Regularly, existing programmes include sophisticated communication training (Raider, Coleman, & Gerson, 2000; Redlich & Elling, 2000; Schneider, 2000), methods intended to enhance emotional and self development (Prokop-Zischka, 2000; Reilly, 2005; Schreier, 2002; Shearouse, 2003; Stains, 2003) and opportunities for self-reflection (Marsick & Sauquet, 2000; Picard, 2003). In general, therefore, mediation training curricula draw on the same instructional

1.1 Mediation Training as Soft Skills Development in Higher Education The need for evaluative research

techniques and, possibly, learning paradigms as employed in other fields of human resource and soft skill development (some of which are outlined above).

Some justifiably call for mediation training that aims more directly at developing the trainees’ personal skills in handling emotions as well as their emotional awareness and self-regulation skills (Schreier, 2002). However, in a precursory study of the mediation training curriculum evaluated here, Langhorst (2005) suggests that mediation training curricula may have the potential to further the personal social skills of curriculum participants despite their different focus on the role of neutral third parties, for instance in the form of enhanced individual negotiation skills and increased self-efficacy in individual dispute handling.

The need for evaluative research

The need for evaluative research.

Albeit today there is an ever-growing body of research and practice literature dealing with mediation, its models and processes, the role of third parties in conflict management, mediation skills, and even the training of neutrals, few researchers have yet attempted to evaluate learning progress and outcome in mediation training programmes.

In the on-going process of professionalization in mediation (Maiwald, 2004; Picard, 1994), professional societies as well as juridical and governmental bodies attempt to channel this very process by issuing training guidelines and accreditation standards (Wassner, 2002). In good accordance with their objective to provide a basis for quality in mediation (SPIDR Board of Directors' Commission on Qualifications, 1995), many of these guidelines address the evaluation of trainee progress and trainer responsibilities (cf. e.g. Pou, 2002). Here, the solicitation of evaluative comments from trainers or trainees, the assessment of participants, and the use of training evaluation forms are called for (e.g. CAADRS Center for Analysis of Alternative Dispute Resolution Systems, 2004; Conflict Management in Higher Education Resource Center, 2004) or even mandated in court programmes. Interestingly, in contrast to these pleas, a comprehensive mediation training program evaluation has not yet been an objective in applied psychological, educational, or social research. In 2000, Raider, Coleman and Gerson found that

“although there is an extensive theoretical and empirical literature on the nature of conflict and the processes of negotiation and mediation as applied in diplomacy, business, and labor relations, there is very little systematic research on the pedagogy of conflict resolution or on the models and methods used to teach these skills to adult or student learners” (p. 499).

Evaluating CSSL in mediation training 1. Introduction

Since, a growing body of research has further explored the field of in-school conflict resolution education (Deutsch, 2000; Jones, 2004), manifested primarily in peer mediation programmes (e.g. Burrell, Zirbel, and Allen, 2003) and “life skills” programmes aimed at developing student’s emotional, cognitive, and behavioural competencies thought to be required to deal with personal conflict (e.g. Greenberg et al., 2000). Overall, however, there still is extremely little ‘developmental’ research concerned with helping to shape effective educational and training programs in the conflict resolution arena (Deutsch, 2000). In addition to difficulties evaluation researchers typically come across in higher education settings (e.g. cf. Bülow-Schramm, 1995; Lohnert & Rolfes, 1998; McEvoy & Buller, 1990; and, discussing specifics of e-learning evaluation in higher education, Schwarz, 2001), a number of additional reasons could account for this phenomenon.

No evaluation tradition.

No evaluation tradition.

Firstly, in the relatively young field of conflict resolution, mediation program review processes in higher education are usually an “internal matter” and curricula, course content, student performance standards, or program structures are usually not subjected to evaluation (Botes, 2004).

Vocational assessment of practitioners.

Vocational assessment of practitioners.

Secondly, as it is a common notion in the field that training is only one of a multiplicity of paths to acquire mediation skills, much emphasis has been put on defining mediator competencies and developing methods to assess these (Friedman & Silberman, 1993; Herrman, Hollett, Gale, & Foster, 2001, 2002; Honeyman, 1990, 1993; Honoroff, Matz, & O'Connor, 1990; LeBaron Duryea, 1994; Matz, 1993; McEwen, 1993; Pou, 2002; SPIDR Board of Directors' Commission on Qualifications, 1995; The Test Design Project, 1995; Wassner, 2002) rather than evaluating the acquisition of these skills in training programmes. Moreover, despite all attempts to pinpoint knowledge, skills, and other attributes (commonly referred to as “KSAOs”) needed for successful mediation, recent articles suggest that it remains to be debated whether and which KSAOs should or could be tested or, indeed, if they are prerequisite for the employment of mediators at all (Honeyman, Go, & Kelly, 2004).

Foci of mediation research.

Foci of mediation research.

Thirdly, much evaluative research in the field has focused on determining the efficacy and effectiveness of mediation programmes, while theory-driven dispute resolution research

1.1 Mediation Training as Soft Skills Development in Higher Education Funding of mediation research.

has accentuated experimental research strategies. In doing so, both have generally contributed little to either practice or mediation training (McEwen, 1999).

Funding of mediation research.

Funding of mediation research.

And, fourthly, while there may be an interested consumer audience for research aimed at evaluating programmes aimed at qualifying neutrals, evaluation research with the objective to go beyond mere efficacy control typically requires additional funding and thus, the need to accept additional evaluation stakeholders.

In sum, there is a dearth of educational evaluative research, however. Fresh contributions to the field aimed at evaluating training programs for mediation that can help enhancing existing programs are highly warranted.

DIGEST 1.1

It is a challenging undertaking to introduce competency-based trainings for key qualifications into the curricula offered at higher education institutions across Europe. These training courses employ specific methods that aim to reduce the mismatch found between the level of social skills required of graduates entering professional life and their actual skill levels. One special form of social skills training is mediation training. For various reasons, there is a dearth of educational evaluative research in this area; innovative contributions are therefore highly warranted.

Evaluating CSSL in mediation training 1. Introduction

1.2 Social Skills Training: ‘Face-to-face’ and ‘Blended’ Delivery

The past decades have seen an unanticipated evolution of learning and instruction by means of electronic media, such as audio and, later, video tapes, -disks, or –broadcasts, computer-based applications with an increasing degree of interactivity, the use of the Internet and other forms of digital media. The utilization of these media for learning purposes have greatly affected (a) the ways learning content is distributed or delivered, (b) the potentials of learner control, adaptive tutoring, and collaboration in learning, and (c) teaching activities. It will be explored below, in what ways and to what extent technology may complement 'traditional' social skills training methods as outlined above.

In the course of technology advancement, a vast variety of expressions have been coined to characterise the phenomenon that traditional learning activities and learning management are today supplemented by – or even replaced with – technological means. Some of these expressions are “computer-“ or, more broadly, “technology-assisted (or –aided, -enhanced, or -mediated) learning”, “e-learning”, “computer-“ or “web-based learning or “training”, “hybrid” or “blended learning”, “online”, distance” and “multimedia learning”, “hypermedia-“ or, shorter, “hyper-learning”, “open learning system”, “flexible” and/or “distance learning”, etc.1 The same holds true for measurement and assessment processes in education and personnel selection, with the “learning” or “training” part of the expressions listed above being replaced by “assessment”, “testing” “evaluation”, or “measurement”.

As Goldman-Segall and Maxwell (2003) explain, the functional roles technology can have in learning are manifold. Technology can serve as an information provision system, as a subject and curriculum area in itself, as a communications medium, as a “thinking tool”, as an experiential environment, as a developmental scaffold, or as a discourse and perspectivity toolkit. However, they also note that research often has “the tendency … to use an instrumentalist and instructionist approach” (Goldman-Segall & Maxwell, 2003, p. 397) with little regard for social dimensions of learning. As a result, learning technologies have often been viewed as a possible substitutes for “traditional” learning activities (often termed “face-to-face-“ or “classroom” learning), and, consequently, researched as such: Numerous

1 _{It seems to depend on tradition, fashion (Northrup, 2002) and progress of technology (Reinmann-Rothmeier,}

2003a) yet which expression is being used and what exactly is meant by it. At this point, the term "e-learning" will be used in the following. Subsequently, for the purposes of this study, more detailed definitions will be given below.