In order to fully actualize the importance of parameter optimization in machine learning algorithms, SMAC and AutoWEKA were used.
All of the datasets and machine learning algorithms were optimized by using 10-fold cross-validation when generating their corresponding instances.
Due to the limitations of AutoWEKA, the only result metric that could be used
chine learning algorithms, therefore it was not possible to obtain the optimization results for AdaBoost (AB), radial basis function network (RFN) and minimal cost-complexity pruning (MCP).
The following AutoWEKA configuration parameters were used for each dataset and machine learning algorithm:
• Optimization Timeout (hours): 1
• Training Memory Limit (MB): 2048
• Training Run Timeout (minutes): 60
• Use Attribute Selection: Yes
• Attribute Selection Timeout (minutes): 60
As soon as one of the timeouts was reached, AutoWEKA stopped the optimization process and used the parameters that gave the best results at that particular point in time.
The results obtained by AutoWEKA after parameter optimization for Classifica-tion Accuracy (CA) can be seen in figure 3.9.
Figure 3.9: Classification accuracy for all subjects after parameter optimization.
The average results for Classification Accuracy (CA) with and without Parameter Optimization (PO) can be seen in figure 3.10.
Figure 3.10: Average classification accuracy for all subjects with and without parameter optimization.
The optimized parameter values for all of the subjects can be found in Appendix C. More detailed information about the optimized parameters can be found at:
https://github.com/madism/ECMLA/tree/master/Parameters
Chapter 4 Conclusion
This thesis was created in order to find the best machine learning algorithm for classifying EEG data by means of an empirical comparison. In order to achieve this, three datasets, which were created for various brain-computer interface (BCI) competitions, were used. A total of 10 subjects from the datasets were used.
These datasets were preprocessed and then classified by the following machine learning algorithms/methods: AdaBoost, k-Nearest Neighbours, C4.5 decision tree, logistic regression model, multilayer perceptron network, naive Bayes, ran-dom forest, radial basis function network, minimal cost-complexity pruning and support vector classifier. The classification results were evaluated using 10-fold cross-validation.
The visual comparison which was done by using clustergrams showed that some of the predictions generated by the classifiers were highly correlated. This was confirmed by the comparison done by using correlation matrices, which showed that there was a moderately strong (value of the correlation being in the range of 0.4 to 0.6) correlation between the following classifiers:
• AdaBoost and random forest;
• AdaBoost and minimal cost-complexity pruning;
• Logistic regression and SVM;
• Multilayer Perceptron Network and SVM;
• Random forest and SVM.
The comparison that was done by using correlation matrices showed that the highest correlation coefficient was in Subject 2 from Dataset 1 between sequential minimal optimization algorithm for training a support vector classifier (SVM) and Multilayer Perceptron Network (MPN), which had a correlation value of 0.77, therefore indicating a strong correlation between the two classifiers.
The comparison that was done by using three performance metrics (classifica-tion accuracy, time taken to build a model and F-score) showed that according to our datasets, the best classifier for classifying EEG data without parameter optimization is the logistic regression model (LM), followed by sequential mini-mal optimization algorithm for training a support vector (SVM) and Multilayer Perceptron Network (MPN).
The comparison that was done by using SMAC and AutoWEKA showed the im-portance of parameter optimization. All of the machine learning algorithms which were able to be optimized benefited greatly from the optimization. Average Clas-sification Accuracy (CA) was improved by the amount of 14.10% in the case of the logistic regression and up to 24.05% in the case of the C4.5 decision tree model.
The results from the comparison by using optimized parameters showed that ac-cording to our datasets, the best classifier for classifying EEG data with parameter optimization is random forest (RF), followed by sequential minimal optimization algorithm for training a support vector (SVM) and logistic regression model (LM).
Bibliography
[1] Y. Freund and R. E. Schapire, “Experiments with a new boosting algo-rithm,” in Thirteenth International Conference on Machine Learning, (San Francisco), pp. 148–156, Morgan Kaufmann, 1996.
[2] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” Journal of computer and system sciences, vol. 55, no. 1, pp. 119–139, 1997.
[3] W. Iba and P. Langley, “Induction of one-level decision trees,” inProceedings of the ninth international conference on machine learning, pp. 233–240, 1992.
[4] D. Aha and D. Kibler, “Instance-based learning algorithms,”Machine Learn-ing, vol. 6, pp. 37–66, 1991.
[5] N. S. Altman, “An introduction to kernel and nearest-neighbor nonparametric regression,” The American Statistician, vol. 46, no. 3, pp. 175–185, 1992.
[6] R. Quinlan,C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann Publishers, 1993.
[7] D. R. Cox, “The regression analysis of binary sequences,”Journal of the Royal Statistical Society. Series B (Methodological), pp. 215–242, 1958.
[8] K. Søreide, A. J. Kr¨uger, A. L. V˚ardal, C. L. Ellingsen, E. Søreide, and H. M. Lossius, “Epidemiology and contemporary patterns of trauma deaths:
changing place, similar pace, older face,” World journal of surgery, vol. 31, no. 11, pp. 2092–2103, 2007.
[9] S. H. Walker and D. B. Duncan, “Estimation of the probability of an event as a function of several independent variables,” Biometrika, vol. 54, no. 1-2, pp. 167–179, 1967.
[10] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal rep-resentations by error propagation,” tech. rep., DTIC Document, 1985.
[11] G. Cybenko, “Approximation by superpositions of a sigmoidal function,”
Mathematics of control, signals and systems, vol. 2, no. 4, pp. 303–314, 1989.
[12] F. Rosenblatt, “Principles of neurodynamics. perceptrons and the theory of brain mechanisms,” tech. rep., DTIC Document, 1961.
[13] G. H. John and P. Langley, “Estimating Continuous Distributions in Bayesian Classifiers,” in Eleventh Conference on Uncertainty in Artificial Intelligence, (San Mateo), pp. 338–345, Morgan Kaufmann, 1995.
[14] J. D. Rennie, L. Shih, J. Teevan, D. R. Karger, et al., “Tackling the poor assumptions of naive Bayes text classifiers,” in ICML, vol. 3, pp. 616–623, Washington DC), 2003.
[15] I. Rish, “An empirical study of the naive Bayes classifier,” in IJCAI 2001 workshop on empirical methods in artificial intelligence, vol. 3, pp. 41–46, IBM New York, 2001.
[16] L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.
[17] T. K. Ho, “Random decision forests,” inDocument Analysis and Recognition, 1995., Proceedings of the Third International Conference on, vol. 1, pp. 278–
282, IEEE, 1995.
[18] Y. Amit and D. Geman, “Shape quantization and recognition with random-ized trees,” Neural computation, vol. 9, no. 7, pp. 1545–1588, 1997.
[19] D. S. Broomhead and D. Lowe, “Radial basis functions, multi-variable func-tional interpolation and adaptive networks,” tech. rep., DTIC Document, 1988.
[20] F. Schwenker, H. A. Kestler, and G. Palm, “Three learning phases for radial-basis-function networks,” Neural networks, vol. 14, no. 4, pp. 439–458, 2001.
[21] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Belmont, California: Wadsworth International Group, 1984.
[22] J. Pearl, “Heuristics: intelligent search strategies for computer problem solv-ing,” 1984.
[23] J. Plattet al., “Fast training of support vector machines using sequential min-imal optimization,”Advances in Kernel Methods — Support Vector Learning, vol. 3, 1999.
[24] S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy, “Im-provements to Platt’s SMO algorithm for SVM classifier design,”Neural Com-putation, vol. 13, no. 3, pp. 637–649, 2001.
[25] T. Hastie, R. Tibshirani, et al., “Classification by pairwise coupling,” The annals of statistics, vol. 26, no. 2, pp. 451–471, 1998.
[26] J. C. Platt et al., “Using analytic QP and sparseness to speed training of support vector machines,”Advances in neural information processing systems, pp. 557–563, 1999.
[27] C. Cortes and V. Vapnik, “Support-vector networks,” Machine learning, vol. 20, no. 3, pp. 273–297, 1995.
[28] R. Caruana and A. Niculescu-Mizil, “An empirical comparison of supervised learning algorithms,” in Proceedings of the 23rd international conference on Machine learning, pp. 161–168, ACM, 2006.
[29] B. Karlık and S¸. B. Hayta, “Comparison Machine Learning Algorithms for Recognition of Epileptic Seizures in EEG,”
[30] P. Sajda, A. Gerson, K. Muller, B. Blankertz, and L. Parra, “A data anal-ysis competition to evaluate machine learning algorithms for use in brain-computer interfaces,” Neural Systems and Rehabilitation Engineering, IEEE Transactions on, vol. 11, no. 2, pp. 184–185, 2003.
[31] C. Hope, “A cap holds electrodes in place while recording an EEG,” 2012.
[32] M. Tudor, L. Tudor, and K. I. Tudor, “Hans Berger (1873-1941)–the history of electroencephalography,”Acta medica Croatica: casopis Hravatske akademije medicinskih znanosti, vol. 59, no. 4, pp. 307–313, 2004.
[33] M. H¨am¨al¨ainen, R. Hari, R. J. Ilmoniemi, J. Knuutila, and O. V. Lounas-maa, “Magnetoencephalography—theory, instrumentation, and applications
to noninvasive studies of the working human brain,” Reviews of modern Physics, vol. 65, no. 2, p. 413, 1993.
[34] J. Claassen, S. Mayer, R. Kowalski, R. Emerson, and L. Hirsch, “Detection of electrographic seizures with continuous EEG monitoring in critically ill patients,” Neurology, vol. 62, no. 10, pp. 1743–1748, 2004.
[35] D. J. Kupfer and M. E. Thase, “The use of the sleep laboratory in the diag-nosis of affective disorders,” Psychiatric Clinics of North America, 1983.
[36] F. Plum and J. B. Posner,The diagnosis of stupor and coma, vol. 19. Oxford University Press, 1982.
[37] P. Ferenci, A. Lockwood, K. Mullen, R. Tarter, K. Weissenborn, and A. T.
Blei, “Hepatic encephalopathy—definition, nomenclature, diagnosis, and quantification: final report of the working party at the 11th World Congresses of Gastroenterology, Vienna, 1998,” Hepatology, vol. 35, no. 3, pp. 716–721, 2002.
[38] E. F. Wijdicks, “Determining brain death in adults,”Neurology, vol. 45, no. 5, pp. 1003–1011, 1995.
[39] C. Miniussi and G. Thut, “Combining TMS and EEG offers new prospects in cognitive neuroscience,” Brain topography, vol. 22, no. 4, pp. 249–256, 2010.
[40] W. J. Ray and H. W. Cole, “EEG alpha activity reflects attentional de-mands, and beta activity reflects emotional and cognitive processes,”Science, vol. 228, no. 4700, pp. 750–752, 1985.
[41] N. Braisby and A. Gellatly, Cognitive psychology. Oxford University Press, 2012.
[42] H. M. M¨uller, “Neurolinguistic findings on the language lexicon: The special role of proper names,”Chinese Journal of Physiology, vol. 53, no. 6, pp. 351–
358, 2010.
[43] A. Kales, A. Jacobson, M. J. Paulson, J. D. Kales, and R. D. Walter,
“Somnambulism: Psychophysiological correlates: I. All-night EEG studies,”
Archives of General Psychiatry, vol. 14, no. 6, pp. 586–594, 1966.
[44] D. Millet, “The Origins of EEG,” in7th Annual Meeting of the International Society for the History of the Neurosciences (ISHN), 2002.
[45] A. L. Goldberger, L. A. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G.
Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley, “Phys-iobank, physiotoolkit, and physionet components of a new research resource for complex physiologic signals,” Circulation, vol. 101, no. 23, pp. e215–e220, 2000.
[46] R. Kohavi and F. Provost, “Glossary of terms,” Machine Learning, vol. 30, no. 2-3, pp. 271–274, 1998.
[47] C. M. Bishopet al.,Pattern recognition and machine learning, vol. 4. springer New York, 2006.
[48] Y. Koren, “The Bellkor solution to the Netflix grand prize,” Netflix prize documentation, vol. 81, 2009.
[49] S. Patterson, “Letting the machines decide,” Wall Street Journal (Online), 2010.
[50] B. Saleh, K. Abe, R. S. Arora, and A. Elgammal, “Toward automated dis-covery of artistic influence,” Multimedia Tools and Applications, pp. 1–27, 2014.
[51] P. R. Norvig and S. A. Intelligence, “A modern approach,” 2002.
[52] D. H. Wolpert, “The lack of a priori distinctions between learning algorithms,”
Neural computation, vol. 8, no. 7, pp. 1341–1390, 1996.
[53] A. B. Tucker, Computer science handbook. CRC press, 2004.
[54] G. Dantzig, “The nature of mathematical programming,” Mathematical Pro-gramming Glossary, 2010.
[55] F. Hutter, H. H. Hoos, and K. Leyton-Brown, “Sequential model-based op-timization for general algorithm configuration,” in Learning and Intelligent Optimization, pp. 507–523, Springer, 2011.
[56] C. Thornton and F. Hutter and H. H. Hoos and K. Leyton-Brown, “Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classifi-cation Algorithms,” in Proc. of KDD-2013, pp. 847–855, 2013.
[57] I. D. A. G. Fraunhofer FIRST, “Data set IVa<motor imagery, small training sets>,” 2004.
[58] G. Dornhege, B. Blankertz, G. Curio, and K. Muller, “Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and mul-ticlass paradigms,” Biomedical Engineering, IEEE Transactions on, vol. 51, no. 6, pp. 993–1002, 2004.
[59] I. D. A. G. Fraunhofer FIRST, “Data set IVb <motor imagery, uncued clas-sifier application>,” 2004.
[60] B. B. group and F. FIRST, “Data sets 1 <motor imagery, uncued classifier application>,” 2007.
[61] B. Blankertz, G. Dornhege, M. Krauledat, K.-R. M¨uller, and G. Curio, “The non-invasive Berlin brain–computer interface: fast acquisition of effective per-formance in untrained subjects,” NeuroImage, vol. 37, no. 2, pp. 539–550, 2007.
[62] N. Z. University of Waikato, “Weka 3: Data Mining Software in Java,” 2015.
[63] J. Fourier, Theorie analytique de la chaleur, par M. Fourier. Chez Firmin Didot, p`ere et fils, 1822.
[64] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification. John Wiley
& Sons, 2012.
[65] A. Ajanki, “Example of k-nearest neighbour classification — Wikipedia, The Free Encyclopedia,” 2007.
[66] J. R. Quinlan, “Induction of decision trees,” Machine learning, vol. 1, no. 1, pp. 81–106, 1986.
[67] M. Strano and B. M. Colosimo, “Logistic regression analysis for experimental determination of forming limit diagrams,” International Journal of Machine Tools and Manufacture, vol. 46, no. 6, pp. 673–682, 2006.
[68] S. K. Palei and S. K. Das, “Logistic regression model for prediction of roof fall risks in bord and pillar workings in coal mines: an approach,”Safety science, vol. 47, no. 1, pp. 88–96, 2009.
[69] PmWiki, “CIS 520: Machine Learning: Fall 09 - 2D binary classification with Naive Bayes,” 2009.
[70] Cyc, “Graphic showing the maximum separating hyperplane and the margin
— Wikipedia, The Free Encyclopedia,” 2008.
[71] C. E. Metz, “Basic principles of ROC analysis,” in Seminars in nuclear medicine, vol. 8, pp. 283–298, Elsevier, 1978.
[72] C. J. van Rijsbergen, “A theoretical basis for the use of co-occurrence data in information retrieval,” Journal of documentation, vol. 33, no. 2, pp. 106–119, 1977.
[73] D. M. Powers, “Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation,” 2011.
[74] MathWorks Documentation, “clustergram - Compute hierarchical clustering, display dendrogram and heat map, and create clustergram object,” 2015.
Internet URLs were valid on January 7, 2016.
Appendices
Appendix A: Clustergrams
Figure4.1:ClustergramforSubject1fromDataset1.
Figure4.2:ClustergramforSubject2fromDataset1.
Figure4.3:ClustergramforSubject3fromDataset1.
Figure4.4:ClustergramforSubject4fromDataset1.
Figure4.5:ClustergramforSubject5fromDataset1.
Figure4.6:ClustergramforSubject1fromDataset2.
Figure4.7:ClustergramforSubject1/1afromDataset3.
Figure4.8:ClustergramforSubject2/1bfromDataset3.
Figure4.9:ClustergramforSubject3/1ffromDataset3.
Figure4.10:ClustergramforSubject4/1gfromDataset3.
Appendix B: Correlation Matrices
Figure4.11:PearsoncorrelationmatrixforSubject1fromDataset1.
Figure4.12:PearsoncorrelationmatrixforSubject2fromDataset1.
Figure4.13:PearsoncorrelationmatrixforSubject3fromDataset1.
Figure4.14:PearsoncorrelationmatrixforSubject4fromDataset1.
Figure4.15:PearsoncorrelationmatrixforSubject5fromDataset1.
Figure4.16:PearsoncorrelationmatrixforSubject1fromDataset2.
Figure4.17:PearsoncorrelationmatrixforSubject1fromDataset3.
Figure4.18:PearsoncorrelationmatrixforSubject2fromDataset3.
Figure4.19:PearsoncorrelationmatrixforSubject3fromDataset3.
Figure4.20:PearsoncorrelationmatrixforSubject4fromDataset3.
Figure4.21:AveragePearsoncorrelationmatrixoverallsubjects.
Appendix C: Optimized Parameter Values
k-Nearest Neighbours Subject 1: KNN = 24; lookupCacheSize = 2; searchTer-mination = 7 attribute eval = SymmetricalUncertAttributeEval; attribute search
= BestFirst;
k-Nearest Neighbours Subject 2: KNN = 21; searchTermination = 15 at-tribute eval = GainRatioAtat-tributeEval; atat-tribute search = GreedyStepwise;
k-Nearest Neighbours Subject 3: KNN = 51; lookupCacheSize = 0; searchTer-mination = 6 attribute eval = ReliefFAttributeEval; numNeighbours = 3; weight-ByDistance = False; sigma = 1; attribute search = BestFirst;
k-Nearest Neighbours Subject 4: KNN = 1; searchTermination = 124 at-tribute eval = CfsSubsetEval; atat-tribute search = GreedyStepwise;
k-Nearest Neighbours Subject 5: KNN = 20; searchTermination = 15; at-tribute eval = SymmetricalUncertAtat-tributeEval; atat-tribute search = GreedyStep-wise;
k-Nearest Neighbours Subject 6: KNN = 25; searchTermination = 203; at-tribute eval = CfsSubsetEval; atat-tribute search = GreedyStepwise;
k-Nearest Neighbours Subject 7: KNN = 13; lookupCacheSize = 0; searchTer-mination = 5; attribute eval = ReliefFAttributeEval; numNeighbours = 10; weight-ByDistance = False; sigma = 7; attribute search = BestFirst;
k-Nearest Neighbours Subject 8: KNN = 16; attribute search = NONE;
k-Nearest Neighbours Subject 9: KNN = 52; lookupCacheSize = 0; searchTer-mination = 4; attribute eval = CfsSubsetEval; attribute search = BestFirst;
k-Nearest Neighbours Subject 10: KNN = 16; lookupCacheSize = 0; searchTer-mination = 6; attribute eval = CorrelationAttributeEval; attribute search = Best-First;
C4.5 Decision Tree Subject 1: minNumObj = 13; confidenceFactor = 0.0159;
lookupCacheSize = 0; searchTermination = 5; attribute eval = SymmetricalUncer-tAttributeEval; attribute search = BestFirst;
C4.5 Decision Tree Subject 2: minNumObj = 6; confidenceFactor = 0.1929;
lookupCacheSize = 1; searchTermination = 7; attribute eval = CfsSubsetEval;
attribute search = BestFirst;
C4.5 Decision Tree Subject 3: minNumObj = 8; confidenceFactor = 0.2566;
threshold = 14.6656; attribute eval = GainRatioAttributeEval; attribute search
= GreedyStepwise;
C4.5 Decision Tree Subject 4: minNumObj = 10; confidenceFactor = 0.7056;
searchTermination = 21; attribute eval = CfsSubsetEval; attribute search = GreedyS-tepwise;
C4.5 Decision Tree Subject 5: minNumObj = 11; confidenceFactor = 0.3397;
centerData = True; maximumAttributeNames = 1; varianceCovered = 0.9280;
lookupCacheSize = 0; searchTermination = 10; attribute eval = PrincipalCompo-nents; attribute search = BestFirst;
C4.5 Decision Tree Subject 6: minNumObj = 4; confidenceFactor = 0.2527;
lookupCacheSize = 1; searchTermination = 8; attribute eval = CfsSubsetEval;
attribute search = BestFirst;
C4.5 Decision Tree Subject 7: minNumObj = 43; confidenceFactor = 0.3747;
folds = 2; minimumBucketSize = 1; threshold = 1.6254; attribute eval = OneR-AttributeEval; attribute search = Ranker;
C4.5 Decision Tree Subject 8: minNumObj = 14; confidenceFactor = 0.3500;
folds = 14; minimumBucketSize = 2; threshold = 9.2158; attribute eval = OneR-AttributeEval; attribute search = Ranker;
C4.5 Decision Tree Subject 9: minNumObj = 30; confidenceFactor = 0.0974;
centerData = True; maximumAttributeNames = -; varianceCovered = 0.9858;
lookupCacheSize = 2; searchTermination = 5; attribute eval = PrincipalCompo-nents; attribute search = BestFirst;
C4.5 Decision Tree Subject 10: minNumObj = 21; confidenceFactor = 0.0363;
folds = 10; minimumBucketSize = 5; threshold = 5.5871; attribute eval = OneR-AttributeEval; attribute search = Ranker;
Logistic Regression Model Subject 1: ridge = 5.2227; attribute search = NONE;
Logistic Regression Model Subject 2: ridge = 0.3669; numToSelect = 38;
attribute eval = CorrelationAttributeEval; attribute search = GreedyStepwise;
Logistic Regression Model Subject 3: ridge = 2.8541; lookupCacheSize = 2; searchTermination = 10; attribute eval = CfsSubsetEval; attribute search = BestFirst;
Logistic Regression Model Subject 4: ridge = 8.3614E-7; numNeighbours = 44; weightByDistance = False; sigma = 1; threshold = 3.0904; attribute eval = ReliefFAttributeEval; attribute search = GreedyStepwise;
Logistic Regression Model Subject 5: ridge = 0.0246; folds = 8; minimum-BucketSize = 2; threshold = 2.8797; attribute eval = OneRAttributeEval; at-tribute search = Ranker;
Logistic Regression Model Subject 6: ridge = 2.0988; threshold = 17.7142;
attribute eval = GainRatioAttributeEval; attribute search = GreedyStepwise;
Logistic Regression Model Subject 7: ridge = 9.9170; attribute search = NONE;
Logistic Regression Model Subject 8: ridge = 0.7175; numNeighbours = 15;
weightByDistance = False; sigma = 4; lookupCacheSize = 2; searchTermination
= 10; attribute eval = ReliefFAttributeEval; attribute search = BestFirst;
Logistic Regression Model Subject 9: ridge = 3.6800; folds = 11; mini-mumBucketSize = 8; threshold = 0.2781; attribute eval = OneRAttributeEval;
attribute search = Ranker;
Logistic Regression Model Subject 10: ridge = 0.7175; numNeighbours = 15;
weightByDistance = False; sigma = 4; lookupCacheSize = 2; searchTermination
= 10; attribute eval = ReliefFAttributeEval; attribute search = BestFirst;
Multilayer Perceptron Network Subject 1: learningRate = 0.2969; momen-tum = 0.8217; hiddenLayers = o; folds = 13; minimumBucketSize = 51; lookup-CacheSize = 2; searchTermination = 4; attribute eval = OneRAttributeEval; at-tribute search = BestFirst;
Multilayer Perceptron Network Subject 2: learningRate = 0.6786; momen-tum = 0.1688; hiddenLayers = t; attribute search = NONE;
Multilayer Perceptron Network Subject 3: learningRate = 0.3319; momen-tum = 0.7439; hiddenLayers = t; threshold = 12.1851; attribute eval = GainRa-tioAttributeEval; attribute search = GreedyStepwise;
Multilayer Perceptron Network Subject 4: learningRate = 0.5675; momen-tum = 0.1054; hiddenLayers = t; lookupCacheSize = 2; searchTermination = 8;
attribute eval = CfsSubsetEval; attribute search = BestFirst;
Multilayer Perceptron Network Subject 5: learningRate = 0.2185; momen-tum = 0.5771; hiddenLayers = a; folds = 6; minimumBucketSize = 49; threshold
= 9.8556; attribute eval = OneRAttributeEval; attribute search = Ranker;
Multilayer Perceptron Network Subject 6: learningRate = 0.2126; momen-tum = 0.3269; hiddenLayers = i; folds = 2; minimumBucketSize = 11; threshold = 9.1441; attribute eval = OneRAttributeEval; attribute search = GreedyStepwise;
Multilayer Perceptron Network Subject 7: learningRate = 0.4645; momen-tum = 0.1866; hiddenLayers = o; folds = 2; minimumBucketSize = 1; threshold = 3.5890; attribute eval = OneRAttributeEval; attribute search = GreedyStepwise;
Multilayer Perceptron Network Subject 8: learningRate = 0.7832; momen-tum = 0.1490; hiddenLayers = t; attribute search = NONE;
Multilayer Perceptron Network Subject 9: learningRate = 0.6786; momen-tum = 0.1688; hiddenLayers = t; attribute search = NONE;
Multilayer Perceptron Network Subject 10: learningRate = 0.5633; mo-mentum = 0.7133; hiddenLayers = a; attribute search = NONE;
Naive Bayes Subject 1: searchTermination = 81; attribute eval = CfsSubsetE-val; attribute search = GreedyStepwise;
Naive Bayes Subject 2: searchTermination = 570; attribute eval = CfsSub-setEval; attribute search = GreedyStepwise;
Naive Bayes Subject 3: searchTermination = 570; attribute eval = CfsSub-setEval; attribute search = GreedyStepwise;
Naive Bayes Subject 4: threshold = 0.2939; attribute eval = CorrelationAt-tributeEval; attribute search = Ranker;
Naive Bayes Subject 5: centerData = True; maximumAttributeNames = 272;
varianceCovered = 0.9964; lookupCacheSize = 0; searchTermination = 2; at-tribute eval = PrincipalComponents; atat-tribute search = BestFirst;
Naive Bayes Subject 6: searchTermination = 161; attribute eval = CfsSub-setEval; attribute search = GreedyStepwise;
Naive Bayes Subject 7: centerData = False; maximumAttributeNames = -1; varianceCovered = 0.9598; lookupCacheSize = 0; searchTermination = 3; at-tribute eval = PrincipalComponents; atat-tribute search = BestFirst;
Naive Bayes Subject 8: centerData = True; maximumAttributeNames = 45;
varianceCovered = 0.9972; lookupCacheSize = 2; searchTermination = 3; at-tribute eval = PrincipalComponents; atat-tribute search = BestFirst;
Naive Bayes Subject 9: centerData = True; maximumAttributeNames = 442;
varianceCovered = 0.5276; lookupCacheSize = 1; searchTermination = 6; at-tribute eval = PrincipalComponents; atat-tribute search = BestFirst;
Naive Bayes Subject 10: centerData = True; maximumAttributeNames = 180;
varianceCovered = 0.9632; threshold = 4.5383; attribute eval = PrincipalCompo-nents; attribute search = GreedyStepwise;
Random Forest Subject 1: numTrees = 57; features HIDDEN = False; num-Features = 0; depth HIDDEN = False; maxDepth = 0; folds = 4; minimum-BucketSize = 10; threshold = 5.4648; attribute eval = OneRAttributeEval; at-tribute search = Ranker;
Random Forest Subject 2: numTrees = 131; features HIDDEN = False; num-Features = 0; depth HIDDEN = False; maxDepth = 0; lookupCacheSize = 2;
searchTermination = 3; attribute eval = InfoGainAttributeEval; attribute search
= BestFirst;
Random Forest Subject 3: numTrees = 127; features HIDDEN = False; num-Features = 0; depth HIDDEN = True; maxDepth = 16; folds = 13; minimum-BucketSize = 14; threshold = 3.5659; attribute eval = OneRAttributeEval; at-tribute search = Ranker;
Random Forest Subject 4: numTrees = 128; features HIDDEN = True; num-Features = 8; depth HIDDEN = True; maxDepth = 9; lookupCacheSize = 2;
searchTermination = 4; attribute eval = CfsSubsetEval; attribute search = Best-First;
Random Forest Subject 5: numTrees = 8; features HIDDEN = True; num-Features = 5; depth HIDDEN = False; maxDepth = 0; threshold = 17.3189;
attribute eval = InfoGainAttributeEval; attribute search = GreedyStepwise;
Random Forest Subject 6: numTrees = 51; features HIDDEN = True; num-Features = 31; depth HIDDEN = False; maxDepth = 0; numNeighbours = 15;
sigma = 2; lookupCacheSize = 2; searchTermination = 3; attribute eval = Relief-FAttributeEval; attribute search = BestFirst;
Random Forest Subject 7: numTrees = 189; features HIDDEN = True; num-Features = 5; depth HIDDEN = False; maxDepth = 0; attribute search = NONE;
Random Forest Subject 8: numTrees = 158; features HIDDEN = True; num-Features = 8; depth HIDDEN = True; maxDepth = 10; attribute search = NONE;
Random Forest Subject 9: numTrees = 8; features HIDDEN = False; num-Features = 0; depth HIDDEN = False; maxDepth = 0; numNeighbours = 2;
sigma = 1; searchTermination = 420; attribute eval = ReliefFAttributeEval; at-tribute search = GreedyStepwise;
Random Forest Subject 10: numTrees = 174; features HIDDEN = False;
numFeatures = 0; depth HIDDEN = True; maxDepth = 4; lookupCacheSize = 0;
searchTermination = 3; attribute eval = InfoGainAttributeEval; attribute search
= BestFirst;
Support Vector Classifier Subject 1: c = 1.1213; filterType = 1; kernel
= RBFKernel; gamma = 0.0027; numNeighbours = 61; sigma = 2; lookup-CacheSize = 1; searchTermination = 2; attribute eval = ReliefFAttributeEval;
attribute search = BestFirst;
Support Vector Classifier Subject 2: c = 0.5199; filterType = 0; kernel = PolyKernel; exponent = 1.3900; attribute search = NONE;
Support Vector Classifier Subject 3: c = 1.2166; filterType = 2; kernel = PolyKernel; exponent = 3.4430; searchTermination = 54; attribute eval = Cfs-SubsetEval; attribute search = GreedyStepwise;
Support Vector Classifier Subject 4: c = 0.5196; filterType = 0; kernel = PolyKernel; exponent = 2.1511; threshold = 10.3495; attribute eval = Symmetri-calUncertAttributeEval; attribute search = GreedyStepwise;
Support Vector Classifier Subject 5: c = 0.5710; filterType = 2; kernel = PolyKernel; exponent = 1.1682; folds = 9; minimumBucketSize = 11; threshold = 17.0519; attribute eval = OneRAttributeEval; attribute search = GreedyStepwise;
Support Vector Classifier Subject 6: c = 1.1873; filterType = 1; kernel = RBFKernel; gamma = 0.0059; attribute search = NONE;
Support Vector Classifier Subject 7: c = 1.2884; filterType = 0; kernel = RBFKernel; gamma = 0.8826; folds = 11; minimumBucketSize = 4; threshold = 2.8913; attribute eval = OneRAttributeEval; attribute search = Ranker;
Support Vector Classifier Subject 8: c = 0.6114; filterType = 0; kernel = PolyKernel; exponent = 1.9743; attribute search = NONE;
Support Vector Classifier Subject 9: c = 1.4620; filterType = 0; kernel = Puk; sigma = 4.2011; omega = 0.6600; attribute search = NONE;
Support Vector Classifier Subject 10: c = 1.0273; filterType = 0; kernel = PolyKernel; exponent = 2.2284; attribute search = NONE;
Appendix D: Scripts and Other Materials
The Matlab scripts and other materials that were created in the making of this thesis can be found in an open source online repository at:
https://github.com/madism/ECMLA
It contains the following items:
• README.md- contains the guidelines for running the scripts and the changelog.
• Dataset name converter.m - script for preprocessing the specified dataset
• Dataset name converter.m - script for preprocessing the specified dataset