Ensemble Learners

In addition to making predictions using individual classification algorithms, ML-Flex can also make predictions using various ensemble-learning methods. These methods combine evidence across multiple individual predictions to derive an aggregate prediction. If multiple data processors and/or algorithms are specified in the experiment settings, ensemble learners will aggregate evidence across all combinations of these; when only one combination exists, ensemble learning will not be performed.

Below is a short description of each ensemble-learning approach:

• Majority Vote = This method counts the number of times a given data instance was predicted for each class. The aggregate prediction favors the class that received the most votes. In situations where multiple classes received the same number of votes, the predicted class will be selected at random from those classes.
  
  
• Weighted Vote = In an attempt to place most emphasis on individual predictions that are most informative, this method assigns weights to individual predictions based on the AUC attained via nested cross validation for the relevant combination of data category and algorithms.
  
  
• Select Best = This method makes a prediction based on the individual prediction that received the highest weight (AUC) in nested cross validation.
  
  
• Max Probability = This method examines the probabilities for each class across all individual predictions: the class with the highest individual probability is selected.
  
  
• Mean Probability = This method examines the probabilities for each class across all individual predictions. The class with the highest mean (average) probability is selected.
  
  
• Weighted Mean Probability = This method is similar to Mean Probability but also assigns a weight (derived as in Weighted Vote) to each individual probability.
  
  
• Stacked = This method uses the probabilities from the individual predictions and trains a second-level classification algorithm to make aggregate predictions based on those values. Any classification algorithm can be used for the second-level predictions. By default, ML-Flex uses a Decision Trees algorithm for the second-level predictions; however, this is configurable (see Experiments/README).
  
  

Table of Contents

Introduction to ML-Flex

Prerequisites

Configuring Algorithms

Creating an Experiment File

List of Experiment Settings

Running an Experiment

List of Command-line Arguments

Executing Experiments Across Multiple Computers

Modifying Java Source Code

Creating a New Data Processor

Third-party Machine Learning Software

Integrating with Third-party Machine Learning Software

About Ensemble Learners