The Retrieveoperator, with the data set name, displays. Start a new process and save it, for example as Cross validation process, in the processes folder ofGetting Started.ĭrag the customer-churn-data data set you imported in Part 1 onto the Process view. While you can cut-and-paste or disable operators from the process you have already built, you can also quickly recreate the process (and get the extra practice). Now, you will apply the model to examples with a known outcome to determine how accurate it actually is.įor this step, start a new process. You then created a model (set of rules) based on that data. You have filtered the training set to use only those examples in which churn is known. What they all have in common, though, is that they were generated on similar data sets using the same algorithm. In the end, each iteration tests a single model, but after all are complete, the process created and tested 10 different models. The following illustrates steps 1 through 4: Repeat steps 2 and 3, each time using a different subset for training.Īverage the performance values of each iteration and return the average. Test the model on the one remaining set and remember the performance. Split the data into 10 equally sized, non-overlapping sets. For example, for 10 different combinations the process would be: The "de facto" standard and statistically sound method for evaluating machine learning models is to use cross validation.Ĭross validation divides the training data set into a selected number of splits (10 in this tutorial), measures performance for each iteration, and then averages the performance values from each test.
You must be careful not to do this testing on data that has already been used for training - otherwise, you end up overestimating the prediction quality. You can do so by trying out the model predictions for customers for where you already know the true outcome. It is crucially important to evaluate a model, in this case, to test how accurately the model predicts churn. Here is a video to Part 5: RapidMiner Intro Part 5: Evaluating the Model
0 kommentar(er)
