Evaluation of autoregressive time series prediction cross-validation

Views:

Category: Education

Presentation Description

Cross-Validation is a validation technique used to explain how well the estimated values from a fitted statistical model will generalize to the explanatory variables under study. The standard procedure for model validation is the K-fold CV as in Regression Analysis and classification problem. This blog discusses a note on the validity of Cross-Validation for evaluating autoregressive Time Series Prediction. Statswork offers statistical services as per the requirements of the customers. When you Order statistical Services at Statswork, we promise you the following – Always on Time, outstanding customer support, and High-quality Subject Matter Experts. Contact Us: Website: www.statswork.com Email: info@statswork.com UnitedKingdom: +44-1143520021 India: +91-4448137070 WhatsApp: +91-8754446690

Presentation Transcript

slide 1:

EVALUATION OF AUTOREGRESSIVE TIME SERIES PREDICTION USING VALIDITY OF CROSS-VALIDATION Research paper Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Tags: Statswork | Cross-Validation | Time Series Prediction | Time Series Analysis | Statistical Model | Regression Analysis | Serial Correlation | Statistical Models | Time Series Data

slide 2:

Cross-Validation CV Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved CV is a validation technique used to explain how well the estimated values from a fitted statistical model will generalize to the explanatory variables under study.

slide 3:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Why Cross Validation technique is different for time series data Standard procedure for model validation is the K- fold CV as in Regression Analysis and classification problem. However for predicting the time series data this K-fold is not a straightforward because of the non-stationarity and serial correlation present in the data. K-fold CV is applicable for the autoregressive models with uncorrelated errors and have wide scope in predicting the model using Machine Learning methods.

slide 4:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Types of Cross Validation K-fold Cross Validation for time series for purely autoregressive models particularly for the dependent case instead of applying to the independent case. Hold-out cross-validation technique is performed by splitting the data into training and test set. K-fold Cross Validation Hold-out Cross Validation 01 02

slide 5:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved ILLUSTRATIVE EXAMPLE

slide 6:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Model selection adopted in that paper is 5-fold Cross Validation using leave-one-out non- dependent and out-of-sample evaluation methods using AR1 to AR5 models and residuals have been taken care for the uncorrelated errors using Serial Correlation. Three different experiments are carried out to forecast the AR model using Monte Carlo simulation. The monthly seasonality from the ACF and PACF plots for the data considered for Monte- Carlo simulation is depicted in the following figure and the results of the experiment is discussed. Applicability of the procedure using the yearly sun spot data available in R. The data involves 289 observations recorded from 1700 to 1988 and are depicted in the following figure. 01 02 03 04 Contd...

slide 7:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Results

slide 8:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Auto correlation Relevant auto correlation is captured using Ljung-Box test with the residual series of 1560 model configurations. Out of 1560 763 satisfies the Box test and the configuration which have minimum root mean square error is chosen. The 5-fold CV yields an error rate of 2.247 and out-of-sample OOS method yields 2.281 and the CV procedure is not an over fitting model and the results are tabulated. Method RMSE 5 fold CV 2247 005 2281 Discussions

slide 9:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Discussed the applicability of the K-fold Cross Validation procedure for the purely AR models beyond the usual practice and it is found to be useful when the residuals in the model is uncorrelated. From the Monte-Carlo simulation the K-fold classification yields better result than out-of- sample method for time series data and the real-time data explains the application of this using AR model with uncorrelated errors. The over-fitting and under-fitting of the model is accessed using the Ljung-Box test and it results in there is no such issues occurred in the model. Examines the applicability of the K-fold cross validation using neural network model with a monte-carlo simulations it can also be extended to the other regression models such as random forest and other machine learning techniques. The interesting point is that if we use whatever regression model for these kinds of Time Series Data this method still proves the K-fold cross validation is the best method. 0 1 0 2 0 3 0 4 0 5 Summary

slide 10:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Statswork Lab Statswork.com www.statswork.com

slide 11:

Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved Freelancer Consultant Guest Blog Editor UK : +44-1143520021 INDIA : +91-4448137070 EMAIL ADDRESS infostatswork.com GET IN TOUCH WITH US PHONE NUMBER CONTACT hrworkfoster.com