Elizabeth Willard · blog

A Small Survey of Very Short-Term Load Forecasting Methods

850 words 4 min read #machine-learning#deep-learning#forecasting
Categories technical

Source: Forecasting very short-term power load with hybrid interpretable deep models

“In this study, using a public dataset, we identified the most significant attributes of VSTLF and then trained four end-to-end deep models based on Convolutional Neural Networks (CNN), LSTM and an attention mechanism. Experiments demonstrated that the hybrid model comprised of CNN and Bidirectional LSTM (BiLSTM) outperformed the other models. Moreover, we utilized the Shapley Additive Explanations (SHAP) method to interpret the model and evaluate the feature importance. The contributions of our work are threefold: (1) we understand and predict a very short power load with high performance and achieve state-of-the-art; (2) the hybrid model, BiLSTM and CNN without any attention mechanism, is validated to be effective and efficient in Very Short-Term power Load Forecasting, and (3) we rank feature significances with the SHAP method, which may shed light on future power scheduling.”

The driving force behind this phenomenon is that they can provide powerful nonlinear modelling capabilities by cascading multiple hidden layers; hence, they can more effectively fit complex correlations across time, weather, and seasonal attributes

paper_models

Datasets Used:

  • Real Load History
  • 3 Cities
  • January 2015 - June 2020
  • Once per hour
    • 48048 samples no missing values
  • 16 attributes:
    • nat_demand (load history)
    • temperature per city
    • humidity per city
    • wind speed per city
    • precipitation per city
    • school
    • holiday_IS
    • holiday y (nat_demand) = holiday + holiday_IS + school + precipitation_per_city + humidity_per_city + historical_nat_demand + temperature_per_city

Pearson Correlation Analysis to evaluate every individual attributes contribution to power demand in the entire dataset

Normalization of features:

  • Different features have different ranges so they normalize them so all features are on the same interval which accelerates model convergence
  • Smooths the optimization curve to prevent it from being trapped in local optima
  • [0,1] target range

Modeling Direction:

  • Bidirectional models based on the CNN, LSTM, and attention mechanism

BiLSTM

  • Bidirectional Long Short-Term Memory network model
  • Can model both long-term and short-term correlations simultaneously
  • long-run attributes: precipitation and school (why is school a long-run attritbute?)
  • short-run attributes: temperature and wind speed Three Layers:
  1. Input Layer:
    • Accepted preprocessed time-series load data and computes value through timing weight matrices
  2. LSTM Cells accept input layer value computations
    • Notably, each LSTM cell undergoes an iterative process to update the hidden state of load demand / hour
      bilstm_cell
  3. Ouput Layer:
    • Fully connected linear layer

CNN-BiLSTM model

  • This time they built upon their previous BiLSTM model by modifying the input layers of the model by adding a CNN layer to precede the entirety of the BiLSTM model. The convolutional operation will ideally capture local and spatiotemporal patterns
  • The ReLU Function is “expected to improve nonlinearity of the model and learn complicated temporal patterns”

cnn_bilstm_model

The simple cnn+bilstm models and bilstm models showed the best performance out of the four model configurations they tried with and without attention. So I chose to focus on the most applicable models for experimentation.

Source: UniLF A Novel Short-term Load Forecasting Model Uniformly Considering Various Features from Multivariate Data

The Effect of Input Length on Prediction Accuracy in Short-Term Multi-Step Electricity Load Forecasting: A CNN-LSTM Approach

Interpretable Short-term Electricity Load Forecasting Considering Small Sample Heatwaves

Hybrid predictive deep learning with optimization algorithm to forecast precise electricity demand for 30minute intervals

Data Preparation:

  • Look at max and minimum ranges of various parameters using historical data
  • January 1 2021 - December 31 2021 daterange of real-time load dataset
    • Undergoes preprocessing to reconstruct electrical characteristics?
    • Zero-mean normalization applied to load and temperature data to standardize
    • Redundant information can hinder accurate pattern extraction for load forecasting

Feature Engineering: Principal Component Analysis Network (PCANet) identifies relevant features DeepWalk Gated Recurrent Unit Model (DWGRU) captures temporal dependencies from historical sequences All features are dynamically combined using attention mechanism

Actual Forecast Model: Hybrid Sampling and Self Attention with Deep Neural Network (HSSA-DNN) is employed to forecast 30-minute peak loads efficiently, utilizing Improved Moth Flame Optimization

Implemented using Matlab

Accuracy Assessment Metrics:

  • MAPE
  • MAE
  • RMSE block diagram of work from paper

Dataset:

  • Year
  • Month
  • Date
  • Day
  • Hour
  • Weekday
  • Working/Holiday
  • Dewpoint
  • Dry bulb temp
  • hourly load temporal granularity: each hour from 01/01/2021 - 12/31/2021 The dataset also

TNEB load dispatch centre website from Tamil Nadu districts of Madurai and Chennai

PCANet used for feature extraction process

  • streamlined and faster variant of CNN designed for image classification
  • employs *cascaded principal component analysis (look this up later?)
  • binary hiashing
  • blockwise histograms across three layers
  • Key component: PCA filter to extract highest energy eigenvector from input training data and convolution filters which capture primary variance of input — I need to look more into PCA

Energy Load Forecasting with Machine Learning: Models, Metrics, and Future Directions