Publicly Available Datasets For Electric Load Forecasting

A (hopefully eventually) complete listing of the most popular electric LF datasets

Why?

We found it difficult to find suitable datasets in the flood of information. So we came up with the idea of doing a proper search and making the results available to the public.

What?

Based on a sample set of representative publications, relevant, publicly accessible data sets were extracted, structured and analyzed. The details of the search can be found in the scientific publication: https://doi.org/10.15488/17659

Improvements? 🤝

We are happy about any kind of cooperation, feedback or extension to make the list even more valuable for other scientists. So feel free to expand the list and initiate a pull request.

The list

ID	Abbrev	Name	Domain1	Resolution2	Features3	Duration4	Spanned years	Horizons5	Regions6	Type7	Links	Access8
1	ISO-NE	New England Independent System Operator	S	60	E	108	2003-2014	❌✔️✔️❌	✔️	📦	🔗Link	🔓
2	NYISO	New York Independent System Operator	S	5	E	264	2001-2023	✔️✔️✔️❌	✔️	📦	🔗Link	🔓
3	PJM	PJM Hourly Energy Consumption	S	60	E	240	1998-2018	❌✔️✔️✔️	✔️	📦	🔗Link	🔓
4	CIF	CIF 2016 competition dataset	?	d,m,y	Undef.	8-909	unknown	❌❌✔️✔️	❌	📦	🔗Link	🔓
5	GEFCOM14	GEFCom 2014	S	60	E, W, T, PV	10	2021	❌✔️❌❌	❌	📦	🔗Link	🔓
6	EUNITE	EUNITE 2001	S	30	E, T, H	24	1997-1999	❌✔️✔️❌	❌	📦	🔗Link	🔓
7	ENTSO-E	ENTSO-E electric load dataset	S	60	E	<=288	till 2015	❌✔️✔️✔️	✔️	📦	🔗Link	🔓
299	EWELD	Large-Scale Industrial and Commercial Load Dataset in Extreme Weather Events	I	15	E, W, xW	<=74	2016-2022	✔️✔️✔️✔️	✔️ (386)	📦	🔗Link	🔓
289	WPuQ	Electrical single-family house and heat pump load	R	<1	E	30	2018-2020	❌✔️✔️❌	✔️ (38)	📦	🔗Link	🔓
329	PanETESA	Panama ETESA	S	60	E, W, H	66	2015-2020	❌✔️✔️✔️	❌	📦	🔗Link	🔓
389	REFIT	REFIT: Electrical Load Measurements	R	8sec	E	20	2013-2015	✔️✔️✔️❌	✔️(20)	📦	🔗Link1 🔗Link2	🔓
399	ECD-UY	household electricity consumption dataset of Uruguay	S, R	1-15	E	11-23	2019-2020	✔️✔️❌❌	✔️(9)	📦	🔗Link1 🔗Link2	🔓
409	IDEAL	IDEAL UK Household Energy Dataset 255	R	1-12sec	E, W, T	23	2019-2020	✔️✔️✔️❌	✔️(255)	📦	🔗Link1 🔗Link2	🔓
419	HANOI-Res	Residential Apartments Dataset Hanoi, Vietnam (CAMaRSEC Project)	R	15	E, W, T	12	2020-2021	✔️✔️❌❌	✔️(49)	📦	🔗Link1 🔗Link2	🔓
429	UK-DALE	UK Domestic Appliance Level Electricity (UKERC EDC), Disaggregated (6s) and aggregated (1s)	R	1-6sec	E	5-53	2012-2017	✔️✔️✔️✔️	✔️(5)	📦	🔗Link1 🔗Link2 🔗Link3	🔓
449	ELMAS	Hourly electrical load profiles (18 aggregated curves: 1 for each industrial sector) [no individual profiles]	I	60	E, T	12	2018	❌✔️❌❌	❌	📦	🔗Link1	🔓
8	LCL	LCL Load Dataset (London Households)	R	30	E	12	2013	❌✔️❌❌	❌	📁	🔗Link	🔓
9	SET	Energy Consumption Dataset for Milano/Trento	S	10	E	<1	2013	✔️❌❌❌	❌	📁	🔗Link	🔓
10	BDG-Proj	Building Data Genome Project	S	60	E	12	unknown	❌✔️❌❌	✔️	📁	🔗Link	🔓
349	BDG-Proj2	Building Data Genome Project 2 (BDG2)	R	60	E	24	2016-2017	❌✔️✔️❌	✔️ (1636)	📁	🔗Link	🔓
11	IHPC	Individual Household power consumption	S	1	E	48	2006-2010	✔️✔️✔️✔️	❌	📁	🔗Link	🔓
12	GEFCOM12	GEFCom 2012	S	60	E, W, T	42	2004-2008	❌✔️✔️❌	❌	📁	🔗Link	🔓
13	OPSD-TS	Open Power System Data TS	S	15-60	E, PV, W	148	2005-2019	✔️✔️✔️✔️	✔️	📁	🔗Link	🔓
279	OPSD-HH	Open Power System Data Household Data	R, I	1-60	E, PV	diff	2012-2019	✔️✔️✔️✔️	✔️	📁	🔗Link	🔓
14	ELD	ElectricityLoadDiagrams20112014	S	15	E	36	2011-2014	✔️✔️✔️✔️	❌	📁	🔗Link1 🔗Link2	🔓
15	ENERTALK	ENERTALK Dataset Korea (household)	S	15 hz	E	12	2016	✔️✔️❌❌	❌	📁	🔗Link	🔓
16	S-TSO	Spanish Transmission Service operator (TSO)	H	60	>25	24	2017-2018	❌✔️✔️❌	❌	📁	🔗Link	🔓
269	CER	CER Smart Metering Project	R,I	30	E	18	2009-2010	❌✔️✔️❌	✔️(5237)	📁	🔗Link	📧
309	DEDDIAG	domestic electricity demand dataset (individual appliances in Germany)	R	1Hz	E	2-44	2011-2014	✔️✔️✔️❌	✔️(14)	📁	🔗Link1 🔗Link2	🔓
319	AusSmartGrid	Electricity Use Interval Reading	R	60	E	?	2010-2014	❌✔️✔️❌	✔️	📁	🔗Link	🔓
359	UK-GRID	Electricity consumption UK 2009-2024	S	30	E	180	2009-2024	❌✔️✔️✔️	❌	📁	🔗Link	🔓
369	HoustonRes	Houston Residential power usage (one house)	R	60	E, W	49	2016-2020	❌✔️✔️❌	❌	📁	🔗Link	🔓
379	CU-BEMS-Bangkok	Bangkok CU-BEMS, smart building energy and IAQ data	R	1	E, W	18	2018-2019	✔️✔️✔️❌	❌	📁	🔗Link	🔓
439	5359 VEA loadd	5359 industrial VEA load profiles	I	15	E	12	2016	✔️✔️❌❌	✔️(5359)	📁	🔗Link	🔓
459	Germ-Industry-16	20 industrial load profiles for german plants	I	15	E	12	2016	✔️✔️❌❌	❌	📁	🔗Link	🔓
469	Germ-Industry-17	30 industrial load profiles for german plants	I	15	E	12	2017	✔️✔️❌❌	❌	📁	🔗Link	🔓
17	RTE-France	RTE France	S	30	E	12	2012-2020	❌✔️❌❌	✔️	🌐	🔗Link	🔓
18	AEMO	Australian Energy market operator	H	60	E	12	2013	❌✔️❌❌	✔️	🌐	🔗Link	🔓
19	IESO-O	IESO Ontario	H	60	E, P	20+	2022-2023	❌✔️✔️❌	❌	🌐	🔗Link	🔓
20	AESO	Alberta Electric Sys. Op. Electrical Load Dataset	S	60	E	132	2005-2016	❌✔️✔️✔️	❌	🌐	🔗Link	🔓
21	PPS	Polish power system	S	15-60	E	120+	2013- now	✔️✔️✔️✔️	❌	🌐	🔗Link	🔓
22	AUSGRID	Ausgrid: Distribution zone substation	S	15	E	204	2005-2022	✔️✔️✔️✔️	✔️(>100)	🌐	🔗Link	🔓
23	KPX	KPX Korea	H	5	E	240	2003-now	✔️✔️✔️✔️	❌	🌐	🔗Link	🔓
24	ADMIE	Independent Electricity Transmission Operator	S	60	E	120+	2011-now	❌✔️✔️✔️	✔️	🌐	🔗Link	🔓
25	Pecan	Pecan Street dataset	S	15	E, W	24	2017-2018	✔️✔️✔️❌	✔️	🌐	🔗Link	🔓
339	Cal-ISO	California ISO Hourly Load Data	S	60	E	100+	2014-now	❌✔️✔️✔️	✔️	🌐	🔗Link1 🔗Link2	🔓

Legend

¹Domain: Either system level load (S), residential load (R) or Industry (I)

²Resolution: In minutes, if not other stated (d=day, m=month, y=year, hz=1sec)

³Features: Electricity (E), Weather (W), Extreme Weather Events, e.g. heat periods and taifune (xW), Temperature (T), Photovoltaic production (PV), Holiday features (H), Price (P)

⁴Duration: in number of months

⁵Forecasting-Horizons for modeling applicable: Very Short Term (VST), Short Term (ST), Medium Long Term (MT), Long Term (LT)

⁶Dataset records multiple regions / consumers separately (e.g. buildings, cities, countries) or disaggregated single loads available. Numbers in brackets indicate the number of regions / consumers / loads

⁷Type: Either 📦 = a collection (accumulation of datasets), 📁=a file or achive or 🌐=a data platform / API

⁸Access: Either 🔓 = can be accessed directly (no login, no request), 📧 = written application / request has to be sent first

⁹ not part of the original Paper, added later (only here)

for further details, take a look at the publication below ⤵️

In a Rush? Use Our Python-Package:

Installation

pip install padelf

Use

import padelf

# Load a dataset - one line, sensible defaults
df = padelf.get_dataset("OPSD")

# show some lines
print(df.head())

Output

                          consumption_kW     DE_wind_onshore_generation_actual
datetime
2015-01-01 00:00:00+00:00       41209.0      7568.0
2015-01-01 01:00:00+00:00       40029.0      7666.0
2015-01-01 02:00:00+00:00       38891.0      7637.0

See padelf-pip for more details.

Overwhelmed? Use Our Interactive Search Tool:

Finding the right dataset for your task can be hard. Use our PADELF Search Online-Dashboard to filter the above table on-the-fly. Simply specify your required filters and get the subset that is useful for you.

How to cite

If this work has helped you with your scientific work, we would appreciate a proper mention. ❤️

Our citation recommendation is:

Baur, L.; Chandramouli, V.; Sauer, A.: Publicly Available Datasets For Electric Load Forecasting – An Overview. In: Herberger, D.; Hübner, M. (Eds.): Proceedings of the CPSL 2024. Hannover : publish-Ing., 2024, S. 1-12. DOI: https://doi.org/10.15488/17659

BibTeX entry

@inproceedings{baur2024datasets,
  author    = {Baur, Lukas and Chandramouli, Vignesh and Sauer, Alexander},
  title     = {Publicly Available Datasets For Electric Load Forecasting – An Overview},
  booktitle = {Proceedings of the CPSL 2024},
  editor    = {Herberger, D. and Hübner, M.},
  location  = {Hannover},
  publisher = {publish-Ing.},
  year      = {2024},
  pages     = {1--12},
  doi       = {10.15488/17659}
}

How to contribute

See how to contribute in the CONTRIBUTING.md

Acknowledgements

💰 We'd like to thank the German Federal Ministry of Economic Affairs and Climate Action (BMWK) and the project supervision of the Project Management Jülich (PtJ) for the project „FlexGUIde“ which allowed for the work.

💡 We would also like to thank an anonymous reviewer who suggested publishing the datasets not only in the above-mentioned publication but also as a repository.

👨‍🎓 We would like to thank K. Kunkel, whose master's thesis contributed greatly to the expansion of the initial dataset collection.

🧨 We'd like to thank G. Schmid, who contributed towards the overall vision to path a way through the dataset jungle by working on an interactive search dashboard and the pip-package.