E-Solar: Una Herramienta para la evaluación del Recurso Solar basada en una arquitectura Big Data sobre un ambiente PySpark

Luis Eduardo Ordoñez Palacios; Víctor Bucheli Guerrero; Eduardo Caicedo Bravo

doi:10.14482/inde.43.01.456.089

Authors

Luis Eduardo Ordoñez Palacios Universidad del Valle (Colombia) https://orcid.org/0000-0001-5154-9472
Víctor Bucheli Guerrero Universidad del Valle (Colombia) https://orcid.org/0000-0002-0885-8699
Eduardo Caicedo Bravo Universidad del Valle (Colombia) https://orcid.org/0000-0003-0727-2917

DOI:

https://doi.org/10.14482/inde.43.01.456.089

Keywords:

Big data, Machine learning, MapR, PySpark, Solar radiation

Abstract

Over time, diverse researchers have created mathematical, statistical, and predictive models to evaluate solar resources. However, their implementation in technical tools restricts their usability for non-technical users. Additionally, data processing to estimate solar radiation often necessitates powerful hardware. This study introduces a Big data based tool that employs flat files and satellite images to estimate solar radiation in Colombia. A model was developed using machine learning techniques and various programming languages. It operates within MapR, a distribution of the Hadoop ecosystem with an extensive array of Big data capabilities and utilizes the PySpark API for parallel data processing within a computer cluster. The E-Solar tool, deployed on a web server, underwent assessment by professionals within the energy sector. Usability was analyzed, compliance with recent programming standards was confirmed, and profiles of interested users were identified. The solar radiation data generated by the tool are pivotal for solar projects. Furthermore, the tool lends support to researchers and organizations in decision-making for the implementation of photovoltaic systems, as it offers pertinent information regarding the behavior of solar resources in Colombia.

References

X. Zheng, D. Streimikiene, T. Balezentis, A. Mardani, F. Cavallaro, y H. Liao, «A review of greenhouse gas emission profiles, dynamics, and climate change mitigation efforts across the key climate change players», Journal of Cleaner Production, vol. 234, pp. 1113-1133, oct. 2019, doi: 10.1016/j.jclepro.2019.06.140.

L. E. Ordoñez Palacios, V. Bucheli Guerrero, y H. Ordoñez, «Machine Learning for Solar Resource Assessment Using Satellite Images», Energies, vol. 15, n.o 11, Art. n.o 11, ene. 2022, doi: 10.3390/en15113985.

NOAA, «NOAA’s Office of Satellite and Product Operations», 2021. https://www.ospo.noaa.gov/Operations/GOES/13/index.html (accedido 30 de marzo de 2021).

«PySpark Overview — PySpark master documentation». Accedido: 7 de junio de 2024. [En línea]. Disponible en: https://spark.apache.org/docs/latest/api/python/index.html.

Apache Spark, «Overview - Spark 3.5.1 Documentation», 2024. https://spark.apache.org/docs/latest/ (accedido 7 de junio de 2024).

«Get started wirh Django», Django Project. Accedido: 8 de julio de 2024. [En línea]. Disponible en: https://www.djangoproject.com/

«JavaScript | MDN». Accedido: 8 de julio de 2024. [En línea]. Disponible en: https://developer.mozilla.org/es/docs/Web/JavaScript

L. E. Ordónez Palacios, «Dashboard E-Solar». 26 de noviembre de 2022. Accedido: 21 de marzo de 2023. [Python]. Disponible en: https://github.com/luise-phd/dashboard

NASA, «POWER | Data Access Viewer», 2021. https://power.larc.nasa.gov/data-access-viewer/ (accedido 21 de marzo de 2023).

Scikit-learn, «scikit-learn: machine learning in Python — scikit-learn 1.2.2 documentation», 2007. https://scikit-learn.org/stable/ (accedido 21 de marzo de 2023).

developers XGBoost, «XGBoost Documentation — xgboost 1.7.4 documentation», 2022. https://xgboost.readthedocs.io/en/stable/ (accedido 22 de febrero de 2023).

IDEAM, «Consulta y Descarga de Datos Hidrometeorológicos», 2020. http://dhime.ideam.gov.co/atencionciudadano/ (accedido 21 de marzo de 2023).

C. Wu et al., «China’s agricultural machinery operation big data system», Computers and Electronics in Agriculture, vol. 205, p. 107594, feb. 2023, doi: 10.1016/j.compag.2022.107594.

F. Rodríguez, A. Fleetwood, A. Galarza, y L. Fontán, «Predicting solar energy generation through artificial neural networks using weather forecasts for microgrid control», Renewable Energy, vol. 126, pp. 855-864, oct. 2018, doi: 10.1016/j.renene.2018.03.070.

M. K. Boutahir, Y. Farhaoui, M. Azrour, I. Zeroual, y A. El Allaoui, «Effect of Feature Selection on the Prediction of Direct Normal Irradiance | TUP Journals & Magazine | IEEE Xplore». Accedido: 7 de junio de 2024. [En línea]. Disponible en: https://ieeexplore.ieee.org/document/9832772

A. A. Tehrani, O. Veisi, B. V. Fakhr, y D. Du, «Predicting solar radiation in the urban area: A data-driven analysis for sustainable city planning using artificial neural networking», Sustainable Cities and Society, vol. 100, p. 105042, ene. 2024, doi: 10.1016/j.scs.2023.105042.

M. Cuesta, J. Constante, y D. Jijón, «Modelos de Predicción de Radiación Solar y Temperatura Ambiente mediante Redes Neuronales Recurrentes», Revista Técnica Energía, vol. 19, pp. 81-89, ene. 2023, doi: 10.37116/revistaenergia.v19.n2.2023.552.

L. Cárdenas, D. Hurtado, y R. Moreno, «Predicción de radiación solar mediante deep belief network», Revista Tecnura, vol. 20, p. 39, ene. 2016, doi: 10.14483/udistrital.jour.tecnura.2016.1.a03.

Vinod, R. Kumar, y S. K. Singh, «Solar photovoltaic modeling and simulation: As a renewable energy solution», Energy Reports, vol. 4, pp. 701-712, nov. 2018, doi: 10.1016/j.egyr.2018.09.008.

S. R. Pendem y S. Mikkili, «Modeling, simulation and performance analysis of solar PV array configurations (Series, Series–Parallel and Honey-Comb) to extract maximum power under Partial Shading Conditions», Energy Reports, vol. 4, pp. 274-287, nov. 2018, doi: 10.1016/j.egyr.2018.03.003.

N. Abdullahi, C. Saha, y R. Jinks, «Modelling and performance analysis of a silicon PV module», Journal of Renewable and Sustainable Energy, vol. 9, n.o 3, p. 033501, may 2017, doi: 10.1063/1.4982744.

Apache Hive, «LanguageManual - Apache Hive - Apache Software Foundation», 2022. https://cwiki.apache.org/confluence/display/Hive/LanguageManual (accedido 4 de abril de 2023).

MapR, «HPE Ezmeral Data Fabric 7.2 Documentation», 2022. https://docs.datafabric.hpe.com/72/install.html (accedido 20 de marzo de 2023).

Apache Drill, «Documentation - Apache Drill», 2022. https://drill.apache.org/docs/ (accedido 4 de abril de 2023).

Hue, «Quick Start?:: Hue SQL Assistant Documentation», 2022. https://docs.gethue.com/quickstart/ (accedido 4 de abril de 2023).

Highcharts, «Highcharts Documentation | Highcharts», 2020. https://highcharts.com/docs/index (accedido 4 de abril de 2023).

«Apache Hadoop». Accedido: 8 de julio de 2024. [En línea]. Disponible en: https://hadoop.apache.org/

HOMER, «HOMER Pro - Microgrid Software for Designing Optimized Hybrid Microgrids», 2021. https://www.homerenergy.com/products/pro/index.html (accedido 9 de junio de 2021).