With the rapid development of IoT based home appliances, it has become a possibility that home owners share with Utilities in the management of home appliances energy consumption. Thus, the proposed work empowers home owners to manage their home appliances energy consumption and allow them to compare their consumption with respect to their local community total consumption. This serves as a nudge in consumer's behavior to schedule their home appliances operation according to their local community consumption profile and trend. Utilizing the same common communication infrastructure, it also allows the utilities on different consumption levels (community, state, country) to monitor and visualize the energy consumption in their respective grid segments on daily, monthly, and yearly basis. A high-speed distributed computing cluster based on commodity hardware with efficient big data mathematical algorithm is employed in this work. To achieve this, two big data processing paradigms are evaluated with a set of qualitative and quantitative metrics with subsequent recommendations. One million smart meter data is simulated to access individual homes. With the utilization of distributed storage and computing cluster for handling energy big data, the utilities can perform consumer load analysis and visualization on a scale of one million consumers. This helps the utilities in providing consumers a more accurate representation of how much energy they are consuming with greater granularity and with respect to their local community. Consumer and Utility centric queries are developed to create a web-based real time energy consumption management system presented in terms of dashboard charts, graphs, and reports that can be accessed by the consumer and utility providers remotely.


Computer Science

Research Center/Lab(s)

Center for High Performance Computing Research

Second Research Center/Lab

Intelligent Systems Center


This work was supported in part by the Faculty Research, Department of Computer Science and Engineering, American University of Sharjah, Sharjah, UAE, under Grant FRG 17-R-22, and in part by the Open Access Program from the American University of Sharjah.

Keywords and Phrases

Big data; energy management system; IoT; smart meter

International Standard Serial Number (ISSN)


Document Type

Article - Journal

Document Version

Final Version

File Type





© 2020 The Authors, All rights reserved.

Creative Commons Licensing

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.

Publication Date

28 Aug 2020