IoT-based Platform with Big Data using Apache Kylin: Air Quality Monitoring System (AQMS)

Younten Tshering, Suyogya Ratna Tamrakar, Shubhangini Gontia, Smrity Baral

Apache Kylin, Big Data, HBase, IoT, OLAP, REST API, Tableau.

Air quality has become a big concern as the quality is reducing exponentially. PM2.5 particle size is directly linked to their potential for causing health problems. There are many IoT devices such as the PM2.5 sensor that connects to the internet and make people aware of the problem. From such devices, there are lots of data generated every second and in multiple locations which contribute to Big Data. With Big Data there are lots of challenges from storing to processing it. Hence, Apache Kylin was used to overcome the challenges of big data processing. For the dashboard display of the air quality data from the sensor, we connect Kylin and Tableau with JDBC/ODBC driver. The proposed system mostly deals with Online Analytical Processing (OLAP) on the Hadoop platform using Apache Kylin. Kylin creates OLAP cubes from the data in the hive and stores the cubes in HBase for analysis. As we implemented, we had some problems with Tableau and Kylin getting to connect since JDBC drivers were not compatible and for ODBC we had to contact the vendor. In the end, we have implemented the REST API to connect to Kylin and get the required data. Therefore, we developed a web app with Spring Boot which will show all the required details related to air quality, and accordingly, people can take preventive measures. Solving big data issues was a big challenge, so in our application, we have only used PM2.5 as a factor to indicate the air quality of the given location. This has made the scope of application less significant. Some gaps can be fulfilled by extending this application in the future. But still, we believe that our system will provide some convenient information to the users, making people conscious about the quality of air.

https://doi.org/10.31871/WJRR.13.3.12