Real-Time Dynamic Data Control Mechanism Using Auto Rebalancing Strategy for IoT Applications

Hiren Dutta*, Parama Bhaumik**
* Associate Consultant, Tata Consultancy Service Ltd, Kolkata, India.
** Associate Professor, Department of Information Technology, Jadavpur University, Kolkata, India.
Periodicity:March - May'2018
DOI : https://doi.org/10.26634/jcom.6.1.14501

Abstract

Data ingestion is fast and data life becomes very short. When a user searches a product in a retailer website, deriving recommendations at that point of time makes sense to the business to cross-sell other related products. The customer pulse is need to be understood, and IoT is a key driver to understand the market. Online retailers are seeking a solution for sudden data surge scalability protection during special events (planned /unplanned) like festive seasons to support a sudden increase in concurrent user base. With the large concurrent user base, Microservice based solution and application deployment are gaining an edge and soon it will become a defacto standard. With the increase of cloud usage (Platform, User, Service, etc.), industries are looking at solutions for more and more low cost, high scalable application deployment platform (PaaS) to make it a win-win deal with cloud providers. In this paper, the authors have proposed an agent-based architecture for auto scaling of an application deployed on the premises or in PaaS based on data surge sense at minimal cost without compromising on performance and other non-functional requirements in real time. To address this research issue, microservice based Twitter sentiment analysis application is created to ingest realtime data. Then this approach will define the guidelines to auto scale (Lorido-Botran et al., 2014) up/down the application based on incoming data surge to define dynamic and adaptive resource utilization, which in turn reduces infrastructure and service utilization cost of the cloud-based platform.

Keywords

Microservice Architecture, Real Time Data Surge Protection, Adaptive Resource Utilization, Spout and Bolt Architecture.

How to Cite this Article?

Dutta, H., and Bhaumik,P. (2018). Real-Time Dynamic Data Control Mechanism Using Auto Rebalancing Strategy for Iot Applications. i-manager’s Journal on Computer Science, 6(1),9-17. https://doi.org/10.26634/jcom.6.1.14501

References

[1]. Abouee-Mehrizi, H., & Baron, O. (2016). State-dependent M/G/1 queueing systems. Queueing Systems, 82(1-2), 121-148.
[2]. Apache Active MQ. (n.d). ActiveMQ. Retrieved from http://activemq.apache.org
[3]. Apache Storm. (n.d). Storm Deployment. Retrieved from http://storm.apache.org/about/deployment.html
[4]. AWS. (n.d). AWS Price Calculator. Retrieved from https://calculator.s3.amazonaws.com/index.html
[5]. Bersani, M. M., Marconi, F., Tamburri, D. A., Jamshidi, P., & Nodari, A. (2016, April). Continuous architecting of stream-based systems. In Software Architecture (WICSA), 2016 13th Working IEEE/IFIP Conference on (pp. 146-151). IEEE.
[6]. Bhattacharya, D., & Mitra, M. (2013). Analytics on big fast data using real time stream data processing architecture (pp. 1-34). EMC Corporation.
[7]. Butzin, B., Golatowski, F., & Timmermann, D. (2016, September). Microservices approach for the internet of things. In Emerging Technologies and Factory Automation (ETFA), 2016 IEEE 21st International Conference on (pp. 1-6). IEEE.
[8]. Fowler, M. (2017). CQRS. Retrieved from https://martinfowler.com/bliki/CQRS.html
[9]. Fowler, (n.d). Microservices Resource Guide. Retrieved from http://martinfowler.com/microservices/
[10]. Gartner, (n.d). Gartner says 6.4 billion connected “things” will be in use in 2016 up 30 percent from 2015. Retrieved from http://www.gartner.com/newsroom/id/ 3165317
[11]. Kulkarni, S., Bhagat, N., Fu, M., Kedigehalli, V., Kellogg, C., Mittal, S., ... & Taneja, S. (2015, May). Twitter Heron: Stream processing at scale. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (pp. 239-250). ACM.
[12]. Lorido-Botran, T., Miguel-Alonso, J., & Lozano, J. A. (2014). A review of auto-scaling techniques for elastic applications in cloud environments. Journal of Grid Computing, 12(4), 559-592.
[13]. Naserpour, E. (n.d.). Service Load Balancing. In cloudpatterns.org Retrieved from http://cloud patterns.org/design_patterns/service_load_balancing
[14]. Newman, S. (2015). Building Microservices: Designing Fine-grained Systems. O'Reilly Media, Inc.
[15]. van der Veen, J. S., van der Waaij, B., Lazovik, E., Wijbrandi, W., & Meijer, R. J. (2015, March). Dynamically scaling apache storm for the analysis of streaming data. In Big Data Computing Service and Applications (BigDataSer vice), 2015 IEEE First International Conference on (pp. 154-161). IEEE.
[16]. Yang, W., Liu, X., Zhang, L., & Yang, L. T. (2013, July). Big data real-time processing based on storm. In Trust, Security and Privacy in Computing and Communications (TrustCom), 2013 12th IEEE International Conference on (pp. 1784-1787). IEEE.
If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Single Article

North Americas,UK,
Middle East,Europe
India Rest of world
USD EUR INR USD-ROW
Pdf 35 35 200 20
Online 35 35 200 15
Pdf & Online 35 35 400 25

Options for accessing this content:
  • If you would like institutional access to this content, please recommend the title to your librarian.
    Library Recommendation Form
  • If you already have i-manager's user account: Login above and proceed to purchase the article.
  • New Users: Please register, then proceed to purchase the article.