Deep Learning Techniques to Address Challenges in Big Data

Pabitha C *, B. Vanathi**
*-** Department of Computer Science and Engineering, SRM Valliammai Engineering College, Kancheepuram, Chennai, Tamil Nadu, India.
Periodicity:September - November'2021
DOI : https://doi.org/10.26634/jcom.9.3.16794

Abstract

Deep Learning and Big Data Analytics are the major focus in current rapidly growing technical environment. The use of large data has become crucial to different organizations as they are collecting huge amount of domain specific data, which contains critical information about cyber security, theft detection, national resources, business economics, marketing and medical information. The assessment of this huge amount of data needs advanced analytical techniques for surveying and predicting future course of action by creating advanced decision-making strategies. Deep learning algorithms utilize the collected training data, to create a representation model. This model uses the computer for predictions or decision making about new data without the need to train the machine explicitly to perform user task. These techniques and algorithms infer high-level complex abstractions as the data are represented through hierarchical process. A major benefit of deep learning is processing and learning from the huge amounts of unsupervised data, analyzing patterns from the data which can be used for big data analytics in which the raw data is largely unlabeled and not categorized. In this paper, deep learning techniques for addressing data of various types or formats is analyzed, enabling fast and full processing of data by integrating large amounts of different information i.e. data transformation is also addressed. It also deals with the quality of data as machine performance improves with the quality of data. Further exploration on the deep learning techniques to assist big data is done by focusing on two key topics on how Deep Learning assist some of the specific problems like Data Variety and Data Quality in Big Data Analytics, and how these techniques can aid in processing the Big Data.

Keywords

Deep Learning, Big Data Analytics, Data Transformation, Data Quality, Statistical Models.

How to Cite this Article?

Pabitha, C., and Vanathi, B. (2021). Deep Learning Techniques to Address Challenges in Big Data. i-manager's Journal on Computer Science, 9(3), 29-38. https://doi.org/10.26634/jcom.9.3.16794

References

[1]. Baştanlar, Y., & Özuysal, M. (2014). Introduction to machine learning. miRNomics: MicroRNA biology and computational analysis, 105-128. https://doi.org/10.1007/ 978-1-62703-748-8_7
[2]. Chandra, M. J. (2001). Statistical Quality Control. CRC Press, (pp. 1-296). https://doi.org/10.1201/9781420038675
[3]. Chen, M., Mao, S., & Liu, Y. (2014). Big data: A survey. Mobile Networks and Applications, 19(2), 171-209. https://doi.org/10.1007/s11036-013-0489-0
[4]. DeVor, R. E., Chang, T. H., & Sutherland, J. W. (2007). Statistical Quality Design and Control: Contemporary Concepts and Methods. Pearson College Division.
[5]. Emmanuel, I., & Stanier, C. (2016, November). Defining big data. In Proceedings of the International Conference on Big Data and Advanced Wireless Technologies (pp. 1-6). https://doi.org/10.1145/3010089.3010090
[6]. Fürber, C., & Hepp, M. (2011, March). Towards a vocabulary for data quality management in semantic web architectures. In Proceedings of the 1st International Workshop on Linked Web Data Management (pp. 1-8). https://doi.org/10.1145/1966901.1966903
[7]. Géczy, P. (2014). Big data characteristics. The Macrotheme Review, 3(6), 94-104.
[8]. Hawkins, S., He, H., Williams, G., & Baxter, R. (2002, September). Outlier detection using replicator neural networks. In International Conference on Data Warehousing and Knowledge Discovery (pp. 170-180). Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540- 46145-0_17
[9]. Hu, H., Wen, Y., Chua, T. S., & Li, X. (2014). Toward scalable systems for big data analytics: A technology tutorial. IEEE Access, 2, 652-687. https://doi.org/10.1109/ ACCESS.2014.2332453
[10]. Hu, H., Wen, Y., Chua, T. S., & Li, X. (2014). Toward scalable systems for big data analytics: A technology tutorial. IEEE Access, 2, 652-687. https://doi.org/10.1109/ ACCESS.2014.2332453
[11]. IBM. (n.d.). Security and Data Quality. Retrieved from https://www.research.ibm.com/haifa/projects/imt/outlier/in dex.shtml
[12]. Leavenworth, R. S., & Grant, E. L. (2000). Statistical Quality Control. Tata McGraw-Hill Education.
[13]. Maier, M., Serebrenik, A., & Vanderfeesten, I. T. P. (2013). Towards a big data reference architecture. University of Eindhoven, (pp.1-144).
[14]. Michalski, R. S., Carbonell, J. G., & Mitchell, T. M. (Eds.). (2013). Machine Learning: An Artificial Intelligence Approach. Springer Science & Business Media.
[15]. Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT press.
[16]. Natarajan, B. K. (2014). Machine Learning: A Theoretical Approach. Elsevier.
[17]. Strong, D. M., Lee, Y. W., & Wang, R. Y. (1997). Data quality in context. Communications of the ACM, 40(5), 103-110. https://doi.org/10.1145/253769.253804
[18]. Uddin, M. F., & Gupta, N. (2014, April). Seven V's of Big Data understanding Big Data to extract value. In Proceedings of the 2014 zone 1 Conference of the American Society for Engineering Education (pp. 1-5). IEEE. https://doi.org/10.1109/ASEEZone1.2014.6820689
[19]. Ward, J. S., & Barker, A. (2013). Undefined by data: A survey of big data definitions. Cornell University, (pp. 2). https://doi.org/10.48550/arXiv.1309.5821
[20]. Zikopoulos, P., & Eaton, C. (2011). Understanding big data: Analytics for enterprise class hadoop and streaming data. McGraw-Hill Osborne Media.
If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Single Article

North Americas,UK,
Middle East,Europe
India Rest of world
USD EUR INR USD-ROW
Online 15 15

Options for accessing this content:
  • If you would like institutional access to this content, please recommend the title to your librarian.
    Library Recommendation Form
  • If you already have i-manager's user account: Login above and proceed to purchase the article.
  • New Users: Please register, then proceed to purchase the article.