Map Reduce Architecture for Grid

Neeraj Kumar Rathore*
Assistant Professor, Department of Computer Science and Engineering, Jaypee University of Engineering and Technology, Guna, M.P., India.
Periodicity:July - September'2015
DOI : https://doi.org/10.26634/jse.10.1.3629

Abstract

Recently many large scale computer systems are built in order to meet the high storage and processing demands of compute and data-intensive applications. MapReduce is one of the most popular programming models designed to support the development of such applications. MapReduce is a software framework for easily writing applications which process vast amount of data in-parallel, by using multiple CPUs on various machines, in a reliable, and fault tolerant manner. The various input and output parameters, that are part of this model have been identified. The proposed architecture is implemented in open source Java. The Map Reduce programming model is easy to use, even for programmers without experience with parallel and distributed systems, since it hides the details of parallelization, faulttolerance, locality optimization, and load balancing. It has a large variety of problems which are easily expressible as MapReduce computations. Finally, an implementation of MapReduce that scales to large clusters of machines comprising thousands of machines has been developed. The implementation makes efficient use of these machine resources and therefore is suitable for use on many of the large computational problems encountered.

Keywords

MapReduce, Fault-tolerant, Architecture, Framework, Data-intensive.

How to Cite this Article?

Rathore, N. (2015). Map Reduce Architecture for Grid. i-manager’s Journal on Software Engineering, 10(1), 21-30. https://doi.org/10.26634/jse.10.1.3629

References

[1]. J. Dean, and S. Ghemawat. (2004). “MapReduce: Simplified Data Processing on Large Clusters”. In Proc. of the 6th Symposium on Operating System Design and Implementation.
[2]. H. C. Yang, A. Dasdan, R. L. Hsiao, and D. S. P. Jr. (2007). Map-Reduce-Merge: simplified relational data processing on large clusters. In Proc. of SIGMOD.
[3]. Apache. Hadoop. In http://lucene.apache.org/ hadoop/.
[4]. http://en.wikipedia.org/wiki/Directed_acyclic_graph
[5]. Jim Gray. Sort benchmark home page. Retrieved from http:// research. microsoft.com / barc / Sort Benchmark/.
[6]. Jim Wyllie. Spsort: How to sort a terabyte quickly. Retrieved from http://alme1.almaden.ibm.com/cs/ spsort.pdf.
[7]. Parallel Programming in the Age of Big Data Retrieved from http://gigaom.com/2008/11/09/mapreduce-leadsthe- way-for-parallel-programming/
[8]. http://en.wikipedia.org/wiki/Functional_programming.
[9]. A.W. McNabb, C. K. Monson, and K. D. Seppi. (2007). “Parallel PSO Using MapReduce”. In Proc. of the Congress on Evolutionary Computation.
[10]. S. Chen, S. W. Schlosser. (2008). “Map-Reduce Meets Wider Varieties of Applications”. In IRP-TR-08-05, Technical Report, Intel Research Pittsburgh.
[11]. C. Jin, C. Vecchiola and R. Buyya. (2008). “MRPGA: An Extension of MapReduce for Parallelizing Genetic Algorithms”. In Proc. of 4th International Conference on e-Science.
[12]. R. E. Bryant. (2007). “Data-Intensive Supercomputing: The Case for DISC”. In CMU-CS-07-128, Technical Report, Carnegie Mellon University.
[13]. Report on “The Global Competitiveness Report”, 2012–2013.
[14]. “MapReduce: Simplified Data Processing on Large Clusters”, Symposium on Operating Systems Design and Implementation USENIX Association 2012.
[15]. Vishal Sharma, Rajesh Kumar, and Neeraj Kumar Rathore, (2015). “Topological Broadcasting Using Parameter Sensitivity Based Logical Proximity Graphs in Coordinated Ground-Flying Ad Hoc Networks”, Journal of Wireless Mobile Networks Ubiquitous Computing, and Dependable Applications (JoWUA), SCOPUS indexed, Vol.6, No.3, pp.54-72.
[16]. N Rathore and I Chana, (2015). Variable Threshold Based Hierarchical Load Balancing Technique in Grid, Engineering with computers, Springer publication-London (England (UK), Page 597-615, ISSN: 0177-0667 (print version) ISSN: 1435-5663 (electronic version), Vol.31, Issue 3, IF- 1.451.
[17]. Neeraj Kumar Rathore, (2015). “Load Balancing Algorithm for Grid”, in 30th M.P. Young Scientist congress, Bhopal, M.P., pp-56.
[18]. N Rathore and I Chana, (2014). “Load Balancing and Job Migration Techniques in Grid : A Survey of Recent Trends”, Wireless Personal Communication, Springer Publication-New-York (USA), ISSN print 0929-6212, ISSN online 1572-834X, Vol.79(3), pp.2089-2125, IF - 0.979.
[19]. N Rathore and I Chana, (2014). “Job migration with fault tolerance based QoS scheduling using hash table functionality in Social Grid Computing", Journal of Intelligent & Fuzzy Systems, IOS Press publication- Netherland, Vol.27(6), pp.2821–2833, ISSN print 1064- 1246, IF- 0.936.
[20]. Neeraj Kumar Rathore, (2014). “An Efficient Hierarchical Load Balancing Technique for Grid”, in 29th M.P. Young Scientist congress, Bhopal, M.P., pp.55.
[21]. Neeraj Kumar Rathore, and Inderveer Chana, (2013). “Report on Hierarchal Load Balancing Technique in Grid Environment”, International journal of scientific and innovative technology, i-manager's Journal on Information Technology, Vol.2, No.4, ISSN Print: 2277- 5110, ISSN Online: 2277-5250, pp-21-35.
[22]. Neeraj Kumar Rathore, and Inderveer Chana, (2013). “A Sender Initiate Based Hierarchical Load Balancing Technique for Grid Using Variable Threshold Value”, International conference IEEE-ISPC, ISBN- 978-1- 4673-6188-0, pp.1-6, 26-28.
[23]. Rohini Chouhan and Neeraj Kumar Rathore, (2012). “Comparision of Load Balancing Technique in Grid”, 17th Annual conference of Gwalior Academy of mathematical science and Natonal symposium on computational mathamatics & Information Technology, pg no-87, JUET, Guna, M.P., 7-9.
[24]. Neeraj Kumar Rathore and Inderveer Chana, (2011). “A Cogitative Analysis of Load Balancing Technique with job migration in Grid Environment”, World Congress on Information and Communication Technology (WICT), Mumbai, IEEE proceedings paper, ISBN -978-1-4673-0127-5 pp.77-82.
[25]. Neeraj Kumar Rathore and Inderveer Chana, (2010). Checkpointing Algorithm in Alchemi.NET, in Lambert Academic Publication House, Germany ISBN-10: 3843361371, ISBN-13: 978-3843361378.
[26]. Neeraj Kumar Rathore and Inderveer Chana, (2010). “Checkpointing Algorithm in Alchemi.NET”, Pragyaan: Journal of Information Technology, IMS Dehradun. ISSN No.: 0974-5513, IEEE, CSI and MPCET, Vol.8, Issue 1, pp.32-38, Dehradun.
[27]. Neeraj Kumar Rathore and Inderveer Chana, (2010). “Fault Tolerance Algorithm in Alchemi.NET Middleware”, National Conference on Education & Research (ConFR10), Third CSI National Conference of CSI Division V, Bhopal Chapter, IEEE Bombay, and MPCST Bhopal, organized by JUIT, India.
[28]. Neeraj Kumar Rathore and Inderveer Chana, (2009). “Checkpointing Algorithm in Alchemi.NET”, Annual conference of Vijnana Parishad of Iindia and National Symposium Recent Development in Applied Mathematics & Information Technology, JUET, Guna, M.P.
[29]. Neeraj Kumar Rathore and Inderveer Chana, (2008). “Comparative Analysis of Checkpointing”, PIMR Third National IT conference, IT Enabled Practices and Emerging Management Paradigm book and category is Communication Technologies and Security Issues, pp.32- 35, Topic No/Name-46, Prestige Management and Research, Indore, (MP), India.
If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Single Article

North Americas,UK,
Middle East,Europe
India Rest of world
USD EUR INR USD-ROW
Pdf 35 35 200 20
Online 35 35 200 15
Pdf & Online 35 35 400 25

Options for accessing this content:
  • If you would like institutional access to this content, please recommend the title to your librarian.
    Library Recommendation Form
  • If you already have i-manager's user account: Login above and proceed to purchase the article.
  • New Users: Please register, then proceed to purchase the article.