Figure 1. Block Diagram of Educational Augmented Reality System
Based on the specific information received and skills, people can be educated in various ways. Traditionally, classroom lectures using textbooks was the way to teach. At present, computers, handheld devices and other electronic appliances are also used as learning tools. The choice of innovative learning depends on the access to varied technologies and therefore the infrastructure surrounding the learner influence the learning experience. In rapidly changing society, where there's an excellent deal of obtainable information and knowledge, adopting and applying information at the proper time and right place is required to gain efficiency in both school and colleges. Augmented Reality (AR) is one technology that dramatically shifts the situation and timing of education and training. The basic idea of AR is overlaying virtual objects on a true world image thus "augmenting" the important world image which we currently see. This technology might be very useful in various domains like medicine, automobile, aerospace industry and space research.
According to a source, 71% of individuals from the USA between 16 to 24 years have smartphones (VRducation, 2018). After the Covid-19 pandemic, India too follows the same trend as the schools shifted to online education. Most of the youth are active smartphone users, using these gadgets to enter social platforms, play games and to be in social contact with friends and relatives.
Two- dimensional media in education is extremely convenient, familiar, flexible, portable and cheap. But it is static and does not offer dynamic content. Without proper teaching technique, concepts that are taught will be difficult to understand. So, the aim is to develop an application to learning the concepts easily. The main problem with the development of educational tools is associated with the lack of funds as most of the educational institutions spend it on the development of infrastructure, research laboratories, libraries, etc.
Augmented Reality (AR) is a relatively young technology and is the way of future technologies and innovations. It has large number of applications in the fields of medical, marketing, advertisement, entertainment, education and training, industrial design, manufacturing and maintenance, architecture, navigation, emergency, search and rescue systems, interactive gaming, and tourism, etc. Augmented reality is a new technology, which can enhance the theoretical view of the real world. Moreover, augmented reality combines virtual world, which are objects generated by computer and also represent them within the world simultaneously. As an example, holding the phone camera over the pictures, posters, or street can ascertain extra information associated with that object (Qiao et al., 2019). Augmented reality can be used not only in advertising but also in education. As part of the education, students will be encouraged to use other channels to make it easier, faster and more convenient to understand the lesson by moving the camera on the smartphone. The benefits of AR include the integration of reality with virtual, interactive, real-time, and accurately visualizing 3D objects. AR supported system allows students to interact with context rather than paper on the normal course, even more, immerse students during a knowledge from learning activity and enhance the communication between group members.
There are various researches published on the Augmented Reality applications within the field of education. Therefore, before describing our contribution and the related methodology, we offer a quick overview of the related projects that utilize AR technology for developing educational applications.
Kraut and Jeknić (2015) mentioned that AR has the potential to improve the educational experience. AR tools can help learners in their learning experience in a better way, as AR can update traditional books with a digital layer. They predicted that the new layer would encompass multiple senses, which could speed up the memorization process and improve the common understanding present in the textbooks. Furthermore, the learning activity through AR tool can be supervised by a mentor or through an automated process.
Bacca et al. (2014), presented a systematic review of 32 studies between 2003 and 2013 on AR research in education. This paper reported the various factors suitable for AR setting in education. They also envisioned the future prospects in the field of augmented reality in education.
Dalim et al., (2016) presented a paper on the development of an interactive augmented reality tool for teaching basic English to children who did not have English as their first language. This tool used audio as well as provided visual representation using augmented reality.
Shoikova et al. (2018) emphasized on smart learning with AR and enhanced IoT data. In their perception, the environment is not yet set for learning through AR as there are still limitations in presentation of the learning content in AR platform.
Alhumaidan et al. (2015) and Lakshmiprabha et al. (2014) pointed out that AR book will provide young students thrill while learning. AR book is on the brink of mixed reality that provides them with the visualization and actual sound of 2D models of things. With AR and Virtual Reality (VR) this platform will twiddle all 2D models, move the learners altogether into a different paradigm and train autistic children in learning effectively.
Al Qassem et al. (2016) and Lv et al. (2015) described on implementation of VR in personal computers connected with headsets, AR uses smart glasses and AR headsets, portable devices and MR (mixed reality) in holographic devices and immersive devices which will all improve the learning experience to a different level for future students.
Lee (2012) discussed the status of AR in education and training. This paper reviewed many applications that used smart glasses and applications that implemented AR and VR for effective teaching and training. Interactive roleplay and AR games will be helping the students in understanding the concepts. With AR facilities in the museums, the learner can have the experience of going back into the past. In short, AR will improve efficiency and effectiveness in learning.
Wang et al. (2018) presented case studies illustrating AR implementation that utilized different pedagogical approaches, that are supposed to impact the future education at a very large scale.
Based on the review, we propose the concept in creating 3D augmented reality for educational purpose and the challenges in developing such a system.
The whole system of this application is represented by the flowchart in Figure 1.
Figure 1. Block Diagram of Educational Augmented Reality System
This is an approach to help students to learn more effectively with real time animations. 3D models are created in Blender and will be exported in .obj format, so that it can be imported into the Android's root directory. Using the Firebase Cloud Storage Database, all the .obj files will be stored in the cloud, as it provides free access approximately upto 10GB od data. Using the Sceneform SDK, floors and walls will be defined. Once identified, they can use touch input to place 3D models on the floor for real-time interaction.
ArFragment are often added to an Android layout file, like all Android Fragment. It checks whether a compatible version of ARCore is installed, and prompts the user to put in or update ARCore as necessary. Additionally, the fragment determines whether the app has access to the camera, and asks the user for permission, if the permission has not been granted yet (Thornsby, 2018).
The ArSceneView renders the camera images from the session onto its surface. It then highlights planes once they are detected by ARCore, which are ahead of the camera. ARSceneView, as its name suggests, also features a scene attached thereto. The scene may be a tree-like arrangement that holds nodes, which are the virtual objects which will be rendered (Thornsby, 2018).
All AR processes, such as motion tracking, environmental understanding, and lighting estimation, happen inside an ARCore session. Session is the main entry point to the ARCore API. It manages the AR system state and handles the session lifecycle and allows the app to create, configure, start, or stop a session (Thornsby, 2018).
These are 3D models consisting of meshes, materials and textures, that can be rendered on the screen by Sceneform. Sceneform provides three ways to create renderables: from standard Android widgets, from basic shapes/materials and from 3D asset files (OBJ, FBX, glTF) and any other relevant characteristics and also the content of mobile AR application (Ruban, 2019). In design phase, conceptual model, flowchart and 3D Models were developed. The design phase involves the development of the conceptual model for the mobile AR application which involves the learning theory, the medium and AR technology to be used in the application; the teaching and learning approach to be implemented and the components involve in this mobile AR application. Using Scene form SDK, the floors and walls will be identified, after the identification they can use touch input for placing the 3D models onto the floor for real-time interactions.
Markerless AR will be used as they allow them to try different combinations of objects, styles, and locations and to decide where exactly to place the virtual object. This technology is based on scanning the environment and creating maps used for placing digital objects into the physical world with the help of smart phones. The user need not have to worry about the location of the virtual objects, and this means that the virtual objects seem to “hover” in the air (Aryal, 2020). This requires minimal computing power to reduce the memory utilization by augmentation program used inside the smartphone chipsets.
All our 3D figures are formatted as “blend”, and rendered using the Blender application.
As part of the paper, we used the globe using easy texture mapping for spheres (Figure 2). It gives maps to make more convincing materials, adding glow, color grading and a warp effect in the compositor (Price, 2016).
Figure 2. Rendering 3D View Using Blender
The system proposed is to use AR for effective learning method and hence the system is supposed to use medium to large images, all stored in the cloud. Data connectivity, retrieval of images from cloud, rendering AR using third-party tools and usage of appropriate hardware resources were all challenges when developing this application. Students using this tool for learning will also face such challenges.
Part of the challenge in many training programs is related to health and safety of the individuals while providing hands-on experience. Certain learning environments put the learners at risk. Augmented Reality can help with this. It could, in theory, help to bridge the gap between the classroom and physical experience (Allsopp et al., 2006; Antonioli et al., 2014). However, bringing AR at practice has a lot of practical constraints, both at the development phase as well as in the implementation phase.
People unconsciously perceive subtle clues about how objects or living things are illuminated in their environment. When a virtual object has no shadow or shiny material that does not reflect its surroundings, users may feel that the object does not quite fit into a particular scene, although they cannot help but explain why. This is often why rendering AR objects according to the lighting during the scene is critical to an immersive and more realistic experience (Google Developers, n.d.). Cognitive differences and inexperience in using the available tool was one of the challenges in implementing lighting effect.
Augmented reality is widely utilized in education in several prospects. AR applications increase the learning and understanding ability of the students and also increase creativity. A student becomes more interactive to review the progress of their learning progress. From teachers point of view, it becomes easy to form students understand a tough topic in easy ways. The proposed work is a continuation of the past researches and application of the evolved technology over a period of time. This paper is a learning phase for finding the right tools to implement AR in learning. We have planned to cover more topics into augmented reality, starting with simpler concepts and moving over to complex topics. The vision is to make learning easier and fun for students.