Application of virtual reality in the mining industry- Where are we now?

From the initial planning phases to the closing of a mine, mining is an industry in which high attention, care and sensitivity is needed in every step of the mine life. Since mining projects have a long life span, managing a whole mine project is challenging, especially when generational differences are considered, such as tracking and logging data, or the change in engineering solutions over time. Present-time operations are very costly where each minute counts. Every decision and every minute during planning and operations has an impact on the efficiency, safety and economics of the project. Having extremely harsh conditions in both excavation and mineral processing operations not only slows down the processes but creates dangerous environments that need to be handled with precise safety measurements. To support the already established processes, new technologies could be implemented. Virtual reality (VR) is one such technology that could be implemented in the mining industry. VR is a technology that emerged in the 1960s immersing users in virtual environments and platforms. A user can experience these virtual environments from head-mounted headsets to rooms with 360 screens, all with different levels of immersion. Higher levels of immersion create a higher level of presence making the users more engaged in the activity they are trying to accomplish. These two factors of immersion and presence are what make VR unique and beneficial. VR has some applications and case studies in the mining industry along with some research being conducted in universities. Unfortunately, there has not been a comprehensive literature review or analysis regarding only VR and its implementations for the mining industry in the last five years. This article aims to describe what VR is and how it can be implemented to help manage projects in the mining industry. In the following sections, an explanation of VR is given followed by the methodology for conducting this literature review and concluding with results and discussion found from the review and analysis. This article aims to give information about VR regarding: What is VR and where it stands with other extended reality (XR) technologies?What is the current literature and applications on VR in the mining industry, how has VR been used in mining?How can VR be used in the mining industry? Virtual reality and key concepts The term virtual reality (VR) was coined by Jaron Lanier in 1987 and VR can be defined as computer-generated virtual environments, observed by the user through immersive hardware, where the user can feel present within that environment. VR is achieved through the stimulation of human senses in the physical world as accurately and closely as possible. VR is one of the branches of extended reality (XR) where other immersive 3D paradigms are included such as mixed reality (MR) and augmented reality (AR) (Rauschnabel et al., 2022). Where VR falls under the XR umbrella has been clearly defined with the virtual continuum proposed by Milgram and Kishino, 1994. The virtual continuum has been effectively illustrated by Stothard et al. as seen in Fig. 1. VR falls on the far-right side of the continuum where a user is fully immersed inside a virtual environment, blocking out the real world. The immersion is achieved through hardware technology, such as headsets and controllers among others. It determines how immersed a user is in a virtual environment. One of the most immersive VR displays is head-mounted displays (HMD). The hardware technology of HMD systems replicates a physical environment by (i) surrounding the user 360 degrees in the virtual environment, (ii) rendering the virtual environment according to the user’s movement in 6 degrees of freedom (6-DoF), (iii) conveying depth perception through stereopsis vision, that other displays such as a 2D screen, cannot achieve. The combination of these cues in a VR headset deceives the human brain to accept the virtual environment as reality. Immersion contributes to a user feeling present. Presence is more of a subjective aspect of VR, the degree of being “present in the virtual environment.” The level of presence users feel is crucial to keep them engaged in the VR application and to make them perceive the world as a “real” world. The degree of presence improves with well-designed virtual environments. Even though VR has been practiced for a while, it was not widely accessible because it mostly lacked commercial availability due to hardware restrictions resulting in lower performances on cheaper systems (Ellis, 1994). Given the developments in technology along with faster rates of computing speeds, the opportunity to create varying virtual simulations in varying scales is more abundant, allowing immersion and presence in VR applications more accomplishable. The main usage areas of VR include training, education, marketing and business, entertainment and gaming, health care, immersive analytics and digital twin displays. Because the need for VR is increasing, research in this area is essential to understand its benefits and future uses. Figure 1 Virtual continuum (modified after Milgram and Kishino (1994) and Stothard et al.). Methods and materials This literature review tries to offer results about the relationship between VR and its conducted research in the mining industry. The research questions for this literature review are: How many studies have been conducted throughout the last five years?What was the focus area of these studies, in terms of VR application areas? The VR research conducted in the mining domain was found and collected using three methods: Web-scraping digital libraries for records: Google Scholar, Science Direct, Springer, Taylor&Francis, IEEEXplore, OnePetro and ResearchGate. Exporting records through the Scopus database to visualize keywords in publications. Manual screening by authors between the years 2021–2022. Method (1) uses Google Scholar, which allows users to find publications across different platforms and digital libraries. To conduct a statistical analysis that finds the trend of VR in the mining industry over the years, we created a simple web-scraping program to automate finding records in Google Scholar. This method “scrapes” all the pages in a Google Scholar search in a short time. VR publications in mining found through this method were saved in an Excel file. Method (2) uses exported publications found in the Scopus database to implement into VOSviewer, a tool to analyze bibliometric data. To better understand the common application areas of VR publication in the mining industry, network analysis and visualization were done of the co-occurrence of keywords using VOSviewer. Conducting statistical and data analysis through methods (1) and (2) helps to find important trends, gaps and patterns. To ensure that the results from methods (1) and (2) are consistent and accurate finally method (3), a more traditional literature method was used. This method was conducted with the guidelines proposed by Kitchenham and Charters (2007). The following search strings were used in the web-scraping code, Scopus database, and through the overall identification of publications from digital libraries and databases: VR AND virtual reality AND mining engineeringVR AND virtual reality AND mining engineering AND minesVR AND virtual reality AND mining engineering OR minesVR AND virtual reality AND underground AND mining engineeringVR AND virtual reality AND surface AND mining engineeringVR AND virtual reality AND mining engineering AND coalVR AND virtual reality AND mining engineering AND metals. Results and discussion VR research through the years. VR in the mining domain has been studied throughout the years. To our knowledge, there has been no analysis done to see how much the literature focuses on VR over time. As seen in Fig. 2, there have been studies conducted since the late 1990s. Up until 2015, the number of publications did not exceed more than four publications per year, with some years having no publications at all. This steady trend drastically changes after the year 2015 and shows a sharp increase with most of the studies being conducted in the last year, 2022. Figure 2 The number of VR-focused publications in the mining domain 1998-2022. The upward trend observed can be attributed to the growing availability and accessibility of VR headsets. The introduction of new VR technologies has significantly expanded its applicability across various study areas. Notably, between 2014 and 2016, there were substantial technological advancements in VR hardware. A prominent example is the release of the Gear VR headset by Samsung in 2014. The Gear HMD converts a mobile phone display in 2D to be experienced as a 3D VR. Other significant examples are the Oculus Rift CV1, and HTC Vive which came out to the public in 2016. These headsets were one of the innovations that played a pivotal role in making VR more accessible and mainstream. Another breakthrough came with the Kickstarter launch of the Virtuix Omni by Virtuix Inc. in 2013 — a remarkable omnidirectional treadmill simulator designed for VR applications, enhancing the immersive experience in many VR applications. The evolution of hardware technology in the last decade appears closely linked to the upward trajectory of VR publications in the field of mining. With continued technological advancements on the horizon, it is reasonable to anticipate sustained growth in VR mining research. Recognizing this trend and maintaining a focus on VR research holds promise for benefiting the mining industry. Notably, Meta’s launch of the enhanced Quest 3 headset in 2023 and Apple’s upcoming Vision Pro release in 2024 mark significant strides by technology leaders, signifying the next wave of VR head-mounted displays. These developments bring improved hardware specifications, such as enhanced processing power and eye tracking. VR research focus areas. There are varying application areas of VR. For this review, the applications areas of VR in the mining domain were classified into three categories, training and education in VR, data analysis and visualization in VR (including immersive analytics (IA)), and digital twin technologies in VR (including monitoring technologies through digital twins, automation through digital twins and marketing through digital twins). With respect to the review conducted from these categories, it can be seen from Fig. 3, that the major focus of VR in the mining domain is training in safety throughout the years 2010 and 2020. The number of studies published in the other categories, compared to training, is very low. The reason this paper focuses on these application categories is given in further detail with some discussion points in the next segments. Training and education. Training for safety was one of the first fundamental application areas of VR other than entertainment. VR technologies have been shown to encourage and endorse users, making them more engaged (J. Collins, 2019). The increase in engagement can enable training to become more memorable in VR with the level of immersion and presence it provides (Murat Coban, 2022). Due to the positive impacts of learning with VR, it can be concluded from our results that the mining industry has been focusing on training through gamification in VR. VR simulations in mining safety training have been continuously developed in the last 10 years, and the number of these training simulations seems to be increasing. The development has been constant and abundant. This, combined with the ongoing advancements in the hardware, allows future research on mining training to be more feasible and accessible. In social environments, such as training underground mine workers, or a mining engineering classroom, VR has an advantage. The sense of presence and sense of space of a user is strengthened with the presence of other people in a simulation (Churchill, 1998). Traditional classrooms already create a social environment for students. VR laboratories in education can only aid the existing process. Immersive education allows students to connect more with the content of the course while their participation and interaction with the course material increases, resulting in an improved learning medium (Villena-Taranilla, 2022). Examinations should be done regarding these laboratory setups to increase inclusivity for varying groups of people (Ayse Collins, 2019). Mining is an industry with extreme conditions that cannot be accessed from a regular classroom. The equipment and machinery used in the mining industry also vary depending on the mine type and ore being excavated. These environments and tools don’t have high accessibility during a mining engineering student’s education life. Immersive education through VR can bring environments and tools that are physically inaccessible to the classroom (Janiszewski, M. 2020). Limitations in this area occur when the created 3D environments and other 3D content must be unique. Each mine site and classroom need individual 3D renderings that correspond to their goals, resulting in the work produced being costly and time consuming. Immersive mining training and education can only improve and increase over time. More dedicated applications at universities and more research conducted will help diminish the limitations of VR environment creation, allowing the mining industry to advance this focus area. Figure 3 The network of keyword co-occurrences of bibliometric data, colorized according to publication years. Data analysis and visualization. In recent years, the quantity of data obtained has grown exponentially. Analyzing data with larger volumes and higher dimensions can be a challenge, resulting in subfields of data analysis and visualization (Korkut Elif, 2022), visual analytics (VA), and immersive analytics (IA) (T. Chandler, et al., 2015). VA focuses on data analysis through visualizations and IA focuses on interactive and immersive data analysis and visualization, which aid observations and decision-making (Korkut, 2022). VR can aid our understanding of data because of its immersive and interactive experience with information. Data analysis methodologies employing visualizations consist of two primary components (Yi et al. in 2007): (1) representation and (2) interaction. Interactions with datasets hold the potential to transform or enhance a user’s cognitive load (Spence, 2007), thereby surmounting the limitations of representation and amplifying the user’s cognition (Yi et al., 2007; Norman, 1993). In their work, Wang et al. (2020) place a significant emphasis on the pivotal role of interactions within data visualizations to enrich the data analysis process. These interactions are achieved by crafting a virtual environment tailored for displaying visualizations through VR headsets. VR facilitates the presentation of data in a manner that transcends the screen-space constraints of conventional 2D displays, offering a flexible workflow. Turning our attention to geotechnical data analysis projects that incorporate interactions, Lato (2021) underscores the critical importance of gathering 3D geological data and subjecting it to four-dimensional (4D) analysis. His work recognizes that the examination of rock mass across both space and time constitutes an indispensable aspect of geotechnical practice. The deployment of extended reality (XR) technologies for visualization enhances communication clarity and increases decisionmaking confidence. The mining industry can use the benefits of IA. In the face of the deluge of data, the risk of encountering false positives and incomprehensible data representations is elevated, potentially burdening cognitive capacities (Zhou et al., 2018). Effective data and information display from monitoring systems become paramount in alleviating such cognitive burdens. IA stands as a valuable aid for engineers and workers, enabling them to extract insights from vast and intricate datasets, ultimately enhancing their overall situational awareness (Zhou et al., 2018). Consequently, this awareness not only promotes safety within the mining environment but also paves the way for more effective risk management. A mining project continues for long periods of time and has complex interoperable processes in multiple phases with large quantities of data. these circumstances, VR data visualization techniques could be a focus area of research in the mining domain. Innovative data analysis and visualization in VR are highly domaindependent, making the implementation of VR applications harder with no known framework (Korkut Elif, 2022). Another challenge faced can be in the transition of 2D data into 3D virtual environments. Research with a focus on possible limitations that may occur in data visualization in VR, specifically, in the mining domain could be a way to face these challenges. This could result in the mining industry having methods and frameworks for IA applications that provide unique solutions. As seen in Fig. 3, the focus on IA or data visualization through VR is not a highly focused area in mining research creating a gap and possibilities for future work. Digital twin technologies. Digital twins (DT) are replicas of either physical processes, working systems, or components of machinery and equipment, matching the process, system, or machinery in question in real time (Batty, M., 2018). They can be used for monitoring, data visualization, automation, marketing, education and training. They are considered to enable data to be transmitted from the real physical world to the virtual world (A. El Saddik, 2018). VR has an important part in the integration of the physical world and the virtual world, making the implementation of VR in digital twins promising (Fei Tao, 2019). Compared to visualization on a conventional computer monitor, where users typically view 3D data as a passive observer, the immersive experience of VR allows users to enter their data in a manner that feels spatially realistic (El Beheiry et al., 2019). This spatial depth perception allows a user to grasp “3D to 3D” visualizations, allowing it to be a great medium to use DT immersive context can change a person’s cognition from memory to spatial judgments (Laha and Bowman, 2014) to procedural training (Sowndararajan et al., 2010). Innovative data analysis and visualization in VR are highly domain-dependent, making the implementation of VR applications harder with no known framework (Korkut, 2022). This could result in the mining industry having methods and frameworks for IA applications that provide unique solutions. Mining projects continue for long periods of time and have complex interoperable processes in multiple phases with large quantities of data. Given these circumstances, VR data visualization techniques such as IA incorporated with DT can be a focus area of research in mining. Visualizing digital twins of large processes, such as mining, through the lens of VR, can aid in understanding them. VR can display the data flow throughout various processes that exist in a mining project in an interoperable way. On a smaller scale, monitoring equipment and automating them in an immersive and interactive medium can be made possible through VR. Displaying a mining system or process through digital twins in VR can help to extract more information about the current situation that the system is in. The mentioned usages of VR with the collaboration of digital twins provide similar benefits to the applications of VR in training and education, and VR in data analysis and visualization, as mentioned before. Making a mining project more visual and understandable, while engaging and interacting with VR, gives an opportunity to educate the public, and raise awareness of mining through different communities thus, providing benefits other than technical. There does not seem to be any systematic framework to develop and implement digital twins, with the collaboration of VR, in mining projects. Recently there is work done regarding creating a monitoring system using digital twins and displaying it with VR, specifically in the coal mining industry, which can be visualized in Fig. 3 (Wang, 2022). Even though some advancements and work are being conducted, creating a framework for better implementations of digital twins can give impactful solutions in the mining industry. To better achieve this advanced and further research can be focused on this application area of VR with digital twins. Conclusion VR is a technology that has been around and will continue to be around, with an increase in its usage across many domains. As technology is evolving, new application areas of VR will continue to emerge, while already existing areas become more abundant and advanced. This article tried to give an understanding of what this technology is, how much it has been the focus of the mining domain, and how it can be implemented in different application areas. A deeper look into the literature could be done by applying a meta-analysis to find more statistical results. Further work could be conducted on application areas outside of the literature, focusing on existing applications and solutions provided by companies. The mining domain seems to notice VR technology but may not be fully aware of its potential usage. Mining is a very challenging industry that requires unique solutions to complex engineering problems. Therefore the mining industry should be open to new ideas, solution methods, and technologies that are provided. VR can be one of the technologies that can highly benefit the industry. Even though VR may not diminish all limitations and challenges, it can aid the process of facing them. It is only possible to understand VR technology better and discover new unique solutions through research and implementations. References Marriott, K. et al. (2018). Immersive Analytics: Time to Reconsider the Value of 3D for Information Visualization. In: et al. Immersive Analytics. Lecture Notes in Computer Science, vol 11190. Springer, Cham. Bowman, Doug A., and Ryan P. McMahan. “Virtual Reality: How Much Immersion Is Enough?” Computer 40, no. 7 (2007): 36–43. Churchill, E.F., Snowdon, D. Collaborative virtual environments: An introductory review of issues and systems. Virtual Reality 3, 3–15 (1998). Otgonbayar, Angarag. 2021. “Adoption of Extended Reality (XR) for Enhancement of Human Capital in the Mining Workforce of the Future.” Electronic Theses and Dissertations (ETDs) 2008+. T, University of British Columbia. Berkman, M.I. (2018). History of Virtual Reality. In: Lee, N. (eds) Encyclopedia of Computer Graphics and Games. Springer, Cham. Huang, C. L., Luo, Y. F., Yang, S. C., Lu, C. M., & Chen, A.S. (2020). Influence of Students’ Learning Style, Sense of Presence, and Cognitive Load on Learning Outcomes in an Immersive Virtual Reality Learning Environment. Journal of Educational Computing Research, 58(3), 596–615. Murat Coban, Yusuf Islam Bolat, Idris Goksu,The potential of immersive virtual reality to enhance learning: A meta-analysis, Educational Research Review,Volume 36,2022,100452,ISSN 1747-938X. J. Collins, H. Regenbrecht, T. Langlotz, Y. Said Can, C. Ersoy and R. Butson, “Measuring Cognitive Load and Insight: A Methodology Exemplified in a Virtual Reality Learning Context,” 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2019, pp.351-362. Villena-Taranilla, Rafael & Tirado-Olivares, Sergio & Cózar, Ramón & González-Calero, José Antonio. (2022). Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educational Research Review. Ayse Collins, Fara Azmat & Ruth Rentschler (2019) ‘Bringing everyone on the same journey’: revisiting inclusion in higher education, Studies in Higher Education, 44:8, 1475- 1487. Jamil Salmi & Anna D’Addio (2021) Policies for achieving inclusion in higher education, Policy Reviews in Higher Education, 5:1, 47-72. T. Chandler, et al., “Immersive Analytics,” in 2015 Big Data Visual Analytics (BDVA), Hobart, TAS, 2015 pp. 1-8. A. El Saddik, “Digital Twins: The Convergence of Multimedia Technologies,” in IEEE MultiMedia, vol. 25, no. 2, pp. 87-92, Apr.-Jun. 2018. Batty, M. (2018). Digital twins. Environment and Planning B: Urban Analytics and City Science, 45(5), 817–820. Janiszewski, M., Uotinen, L., Merkel, J., Leveinen, J., and M. Rinne. “Virtual Reality Learning Environments for Rock Engineering, Geology and Mining Education.” Paper presented at the 54th U.S. Rock Mechanics/Geomechanics Symposium, physical event cancelled, June 2020. Skarbez Richard, Polys Nicholas F., Ogle J. Todd, North Chris, Bowman Doug A. Immersive Analytics: Theory and Research Agenda Frontiers in Robotics and AI VOLUME=6, 2019. Korkut, Elif & Surer, Elif. (2022). Visualization in virtual reality: a systematic review. Fei Tao, Meng Zhang, A.Y.C. Nee, Chapter 11-Digital Twin and Virtual Reality and Augmented Reality/Mixed Reality, Editor(s): Fei Tao, Meng Zhang. A.Y.C. Nee, Digital Twin Driven Smart Manufacturing, Academic Press, 2019, p. 219-241, ISBN 9780128176306. Kizil, M. S., Kerridge, A. P., and Hancock, M. G. (2004). Use of virtual reality in mining education and training. CRCMining Conference, Noosa Head, Queensland, Australia, 15-16 June 2004. Brisbane, Australia: Cooperative Research Centre-Mining (CRCMining). Stothard, Phillip & Kizil, M. & Squelch, Andrew & Schofield, Damian & Stone, Robert & van Wyk, Etienne & Fowle, Ken. (2015). Taxonomy of Interactive Computer-based Visualisation Systems and Content for the Mining Industry - Part 2. Mining Technology IMM Transactions section A. 124. 83-96. Hannah Snyder, Literature review as a research methodology: An overview and guidelines, Journal of Business Research,Volume 104,2019, Pages 333-339, ISSN 0148-2963. Naveen Donthu, Satish Kumar, Debmalya Mukherjee, Nitesh Pandey, Weng Marc Lim, How to conduct a bibliometric analysis: An overview and guidelines, Journal of Business Research,Volume 133, 2021, Pages 285-296. R. Spence, Information Visualization: Design for Interaction, 2nd ed:Prentice Hall, 2007. D. A. Norman, Things that make us smart. Reading, MA, USA:Addison-Wesley Pub. Co., 1993. Xianying Wang, Cong Guo, David A. Yuen, Gang Luo,GeoVReality: A computational interactive virtual reality visualization framework and workflow for geophysical research, Physics of the Earth and Planetary Interiors,Volume 298,2020,106312,ISSN 0031-9201. Matthew Lato. 2021. Canadian Geotechnical Colloquium: three-dimensional remote sensing, fourdimensional analysis and visualization in geotechnical engineering — state of the art and outlook. Canadian Geotechnical Journal. 58(8): 1065-1076. Zhou, Lin, X., Luo, X., Zhao, Y., Chen, Y., Chen, N., & Gui, W. (2018). Visually enhanced situation awareness for complex manufacturing facility monitoring in smart factories. Journal of Visual Languages and Computing, 44, 58–69. Mohamed El Beheiry, Sébastien Doutreligne, Clément Caporal, Cécilia Ostertag, Maxime Dahan, Jean-Baptiste Masson,Virtual Reality: Beyond visualization, Journal of Molecular Biology, Volume 431, Issue 7, 2019, Pages 1315- 1321. B. Laha, D. A. Bowman, and J. J. Socha. Effects of VR system fidelity on analyzing isosurface visualization of volume datasets. IEEE transactions on Visualization & Computer Graphics, 20(4):513–522, Apr. 2014. Ragan, Eric & Sowndararajan, Ajith & Kopper, Regis & Bowman, Doug. (2010). The Effects of Higher Levels of Immersion on Procedure Memorization Performance and Implications for Educational Virtual Environments. Presence. 19. 527-543. Stephen R. Ellis. 1994. What are virtual environments? IEEE Comput. Graph. Appl. 14, 1 (January 1994), 17–22. Philipp A. Rauschnabel, Reto Felix, Chris Hinsch, Hamza Shahab, Florian Alt, What is XR? Towards a Framework for Augmented and Virtual Reality, Computers in Human Behavior, Volume 133, 2022, 107289, ISSN 0747-5632. J. S. Yi, Y. a. Kang, J. Stasko and J. A. Jacko, “Toward a Deeper Understanding of the Role of Interaction in Information Visualization,” in IEEE Transactions on Visualization and Computer Graphics, vol. 13, no. 6, pp. 1224- 1231, Nov.-Dec. 2007.

Application of virtual reality in the mining industry- Where are we now?

Authored by:
Nilufer Akbulut, Angelina Anani

To view the past issues of the UCA magazine you must be a member of SME/UCA.

Mining Engineering’s Weekly Rundown: Episode 73

Get your SME Foundation VIP Access Conference Pass to MINEXCHANGE 2027

“Check the box” and help support the SME Foundation

Mining Engineering’s Weekly Rundown: Episode 73

Get your SME Foundation VIP Access Conference Pass to MINEXCHANGE 2027

“Check the box” and help support the SME Foundation

To view the past issues of the UCA magazine you must be a member of SME/UCA.

Sibanye Stillwater announces job cuts at Montana operations

Automation continues to move the mining industry forward

Related Articles

Get your SME Foundation VIP Access Conference Pass to MINEXCHANGE 2027

“Check the box” and help support the SME Foundation

The Art and Science of the Flowsheet

UCA Division officer nominations

Mining Engineering’s Weekly Rundown: Episode 73

Get your SME Foundation VIP Access Conference Pass to MINEXCHANGE 2027

“Check the box” and help support the SME Foundation

Mining Engineering’s Weekly Rundown: Episode 73

Get your SME Foundation VIP Access Conference Pass to MINEXCHANGE 2027

“Check the box” and help support the SME Foundation

Use Single Sign On