Mixed Reality Engineering Labs with AI Tutoring: XR Case Study

Engineering institutions often cannot give every student hands-on access to expensive or risky machinery. MythyaVerse built mixed reality engineering lab simulations that let students explore complex systems such as jet engines through Meta Quest passthrough experiences. The solution combines high-fidelity 3D models, component-level interaction, self-paced exploration, and AI tutoring powered by OpenAI and Gemini. Students can view systems in their real environment and ask for contextual explanations while they explore. The work turns inaccessible machinery into a safer, repeatable, and more interactive learning experience.

The MR labs give institutions a way to supplement physical training with repeatable, safer access to complex systems. Students can build practical understanding through exploration and on-demand explanation.

This project focused on enabling hands-on learning for complex mechanical systems, including jet engines, using mixed reality and AI assistance. Students can explore intricate engineering systems with LLM-powered contextual explanations.

Students need practical understanding of complex machinery, but physical systems can be expensive, dangerous, or difficult to access.

• Represent complex engineering systems accurately enough for education.
• Enable safe interaction through mixed reality rather than physical equipment handling.
• Provide explanations while students explore components and workflows.
• Support self-paced learning for students with different levels of prior knowledge.

• High-fidelity 3D models simulate machinery and component relationships.
• Meta Quest passthrough places the systems in the learner's real environment.
• Interaction design supports component exploration and workflow understanding.
• AI tutoring provides contextual explanations during exploration.

Engineering institutions face significant barriers in providing hands-on training with complex equipment. Systems like jet engines are expensive, dangerous, and difficult to access. The solution needed to:

• Simulate complex machinery like jet engines accurately
• Enable safe hands-on interaction through mixed reality
• Provide intelligent AI tutoring for contextual explanations
• Support intuitive exploration of components and workflows
• Make inaccessible systems visualizable and interactive

• Build interactive MR models of engineering systems
• Integrate LLM-powered tutor for contextual explanations
• Enable intuitive exploration of components and workflows
• Provide practical understanding of otherwise inaccessible systems
• Support self-paced learning with AI assistance

We created high-fidelity MR simulations using Meta Quest with passthrough capabilities, allowing students to interact with complex engineering systems in their real environment. An AI tutor powered by OpenAI and Gemini provides real-time contextual explanations as students explore components and workflows.

• Interactive MR Models -High-fidelity simulations of engineering systems including jet engines.
• LLM-Powered AI Tutor -Contextual explanations powered by OpenAI and Gemini for on-demand learning support.
• Passthrough Mixed Reality -Real-world environment integration using Meta Quest passthrough.
• Component Exploration -Intuitive interaction with individual parts and workflows.
• Self-Paced Learning -Students explore at their own pace with AI guidance.