Short link: aka.ms/UnityIntroToMixedReality

Overview

In this lesson, you will learn about the basic concepts of Mixed Reality and explore the applications of Mixed Reality in different industries.

In the Project section, you will set up your first Mixed Reality project using Unity and Mixed Reality Tool Kit.

You can jump directly into setting up your first project on the .

Mixed reality is the merging of real and virtual worlds to produce new environments and visualizations, where physical and digital objects co-exist and interact in real time.

We think of Mixed reality as a spectrum from the physical world to an augmented world to fully immersive virtual world and all the possibilities in between.

In this lesson, you will learn about the basic concepts of Mixed Reality and explore the applications of Mixed Reality in different industries.

Read through the questions below. If you feel comfortable with the answer, feel free to skip to project section or next chapters.

• What is Mixed Reality?

The first revolution in computing happened with the creation of mainframe computers: computers that, at times, occupied a whole room. Mainframes were used by large organizations such as NASA for critical applications that process data.

The second wave of computing is defined by the Personal Computers(PC) becoming widely available.

We believe third wave of computing is going to include many devices to manage data and include IoT sensors and Mixed Reality devices.

We have more data than ever before. To be able to process the data and make informed decisions, we need to have access to the data in the right time and right place. Mixed Reality is able to bring that data into our context, real world.

Design & Prototyping: Enables real-time collaborative iteration of 3D physical and virtual models across cross-functional teams and stakeholders.

Training & Development: Provides instructors with better tools to facilitate teaching/coaching sessions. It offers trainees an enhanced and engaging learning experiences through 3D visualizations and interactivity.

Geospatial Planning: Enables the assessment and planning of indoor and outdoor environments (i.e. future construction sites, new store locations, interior designs) and removing the need for manual execution.

Sales Assistance: Improves the effectiveness of individuals in sales-oriented roles by providing tools such as 3D catalogs and virtual product experiences that increase customer engagement and strengthen buyer confidence.

Field Service: Improves the first-visit resolution and customer satisfaction of customer support issues. It is typically used for complex products that would otherwise require a field visit. It can serve as a platform for targeted up-sell opportunities, as well.

Productivity & Collaboration: Transform the space around you into a shared augmented workplace. Remote users can collaborate, search, brainstorm and share content as if they were in the same room

• Go to Edit > Preferences.

• Change the color scheme under General, if it is available.

• You can change the default editor by selecting

How to create a new scene?

• On the Project panel, right click and select Create > Scene.

• 6 Use Cases for Enterprise Mixed Reality

• 5 Use Cases Of Augmented Reality That Boosted Businesses’ Sales

• In the Unity menu, select File > Build Settings... to open the Build Settings window.

• In the Build Settings window, select Universal Windows Platform and click the Switch Platform button.

• Click on Project Settings in the Build Settings window or the Unity menu, select Edit > Project Settings... to open the Project Settings window.

• In the Project Settings window, select Player > XR Settings to expand the XR Settings.

• In the XR Settings, check the Virtual Reality Supported checkbox to enable virtual reality, then click the + icon and select Windows Mixed Reality to add the Windows Mixed Reality SDK.

• Optimize the XR Settings as follows:

  • Set Windows Mixed Reality Depth Format to 16-bit depth.

  • Check the Windows Mixed Reality Enable Depth Sharing checkbox.

• In the Project Settings window, select Player > Publishing Settings to expand the Publishing Settings. Scroll down to the Capabilities section and check the SpatialPerception checkbox.

Save your project and open up the Build Settings. Click on Build button, not Build and Run. When prompted, create a new folder(ex:HoloLensBuild) and select your new folder to build your files into.

When your build is done, your file explorer will automatically open to the build folder you just created.

• Make sure you have added the UnityAR Camera Settings Provider to your scene.

• Switch platform to either Android or iOS in the Unity Build Settings

  When you switch the platform you should see the MRTK Project Configurator Window with settings for your chosen platform. Click Apply to enable platform specific settings.

  iOS Project Configurator Settings.

• If the platform is iOS, Edit > Project Settings > Player > Other Settings, under the Optimization header, uncheck Strip Engine Code

• Build and Run your scene to your device.

• Turn on your HoloLens device.

• Tab your wrist(HoloLens 2) or make a bloom gesture(HoloLens 1) to initiate Windows menu.

• Open the Settings > Update & Security.

• Select For Developers tab on the right hand panel.

Getting Super Powers

Becoming a super hero is a fairly straight forward process:

$ give me super-powers

Once you're strong enough, save the world:

• Windows Mixed Reality Docs:

• Mixed Reality Curriculum Youtube Playlist:

is a github repository that is configured for Windows Mixed Reality development. The repo includes Mixed Reality Toolkit and .gitignore files.

You can create a new project from the seed instead of downloading the different assets and setting up your git project. To be able to use the seed project, you can and setup your development environment or directly download the repository content.

The Settings app on Android includes a screen called Developer options that lets you configure system behaviors that help you profile and debug your app performance. For example, you can enable debugging over USB, capture a bug report, enable visual feedback for taps, flash window surfaces when they update, use the GPU for 2D graphics rendering, and more.

Enable developer options and USB debugging

On Android 4.1 and lower, the Developer options screen is available by default. On Android 4.2 and higher, you must enable this screen. To enable developer options, tap the Build Number option 7 times. You can find this option in one of the following locations, depending on your Android version:

Augmented Reality(AR) is defined as a technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view. Augmented Reality experiences are not limited to visual addition to our world. You can create augmented experiences that are only audio addition to your physical world or both audio and visual.

Augmented Reality experiences are also not limited to headsets like HoloLens. Today, millions of mobile devices have depth-sensing capabilities to augment your real world with digital information.

If you are using HoloLens Seed project, you do not need to follow this step. Seed project already comes with MRTK. Still, it's good to know how to import the MRTK assets for your future projects.

First, you need to download MRTK by going to their github page: and navigating to releases tab. Scroll down to Assets section and download the tools:

• Examples

Before you get started with developing for Mixed Reality for Unity, make sure to check everything in the below list and follow the instructions for each download.

• Latest Windows 10 Education or Pro

• Unity Hub, latest Unity Editor and Windows modules. Optionally, iOS or Android modules if you plan to develop for mobile devices. You can always add a module later.

• Visual Studio

• Windows 10 SDK

• Enable Hyper-V

• Optional: HoloLens Emulator

Windows 10

Install the most recent version of or so your PC's operating system matches the platform for which you are building mixed reality applications.

You can check your Windows version by typing "about" in the Windows search bar and selecting About your PC as shown in the below image.

You can learn more about upgrading your Windows 10 Home to Pro at .

Unity

Go to: page and download the Unity Hub instead of Unity Editor.

Unity Hub allows you to download multiple Unity Editors and organize your projects in one place. Since Unity upgrades are not backward compatible, you have to open the projects with the same Unity version that it was created with. You can update the projects to the latest Unity version but that requires a lot of debugging usually. Easiest way to get going with a project is to keep the same version. I will show you how to debug to update your projects later in this chapter.

You will need to download Windows development related modules along with your Unity Editor. Make sure Universal Windows Platform Build Support and Windows Build Support is checked while downloading Unity Editor through Unity Hub or add it after by modifying the install.

You can add modules or check if you have them in your editor by clicking on the hamburger button for the Unity Editor version and checking the above module check-boxes.

If you would like to build for an Android or iOS mobile device, make sure the related modules are checked as well.

Visual Studio

You can download Visual Studio by adding Microsoft Visual Studio 2019 module to your Unity Editor as shown in previous step or download it at .

You can always add the necessary workflows to Visual Studio after download:

• Search for the Visual Studio Installer in Windows search bar to open the installer.

• Select modify to see the list of modules and select click modify.

• You can also select the Windows SDK versions from individual components tab in the VS installer.

1 ) Make sure you have imported Microsoft.MixedReality.Toolkit.Unity.Foundation as a custom asset or through NuGet.

2 ) In the Unity Package Manager (UPM), install the following packages:

3 ) Enabling the Unity AR camera settings provider.

The following steps presume use of the MixedRealityToolkit object. Steps required for other service registrars may be different.

1. Select the MixedRealityToolkit object in the scene hierarchy.

2. Select Copy and Customize to Clone the MRTK Profile to enable custom configuration.

3. Select Clone next to the Camera Profile.

4. Navigate the Inspector panel to the camera system section and expand the Camera Settings Providers section.

5. Click Add Camera Settings Provider and expand the newly added New camera settings entry.

6. Select the Unity AR Camera Settings provider from the Type drop down.

Mixed Reality Toolkit (MRTK) provides a set of components and features to accelerate cross-platform Mixed Reality application development in Unity. MRTK includes:

• UI and interaction building blocks.

• Tools.

• Example Scenes.

You can learn more about the components at: .

Medical

What are some of the security issues with Mixed Reality Applications?

Since a Mixed Reality application might have access to the user video stream, developers might be able to save or share private information about the user. Be careful to not to save any sensitive data or image anywhere other than users device. Never send sensitive information to any backend.

Why is eye scan data sensitive information?

Iris scan is a more accurate identification method than fingerprint. Since iris scan data can be used to identify and sing in a user, it should never leave the users device. HoloLens 2 does not send the iris scan to the cloud and does not give access to the data.

Why is eye tracking important for users privacy?

Eye tracking, while a very useful tool to make your application more accessible, it can also be used to collect data about the user's attention and might be used to manipulate the user's attention.

Can I open my unity project in the current version, if it is originally saved in an older version?

Unity versions are not backward compatible. If you decide to open a project on a newer version, Unity will try to update your project automatically but it is not guaranteed that the newer version will work with your imported assets. There might be some incompatibilities with your asset or your code and the new version.

What does the Unity Versioning mean and when is it safe to update the Unity version?

Let's take the latest version in the image below, 2019.3.5f:

• 2019: is the year the Unity version was developed. Changes are issued once a year. If there are major changes, that will break your application. Stick to the same year version unless you are creating a new application from scratch for now. We will talk about how to update your project to the latest version in the following lessons.

• 3: implies the 3rd iteration in 2019. When a version updates from 2 to 3, there are minor breaking code. Make sure to read changelog before updating your project from 2 to 3.

• .5f

How to update my Unity version to a newer one?

In your Unity Hub, under the project tab, you can select the Unity version drop down for your application and select a newer version of Unity. Unity will confirm your choice before updating your project. It is a good idea to save a version of your project as a new branch in Github, in case you need to revert back.

What could go wrong with Unity NuGet packages?

Here is a detailed article about the subject:

Scenes

In Unity, areas of the game that a player can interact with are generally made up of one or more Scenes. Small games may only use one Scene; large ones could have hundreds.

Every Unity project you create comes with a SampleScene that has a light and a camera.

You can create a new scene by right clicking under the assets tab and selecting Create > Scene. Organizing scenes under a Scenes folder is only for the organization purposes.

You can use scenes to organize navigation inside your application or adding different levels to a game.

GameObjects and components

Every object in the game world exists as a GameObject in Unity. GameObjects are given specific features by giving them appropriate components which provide a wide range of different functionality.

When you create a new GameObject, it comes with a Transform component already attached. This component controls the GameObject’s positional properties in the 3D (or 2D) gamespace. You need to add all other components manually in the Inspector.

Prefabs

Prefabs are a great way to configure and store GameObjects for re-use in your game. They act as templates, storing the components and properties of a specific GameObject and enabling you to create multiple instances of it within a Scene.

All copies of the Prefab template in a Scene are linked. This means that if you change the object values for the health potion Prefab, for example, each copy of that Prefab within the Scene will change to match it. However, you can also make specific instances of the GameObject different to the default Prefab settings.

• On your Project panel select Assets > MixedRealityToolkit.Examples > Demos.

• Select from the folders that you want to see an example of, ex: HandTracking, EyeTracking...

• Open the Scenes folder and select a scene and double click to open.

• You can press play to try out the scene in your editor window.

The Unity Editor has four main sections:

Scene view

This is where you can edit the current Scene by selecting and moving objects in the 3D space for the game. In this kit, the game level is contained in one Scene.

Check your knowledge by answering below question before you move into the project. Feel free to skip sections you feel comfortable. Make sure you read through the first download section to make sure you have all the modules necessary.