3D JavaScript engines like ThreeJS and BabylonJS use WebGL to render to Canvas to make it easier for JavaScript developers who are not an expert in computer graphics. WebGL is a is a cross-platform, royalty-free web standard for a low-level 3D graphics JavaScript API, or programmable interface, for drawing interactive 2D and 3D graphics in web pages. WebGL connects your web browser up to your device’s graphics card, providing you with far more graphical processing power than is available on a traditional website.

According to caniuse.com, 98.16% of the internet users globally are using WebGL 3D Canvas graphics capable device and browser.

Unlike most Web APIs, WebGL is designed and maintained by the non-profit Khronos Group and not World Wide Web Consortium (W3C). WebXR Device APIs and other related APIs to create a 3D experience like Gamepad API are part of W3C.

To learn more check out Wikipedia, Khronos WebGL Wiki or the official Knronos WebGL Repository.

WebXR Lessons: www.learnwebxr.dev

Wellcome to the introduction to WebXR Lessons. We will talk about:

Links

Short link to WebXR Lessons: www.learnwebxr.dev

FrontEndMasters WebXR Course:

Windows Mixed Reality JavaScript Documentation: aka.ms/WebXR

Why web is important for Mixed Reality Development?

Web is for all and web is always free. As a developer, you will always have the right to your own creation and distribution, which is not the case on any other platform.

• 3D on the Web

• Introduction to WebXR Device APIs

  • What are WebXR Device APIs?

  • Which Devices are Compatible with WebXR?

  • Which Browsers Support WebXR?

• Introduction to Babylon.js

  • What is BabylonJS?

  • How to Create a WebXR Augmented Reality App?

  • How to Debug and Test Tour WebXR Applications with Crome Dev Tools?

  • What is the Lifecycle of a WebXR Application?

World wide web allows anyone share their experiences with the world freely. With the availability of the technologies like WebXR, it is now possible to share immersive experiences on the web as well.

• Concepts

  • What is WebGL?

  • How to work with 3 dimensions, 3D coordinates?

  • What is a Vertex?

  • What is a Normal in 3D?

  • What is Field of View?

• Project

  • How to create a basic 3D scene?

Field of View (FOV) is the extent of the scene that is seen on the display at any given moment. The value is in degrees.

Read more on wikipedia.

The aspect ratio a camera defines the rendered image's width and height ratio using the camera.

You can use the window's inner height and inner width to calculate aspect ratio. If you are not rendering to the entire window, you can choose a specific size for your aspect ratio that would correspond to the container of the renderer.

Near and far clipping defines the render able area of a scene. Anything before near clipping and after far clipping won't be rendered by your camera, and won't be visible.

Visible area is called frustum.

A point in 3D that is represented with x, y and z, or Vector 3

glTF (GL Transmission Format)

Where possible, we recommend using glTF (GL Transmission Format). Both .GLB and .GLTF versions of the format are well supported. Because glTF is focused on runtime asset delivery, it is compact to transmit and fast to load. Features include meshes, materials, textures, skins, skeletons, morph targets, animations, lights, and cameras.

Public-domain glTF files are available on sites like Sketchfab, or various tools include glTF export:

• Blender by the Blender Foundation

• Substance Painter by Allegorithmic

• Modo by Foundry

• Toolbag by Marmoset

• Houdini by SideFX

• Cinema 4D by MAXON

• COLLADA2GLTF by the Khronos Group

• FBX2GLTF by Facebook

• OBJ2GLTF by Analytical Graphics Inc

• …and many more

When glTF is not an option, popular formats such as FBX, OBJ, or COLLADA are also available and regularly maintained by many libraries.

Normal is the direction a face of an object is pointing. It is used to correctly map images as textures to an object.

TODO: Examples...

Touch and Click events

If you are in a 3D scene on a 2D display, you can interact with the objects or buttons on you scene with a mouse or touch event, as you would in any other web app. One difference is the instead of checking if we hit an html element like button, we need to check the 3D position that we are hitting and check if there is any collisions with an object. We do not have a way to directly attach a touch or click event to a 3D object but we can check to see if the ray casted from our touch or click is passing through an object.

Raycasting

Hand Interactions

When we talk about transformations in 3D space, we mean altering the position, orientation, or scale of an object. These transformations can include various operations such as translation (shifting the position), rotation (changing the orientation), shear (distorting the shape), scale (changing the size), reflection (flipping the object), and projection (mapping the 3D point onto a 2D plane).

To perform these transformations, a common technique is to use a transformation matrix. A transformation matrix is a mathematical matrix that stores the information about the desired transformation. By multiplying a Vector3 representing a point(x,y,z position) with the transformation matrix, we can apply the transformation to that point.

//Three.js
const transformationMatrix = new THREE.Matrix4(); 
transformationMatrix.set( 11, 12, 13, 14, 
	   21, 22, 23, 24,
	   31, 32, 33, 34, 
	   41, 42, 43, 44 );
	

When we talk about "applying the matrix to the vector," it means that the transformation matrix is multiplied with the Vector3 point to achieve the desired transformation. The resulting Vector3 will reflect the transformed position, orientation, or scale of the original point in 3D space.

Read more

• Matrix4 in Three.js docs

• OpenGL matrices tutorial

To create a basic scene we need to initialize a scene object. You can think of a scene as a stage that will hold your production. Everything that will be visible to the viewer will be added to the scene.

We need to add a camera object to the scene that will be our viewers perspective. Anything on the scene but not in the view of the camera won't be visible on your canvas.

We also need a light source for our objects to be visible.

Finally, we will create a basic shape(mesh), box.

Let's create the basic scene using different libraries.

Overview

BabylonJS Code

To create a basic BabylonJS scene, we initialize a scene, create camera, light and a mesh and return the scene object.

Camera

Arc Rotate Camera acts like a satellite in orbit around a target and always points towards the target position. In our case, it is pointing towards the box. Where ever you move your camera, it will still point to towards the box.

Arc Rotate Camera parameters are: name you want to give to your camera, alpha(radians) the longitudinal rotation, beta beta (radians) the latitudinal rotation, radius(the distance from the target position), target position(center where the box will be created as a default), scene(optional argument). In this code example scene is not given as an argument and defaults to the scene object on the playground.

Light

A hemispheric light is an easy way to simulate an ambient environment light. In our case, we are doing the bare minimum, just giving a name to the light and set it's location.

Setting y location to 1 while our main object is in the center (0,0,0) location will move the light above the object.

Box Mesh

We are creating a Box Mesh by calling BABYLON.MeshBuilder.CreateBox method with the minimum requirements, a name and an options empty object.

Exercise

• Move your mouse over the box and click & drag.

• Uncomment line 5 and save or play the playground example.

• Again, move mouse over the box and click & drag.

• On the BABYLON.MeshBuilder.CreateBox method replace the empty object with this object: {width: 4, height: 2}.

Next Steps

Continue exploring how to create a 3D scene with other libraries

Although Three.js creates scene, camera and light as well as the object, the code syntax is little different.

Exercise

• Navigate to the code sample by clicking Edit in JSFiddle.

• Uncomment the lines 18 and 19 by deleting '/*' and '*/'

• Press run to see the animation.

• Change the values added to x and y rotation to see the result.

• Change the cube's position instead of the rotation value.

With a frame you can create a scene using <a-scene> html tag and nest the objects tags inside the scene. As a person who is used to working with Canvas and JavaScript, this is confusing to me but I can see why it is easier for some.

Good news is, you can use Three.js to further create interactions for your AFrame scenes.

• Checkout the github repo: https://github.com/Yonet/WebXRWorkshop

• npm install dependencies

• Add a sphere geometry

• Add light

    //light
    const light = new HemisphereLight(0xffffff, 0xbbbbff, 1);
    light.position.set(0.5, 1, 0.25);
    scene.add(light);

Texture and Bump map

Checkout the example below to see how bump map creates more realistic results.

• Add Texture Loader and bump and texture maps

// Material
const texture = new TextureLoader().load("assets/images/globe/earthmap4k.jpg");
const bumpMap = new TextureLoader().load("assets/images/globe/earthbump4k.jpg");
const material = new MeshPhongMaterial({
    // color: 0xffff00 * Math.random(),
    specular: 0x222222,
    shininess: 25,
    bumpMap: bumpMap,
    bumpScale: 10,
    map: texture,
});

Exercise:

• Create another sphere slightly bigger than the earth mesh and call it clouds.

• Assign MeshPhongMaterial and add the properties: {opacity:0.8, transparent: true}

• Load and assign earhclouds4k.png from the assets folder. Assign the texture as map of clouds material.

• Set the position of the clouds same as the earth mesh.

• Add animation slightly faster than earth.

To see how to map real time tweet location data to your globe using web sockets, check out Tweet Migration project.

Lights can change our scenes drastically. Understanding lighting can help you create the effect you intend.

Https server

You need to serve WebXR files over a secure https server. Serving over http server would not render your WebXR scenes.

Tooling

• Node.js download

• WebXR Emulator: Firefox, Chrome

• WebXR Viewer on Apple store

• Web Server for Chrome extension

• Visual Studio Code(VSCode) download

• VSCode Live Server Extension.

• TypeScript in 5 minute introductions

• Learn X in Y: TypeScript

BabylonJS

• VSCode BabylonJS File Viewer Extension

• BabylonJS Viewer

• BabylonJS Tools and Resources

ThreeJS

• ThreeJS Github

• ThreeJS Chrome Dev Tool

• ThreeJS Web Editor

• ThreeJS Discord

• ThreeJS Subreddit

WebXR is a group of standards being implemented by the browsers, which are used together to support rendering 3D scenes to hardware designed for Mixed Reality. Mixed Reality devices are presenting virtual worlds (virtual reality, or VR), or for adding graphical imagery to the real world, (augmented reality, or AR).

The WebXR Device API implements the core of the WebXR feature set, managing the selection of output devices, render the 3D scene to the chosen device at the appropriate frame rate, and manage input, such as controllers and hand interactions.

WebXR Device APIs are replacing the deprecated WebVR API. WebVR API was designed with only VR devices in mind. With the addition of new AR headsets and AR capable handheld devices, the WebVR API is deprecated in favor of WebXR Device APIs, that include AR Modules.