Experiments with HoloLens, Mixed Reality Toolkit and two-handed manipulations
- 2 minsI’ve always been a big fan of manipulations, as in the past I worked on some multi-touch XAML Behaviors implementing rotate, translate and scale on 2D objects.
As I progress with my learning about HoloLens and Windows Mixed Reality, I had on my to-do list the task of exploring how to recreate this scenario in the 3D Mixed Reality context. Finally, during the weekend, I started some research while preparing a demo for some speaking engagements I’ll have over the next weeks.
As usual, the Mixed Reality Toolkit is a fantastic help for speeding up development, so I started my investigation analysing the GitHub repo and found that the dev branch now contains a new readme illustrating all the steps required to use a new Unity script TwoHandManipulatable.cs which enables rotate, translate and scale to 3D objects in Unity using two hands tap gestures with HoloLens and the Motion Controllers with the Immersive headsets.
I decided to give this a try using a model imported from the Remix 3D repository.
Importing the model from Remix 3D
I fired up Paint 3D and selected More models to explore the ones available online in Remix 3D: this is a great container of assets you can use in Mixed Reality apps. I chose to explore a model of the Mars Rover, so I selected and opened it:
Then I exported it as a 3D FBX object to be able to import in Unity and saved it on my local machine as MarsRover.fbx.
Creating the Unity project
I started a new Unity Project, and copied the folder Assets\MixedRealityToolkit from the dev branch of the MRT GitHub repository.
After applying the Mixed Reality Project and Scene settings from the corresponding menu, I was ready to import the Mars Rover model and associate the manipulation script.
I selected Assets->Import New Asset, searched for the previously saved mode MarsRover.fbx and adjusted the scale to X:60, Y:60, Z:60 to have it correctly visualised in my scene. Then, I inserted a new Capsule Collider for enabling interaction with the object:
Adding the TwoHandManipulatable script
After selecting the imported model, I actioned the Add Component button from the inspector tab and searched for the Two Hand Manipulatable script in the toolkit and then added it to the asset together with a BoundingBoxBasic for showing the boundaries when applying manipulations.
And set the manipulation mode to Move Rotate Scale.
The scene was completed, so I only had to test it with the device: I selected the File->Build Settings, generated the package and deployed it to HoloLens, and I got the Mars Rover ready to be explored with two-handed manipulations. Amazing!
The source code is available on my GitHub playground here.
Davide Zordan
Senior Software Engineer