Kinect v2 Examples with MS-SDK and Nuitrack SDK

Kinect2_MsSDKKinect v2 Examples with MS-SDK and Nuitrack SDK is a set of Kinect-v2 (aka ‘Kinect for Xbox One’) examples that use several major scripts, grouped in one folder. The package contains over thirty demo scenes.

Apart of Kinect-v2 and v1 sensors, the K2-package supports the Nuitrack body tracking SDK, as well as Orbbec Astra & Astra-Pro sensors. Please mind, the support for Nuitrack SDK and Orbbec Astra sensors is in early stage, and issues are possible.

The avatar-demo scenes show how to utilize Kinect-controlled avatars in your scenes, gesture-demos – how to use the programmatic or VGB gestures, fitting room demos – how to create your own dressing room, overlay demos – how to utilize the joint overlays, etc. You can find short descriptions of all demo-scenes in the K2-asset online documentation.

This package works with Kinect-v2, Kinect-v1 (aka Kinect for Xbox One & Kinect for Xbox 360), Nuitrack supported sensors, and Orbbec Astra sensors. It can be used in all versions of Unity – Free, Plus & Pro.

Free for education:
This package is free for academic use (i.e. in schools, colleges and universities, by students, teachers or researchers). If you match this criterion, please e-mail me to get the K2-asset directly from me.

One request:
My only request is NOT to share this package or its demo scenes in source form with others, or as part of public repositories, without my explicit consent. Please respect my work.

Customer support:
* First, please check if you can find the answer you’re looking for on the Tips, tricks and examples page, as well as on K2-asset Online Documentation page. See also the comments below the articles here, or in the Unity forums. If it is not there, you may contact me, but please don’t do it on weekends or holidays. This is the only time I can have some rest.
* If you e-mail me, please include your invoice number. I provide free e-mail support for the first 3 months after the product purchase (limited to max 10 issues). More information regarding e-mail support can be found here.
* Please mind, you can always update your K2-asset free of charge, from the Unity Asset store.

How to run the demo scenes:
1. (Kinect-v2) Download and install Kinect for Windows SDK v2.0. The download link is below.
2. (Kinect-v2) If you want to use Kinect speech recognition, download and install the Speech Platform Runtime, as well as EN-US plus the other needed language packs. The download links are below.
3. (Nuitrack or Orbbec) If you want to work with Nuitrack body tracking SDK, look at this tip. If you want to work with Orbbec Astra or Astra-Pro sensors via OpenNI2, look at this tip.
4. Import this package into new Unity project.
5. Open ‘File / Build settings’ and switch to ‘PC, Mac & Linux Standalone’, Target platform: ‘Windows’.
6. Make sure that ‘Direct3D11’ is the first option in the ‘Auto Graphics API for Windows’-list setting, in ‘Player Settings / Other Settings / Rendering’.
7. Open and run a demo scene of your choice, from a subfolder of ‘K2Examples/KinectDemos’-folder. The short descriptions of all demo scenes are available here.

* Kinect for Windows SDK 2.0 (Windows-only) can be found here.
* MS Speech Platform Runtime v11 can be downloaded here. Please install both x86 and x64 versions, to be on the safe side.
* Kinect for Windows SDK 2.0 language packs can be downloaded here. The language codes are listed here.

Documentation:
* The official online documentation of the K2-asset is available here.

Downloads:
* The official release of the ‘Kinect v2 with MS-SDK’ is available at Unity Asset Store. All updates are free of charge.

Troubleshooting:
* If you get compilation errors like “Type `System.IO.FileInfo’ does not contain a definition for `Length’”, you need to set the build platform to ‘Windows standalone’. For more information look at this tip.
* If the Unity editor crashes, when you start demo-scenes with face-tracking components, look at this workaround tip.
* If the demo scene reports errors or remains in ‘Waiting for users’-state, make sure you have installed Kinect SDK 2.0, as well as the other needed components, and the sensor is connected.

* Here is a link to the project’s Unity forum: http://forum.unity3d.com/threads/kinect-v2-with-ms-sdk.260106/
* Many Kinect-related tips, tricks and examples are available here.
* The official online documentation of the K2-asset is available here.

Known Issues:
* If you get compilation errors, like “Type `System.IO.FileInfo’ does not contain a definition for `Length’“, open the project’s Build-Settings (menu File / Build Settings) and make sure that ‘PC, Mac & Linux Standalone’ is selected as ‘Platform’ on the left. On the right side, ‘Windows’ must be the ‘Target platform’, and ‘x86’ or ‘x86_64’ – the ‘Architecture’. If Windows Standalone was not the default platform, do these changes and then click on ‘Switch Platform’-button, to set the new build platform. If the Windows Standalone platform is not installed at all, run UnityDownloadAssistant again, then select and install the ‘Windows Build Support’-component.
* If you experience Unity crashes, when you start the Avatar-demos, Face-tracking-demos or Fitting-room-demos, this is probably due to a known bug in the Kinect face-tracking subsystem. In this case, first try to update the NVidia drivers on your machine to their latest version from NVidia website. If this doesn’t help and the Unity editor still crashes, please disable or remove the FacetrackingManager-component of KinectController-game object. This will provide a quick workaround for many demo-scenes. The Face-tracking component and demos will still not work, of course.
* Unity 5.1.0 and 5.1.1 introduced an issue, which causes some of the shaders in the K2-asset to stop working. They worked fine in 5.0.0 and 5.0.1 though. The issue is best visible, if you run the background-removal-demo. In case you use Unity 5.1.0 or 5.1.1, the scene doesn’t show any users over the background image. The workaround is to update to Unity 5.1.2 or later. The shader issue was fixed there.

What’s New in Version 2.17.1:
1. Fixed Nuitrack-interface to work with Realsense D415/D435 & Orbbec Persee (thanks to Juha Kauppinen & Batuhan Yapanoglu).
2. Added NetworkAvatarDemo2-scene, to demonstrate first-person avatar control over the network.
3. Fixed UserAvatarMatcher-component used by KinectAvatarsDemo4, to track correctly incoming and outgoing users (thanks to Xiaoyang Zhu).

Videos worth more than 1000 words:
Here is a video by Ricardo Salazar, created with Unity5, Kinect v2 and “Kinect v2 with MS-SDK”, v.2.3:

..a video by Ranek Runthal, created with Unity4, Kinect v2 and “Kinect v2 with MS-SDK”, v.2.3

..and a video by Brandon Tay, created with Unity4, Kinect v2 and “Kinect v2 with MS-SDK”, v.2.0:

635 thoughts on “Kinect v2 Examples with MS-SDK and Nuitrack SDK

  1. Hi Rumen: I now get about 10 errors: Assets/KinectScripts/AvatarController.cs(12,14): error CS0101: The namespace `global::’ already contains a definition for `AvatarController’
    such as these. I think I followed your directions pretty well. Any ideas?

  2. hi ,
    Loving the asset
    I am working in untiy 2017.3 version.
    i have an issue in “KinectFittingRoom1” demo scene . there is color image turn opposite side.

    i am changing the shader “userBlendedshader” to “ForegroundBlendedShader” but the depth is gone ..

    please help me !!!

    and also Background removal scene not working

    • Hi, I just checked both FittingRoomDemo1 & BackgroundRemovalDemo1 with Kinect-v2 in Unity 2017.3 editor, and it worked as expected. Are you using Kinect-v2 or some other sensor?

      Please contact me by e-mail and send me some screenshots, and the Unity log-file as well/ This way I could see what happens at your end. Please mention your invoice number, too. Here is how to find the Unity log-files on your platform: https://docs.unity3d.com/Manual/LogFiles.html

      • Ok, already answered. Please check the version of the K2-asset you’re using – see ‘WhatsNew-Kinect2-MsSdk.pdf’ in the _Readme-folder. I think your copy is quite old, because similar issues were fixed already, in the more recent releases.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s