KinectExtras with MsSDK

KinectExtrasMsSDKKinectExtras with MsSDK is an extension of the Kinect with MS-SDK package. It provides additional examples and demos, based on Kinect SDK that are not directly related to transferring human motion onto digital characters. The package consists of Kinect Interaction demo presenting hand cursor control as well as hand grips and releases, Speech recognition example, Face-tracking example and Background-removal demo. It is possible to integrate all Kinect-related features in one project, i.e. to make KinectExtras and KinectManager (from ‘Kinect with MS-SDK’-package) work together.

This package is for Kinect v1 only. It supports 32-bit and 64-bit builds and can work in both Unity Pro and Unity Free editors.

Free for Education:
This package is free for academic use (i.e. for schools, universities, students and teachers). If you match this criterion, please contact me by e-mail to get the KinectExtras asset directly from me.

Known Issues:
Unity 5.0 and later introduced an issue, which causes in many cases Kinect tracking to stop after 1-2 minutes with DeviceNotGenuine error. As a result you will see that the Kinect-enabled game freezes. There is no definite workaround yet. My advice is: If you encounter this issue, please install Unity 4.6 along with Unity 5 (just in another folder) and use the K1-assets in Unity 4 environment. See the updates here.

How to Run the Examples:
1. Download and install Kinect SDK 1.8 or Kinect Runtime 1.8 from here.
2. Download and import the KinectExtras-package.
3. Open and run the scene of your choice, as described in the corresponding Readme pdf-file, located in the Assets-folder.

How to integrate KinectExtras with KinectManager:
To integrate the KinectExtras with the KinectManager (from ‘Kinect with MS-SDK’-asset), do the following:
1. Open KinectScripts/KinectWrapper.cs and near its start uncomment: #define USE_KINECT_INTERACTION_OR_FACETRACKING.
2. To integrate Kinect speech recognition with the KinectManager as well, uncomment also: #define USE_SPEECH_RECOGNITION.
3. Open KinectScripts/InteractionWrapper.cs and near its start uncomment: #define USE_KINECT_MANAGER.
4. Open KinectScripts/FacetrackingWrapper.cs and near its start uncomment: #define USE_KINECT_MANAGER. If you plan to use face-tracking in the scene, don’t forget to enable the ‘Compute color map’-setting of KinectManager-component. It is needed for the face-tracking to work.
5. If you have uncommented ‘USE_SPEECH_RECOGNITION’, open KinectScripts/SpeechWrapper.cs and near its start uncomment: #define USE_KINECT_MANAGER.
6. Make sure that both KinectManager and the Extras’ managers, like the InteractionManager, FacetrackingManager or SpeechManager are components of the MainCamera (or other persistent game object).
7. Open KinectScripts/KinectManager.cs and at the start of its Awake()-method add this line: ‘WrapperTools.EnsureKinectWrapperPresence();’, in order to ensure the presence of the needed native libraries. Also, make sure that the Assets/Resources-folder from KinectExtras-package exists in your project.
8. If you use only the KinectManager-component (but not any of the KinectExtras-managers) in the scene, Open KinectScripts/KinectManager.cs and at the start of its Update()-method add this line: ‘KinectWrapper.UpdateKinectSensor();’. Normally, the KinectExtras-managers do that for you.
9. Run the scene to check if the integration works properly and without errors.

Download:
The official release of ‘KinectExtras with MsSDK’-package is available at Unity Asset Store.
The project’s Git-repository is located here. This repository is private and its access is limited to contributors only.

Troubleshooting:
* If you get DllNotFoundException or other initialization errors, make sure you have installed the Kinect SDK 1.8 or Kinect Runtime 1.8.
* Kinect SDK 1.8 and tools (Windows-only) can be found here.
* The examples were tested with Kinect SDK 1.7 and 1.8.
* Here is a link to the project’s Unity forum: http://forum.unity3d.com/threads/222517-KinectExtras-with-MsSDK

What’s New in Version 1.8:
1. Added support for x32/x64 architecture at runtime.
2. Improved face tracking – fixed native routine responsible for skeleton selection.
3. Updated mouse cursor control to match the game window.
4. Updated speech error handling to display messages instead of error codes.
5. Renamed Extras-folder to Samples to match the respective folder of “Kinect with MS-SDK”

Playmaker Actions for ‘Kinect with MS-SDK’ and ‘KinectExtras with MsSDK’:
And here is “one more thing”: A great Unity-package for designers and developers using Playmaker, created by my good friend Jonathan O’Duffy from HitLab-Australia and his team of talented students. It contains many ready-to-use Playmaker actions for Kinect v1 and a lot of example scenes. The package integrates seamlessly with ‘Kinect with MS-SDK’ and ‘KinectExtras with MsSDK’-assets. I can only recommend it!

160 thoughts on “KinectExtras with MsSDK

  1. Hi,
    I cannot find the kinect extras in asset store, it says no longer available. (kinect ver1)
    Our school just recieve a kinect ver 1 donation and i really need to learn it before, because we will use it for material in next semester.
    And the second option from github also not available.
    Can you tell me where the new download page or is it not available anymore?
    Sorry for my bad English.

    • Hi there, I deprecated the KinectExtras-asset long time ago, because it is now fully integrated into the K2-asset. If you still need it, send me an e-mail and I’ll send you back the KinectExtras-package.

  2. Hello Rumen!
    I have some problem with ‘KinectUnityWrapper.dll’
    >>System.DllNotFoundException
    I’ve tried to fix it for 3 days but I could’t… TT

    Could you send me KinectExtras package? I really need it for my school project TT

  3. I was wondering, with this package, is it possible to create a Grab Gesture? I looked on the Kinect v1 MSsdk KinectGesture.cs and i can’t find a gesture for grabbing. Thanks

  4. Hi!

    I’m using both packages Kinect and Kinect Extras V1, After I build the project the program closes due to the following errors:


    Fallback handler could not load library C:/Projects/Unity/fast-hud-project/bin/EXEs/Tests/Test_Data/Mono/KinectUnityWrapper
    Fallback handler could not load library C:/Projects/Unity/fast-hud-project/bin/EXEs/Tests/Test_Data/Mono/libKinectUnityWrapper.dll
    Fallback handler could not load library C:/Projects/Unity/fast-hud-project/bin/EXEs/Tests/Test_Data/Mono/.\libKinectUnityWrapper.dll

    System.DllNotFoundException: KinectUnityWrapper.dll
    at (wrapper managed-to-native) KinectWrapper:EnableKinectManager (bool)
    at KinectWrapper.NuiInitialize (NuiInitializeFlags dwFlags) [0x00000] in :0
    at KinectManager.Awake () [0x00000] in :0

    System.Exception: Error initializing Kinect/SAPI: 0x80070015
    at SpeechManager.StartRecognizer () [0x00000] in :0

    seems like is not being included into the final binary, I’m using API compatibility .NET 2.0 (mandatory) on unity 5.4.3f1 32bit.

    I appreciate your help. It’s an emergency HELP please!!

    • All needed dlls should be in the Assets/Resources-folder, with extensions ‘.dll.bytes’. At runtime they are copied to the root-folder of your project (the parent folder of your project’s Assets-folder), and then loaded by Unity from there. Make sure you have imported the Resources-folder from KinectExtra-asset into your project. And, as you use both the Kinect-asset and KinectExtras-asset, make also sure that you have followed the steps from ‘How to integrate…’-section above.

  5. Hi Rumen, is the kinect extras still available? Could you please send me a copy? I need it for my college assignments. Thanks

  6. for the Playmaker Actions example for ‘Kinect with MS-SDK’ and ‘KinectExtras with MsSDK’: the playmarker package as I can co-operate

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s