Kinect with MS-SDK is a set of Kinect v1 examples that uses several major scripts, grouped in one folder. It demonstrates how to use Kinect-controlled avatars, Kinect-detected gestures or other Kinect-related stuff in your own Unity projects. This asset uses the Kinect SDK/Runtime provided by Microsoft. For more Kinect v1-related examples, utilizing Kinect Interaction, Kinect Speech Recognition, Face Tracking or Background Removal, see the KinectExtras with MsSDK. These two packages work with Kinect v1 only and can be used with both Unity Pro and Unity Free editors.
Known Issues:
Unity 5.0 and later introduced an issue, which causes in many cases Kinect tracking to stop after 1-2 minutes with DeviceNotGenuine error. As a result you will see that the Kinect-enabled game freezes. There is no definite workaround yet. My advice is: If you encounter this issue, please install Unity 4.6 along with Unity 5 (just in another folder) and use the K1-assets in Unity 4 environment.
Update 12.Aug.2015: The bug-report, along with all details, is in the hands of Unity staff now, for reproduction and fix.
Update 07.Sep.2015: Here is what I got from the Unity support stuff: “We have checked that other people have the issue outside of Unity also, the entire device goes in a broken state. That indicates a driver bug. If we pop up the windows device manager, we can see the device is broken after it gets into this state and even unplugging it and plugging it back in doesn’t fix it so it’s an issue on Microsoft’s driver.”
Update 10.Sep.2015: One other asset user – Thank you Victor Grigoryev! – has experimented with multiple configurations. Here are his conclusions:
- If I connect the Kinect to USB 3.0 on Windows 8 and 10, it works perfect. But the issue was on my Windows 7 machine with USB 2.0 only (no USB 3.0 on motherboard at all).
- When I tried to connect to USB 2.0 on Windows 8 or 10, I’ve seen some lags from kinect, but it still worked.
- On Win 7 – I reinstalled Kinect SDK and RESTARTED the computer then. Without restarting nothing will change. I checked that several times. And not to restart was one of my major mistakes before.
Update 29.Oct.2015: I posted the issue on the MSDN Kinect-v1 forum here. Please up-vote the post and comment it with your own case and machine configuration (CPU, OS, USB port, sensor, Kinect SDK, Unity version, etc.), if you suffer from this issue. This is the last thing we could do, in order to ask the Kinect staff provide a workaround or SDK update.
Last Update, FYI: The best workaround for this issue, as reported so far by different users, is as follows: Uninstall all Kinect-v1 related packages (SDKs, drivers, OpenNI, NiTE, Zugfu, etc.), restart, install Kinect SDK 1.8 only, and restart again.
How to Run the Example:
1. Install the Kinect SDK 1.8 or Runtime 1.8 as explained in Readme-Kinect-MsSdk.pdf, located in Assets-folder.
2. Download and import this package.
3. Open and run scene KinectAvatarsDemo, located in Assets/AvatarsDemo-folder.
4. Open and run scene KinectGesturesDemo, located in Assets/GesturesDemo-folder.
5. Open and run scene KinectOverlayDemo, located in Assets/OverlayDemo-folder.
6. Open and run scene DepthColliderDemo, located in Assets/DepthColliderDemo-folder.
Download:
The official release of ‘Kinect with MS-SDK’-package is available in the Unity Asset Store.
The project’s Git-repository is public and is located here. This repository is private and its access is limited to contributors and donators only.
Troubleshooting:
* If you need integration with the KinectExtras, see ‘How to Integrate KinectExtras with the KinectManager’-section here.
* If you get DllNotFoundException, make sure you have installed the Kinect SDK 1.8 or Kinect Runtime 1.8.
* Kinect SDK 1.8 and tools (Windows-only) can be found here.
* The example was tested with Kinect SDK 1.5, 1.6, 1.7 and 1.8.
* Here is a link to the project’s Unity forum: http://forum.unity3d.com/threads/218033-Kinect-with-MS-SDK
What’s New in Version 1.12:
1. Updated AvatarController to use the Mecanim configured bones. Big thanks to Mikhail Korchun!
2. Added AvatarControllerClassic-component to allow manual assignment of bone transforms. Big thanks to Aaron Brooker!
3. Added ‘Offset relative to sensor’-setting to AvatarController and AvatarControllerClassic, to provide the option to put the avatar into his real Kinect coordinates. Big thanks to Claudio Rufa!
4. Added depth-collider demo scene, to demonstrate the mapping of Kinect space and depth coordinates to Unity world coordinates, and how this can be used for VR collisions.
5. Added gestures debug-text-setting to KinectManager to enable easier gesture development.
6. Updated detection of the available gestures, to make them more robust and easier to use.
7. Fixed sensor initialization, when the speech manager from KinectExtras is integrated.
Playmaker Actions for ‘Kinect with MS-SDK’ and ‘KinectExtras with MsSDK’:
And here is “one more thing”: A great Unity-package for designers and developers using Playmaker, created by my good friend Jonathan O’Duffy from HitLab-Australia and his team of talented students. It contains many ready-to-use Playmaker actions for Kinect v1 and a lot of example scenes. The package integrates seamlessly with ‘Kinect with MS-SDK’ and ‘KinectExtras with MsSDK’-assets. I can only recommend it!
Hi,
Really appreciate your work on Kinect with Unity. I have been able to use Speech and Skeleton services in my unity projects but now the issue is when I am trying to use both of them together.
In KinectWrapper.cs, I have enabled USE_SPEECH_RECOGNITION
In SpeechWrapper, I have enabled USE_KINECT_MANAGER
Both of them added (KinectManager and SpeechManager) to the MainCamera
Now when I start the game, Kinect fails to initialize. I have tried playing around with the code a bit but have not been able to work it out.
Searching around the net, I was not able to find any information. Would be really great if you can give some pointers to enable both of them to work together in a single scene.
-Manpreet
Hello Manpreet,
Have you copied the Assets/Resources-folder from KinectExtras-examples to your pruoject. There are important dlls and other files (such as grammar-file) there that need to be copied from the Resources-folder to the root-folder of your Unity-project in order to make it work. Please check this out and keep me posted, if the problem persists.
Best,
Rumen
Hi Rumen,
Thanks for the quick reply. Yes, everything is copied. If I run them individually in the same project, everything works. E.g. if I remove the speech manager component from MainCamera, then Kinect skeleton starts working. The moment, I add the speech manager back, it stops working.
Regards,
Manpreet
Hi Manpreed,
Please try to download and replace the Kinect wrapper at the root folder of your Unity-project with its updated version: https://www.dropbox.com/s/wu5kwj14o4wpdzk/KinectUnityWrapper.dll?dl=1
I suppose you use only the Speech recognition-component of KinectExtras-package. There is a bug for this specific case, in the wrapper in the KinectExtras-package, currently uploaded to the Unity AssetStore. This will be fixed in the next version of the package.
Please then try again and tell me, if this fix works for you.
Best,
Rumen
Pingback: [#UNITY3D] HowTo: Utilizar TFS para gestionar los archivos de un proyecto Unity3D | El Bruno
Pingback: KinectExtras with MsSDK | The Engineer's Blog - R. Filkov
Hello! Really appreciate your work!
I am doing a project for which I need to access specific SDK functions that should give me access to raw Kinect data in meters. Apparently there is a function in the SDK, called “MapSkeletonPointToDepthPoint” that should serve my purpose. I see that you’ve implemented that in the KinectWrapper.cs, but you do not call directly the dll (of course, because of the .NET compatibility issue),
I was wondering, is the implementation actually based on the contents of the kinect dll? I am sorry if my question may sound stupid, but it really is a doubt of mine.
Thank you,
Best Regards,
Francesca
Hello Francesca,
To your question is: Yes, as far as I remember. This function maps a 3d coordinate (in meters) to 2D depth-map (in pixels). You can see it in real-time work, if you tick ‘Compute User Map’, ‘Display User Map’ and ‘Display Skeleton Lines’-parameters of KinectManager. The DrawSkeleton()-function of KinectManager-script uses this function to map the skeleton joints to user/depth texture points.
Greetings,
Rumen
Thank you Rumen for your quick answer!
I feel the need to thank you again for being so thoughtful of the community and releasing it for free, I hope you realize how much of a help you have been for beginners and amaterus, like me. I wish there were more people like you! Keep up the good work.
Best Regards,
Francesca.
Hi, this example is amazing. i try to work on the green screen function in the unity3d with kinect. Now, i can render the user map on the plane. how can i combine the user map with picture?
Hi, Andy,
enable both ‘Compute User Map’ and ‘Compute Color Map’-settings of the KinectManager-component of MainCamera.
Thank you Rumen
i try this setting. i can get the image which only contain the user. the background is removed. i want to make a green screen function that similar to kinect example “GreenScreen-WPF”. i don’t know how to combine the user map(texture2d) with other texture2d.
best regards
andy
Hi Andy, just enable the both settings. The textures will be combined automatically. If you also enable ‘Display User Map’-setting, you will see the result on the screen.
Pingback: Kinect to Playmaker | RFilkov.com - Technology, Health and More
Hi Rumen,
Thank you for providing the amazing things for us.
I have a question, how can I attach the 3D object in front of the skeleton on color map?(e.g. take a virtual gun or wear the virtual clothes)
Many thanks,
Macy
Hi Macy,
I had this question several times now. Obviously it is an interesting topic 🙂 This is a matter of point and direction calculation. Probably I’d need to provide an example soon. Please Email or Skype me, if you want to discuss this topic further.
Greetings!
Macy, please send me a email, if you want to get a sample scene of a 3d-ball tracking the hand movements.
R.
Hi Rumen,
Thank you for your quickly reply and help.
I have send you a email.
Many thanks,
Macy
hi Rumen , thanks for the great package…is it possible to use get a joint position from kinect with playmaker actions for multiple users ?
Hi Ozan,
GetJointPosition-action returns only positions of a joint of Player1. To use similar action for Player2, you can duplicate this script to (let’s say) GetJointPositionP2-action and change GetPlayer1ID()-invocations in the script with GetPlayer2ID().
Hope this helps.
Greetings!
Thanks for the fast reply Rumen, I will give it a try 🙂
Hi Rumen!
Great work!
I have played with your package and I wonder how could I “read” a complete gesture form another script that is attached to another object?
Best regards,
Tazy
Hi Tazy,
To your question:
1. You can use a gesture listener. As an example see SimpleGestureListener.cs, or
2. Use code like this:
KinectManager manager = KinectManager.Instance;
if(manager && manager.IsInitialized() && manager.IsUserDetected())
{
uint userId = manager.GetPlayer1ID();
if(manager.IsGestureComplete(userId, KinectWrapper.Gestures.SwipeLeft, true))
{
// do something
}
}
Greetings!
Thank you!
It works fine.
Best regards,
Tazy
Pingback: Kinect 二三事 | Kennedy Wong
Pingback: Kinect – Pong Demo | Kennedy Wong
Pingback: Kinect – Balance Demo | Kennedy Wong
Pingback: | Kennedy Wong
Hi Rumen!
Great work !!.
is there any possibilities, we can record the player movements and store it temporarily and play
Thanks,
Arun
Great Work Rumen! 🙂
Is there any possibilities, can we record player movements as video and store and play in game.
Thanks,
Arun
Hi Arun,
Not really. If you want to store video, you just need to store the main camera output (or part of it).
Greetings!
Thanks Rumen!. will check it.
Pingback: Kinect – Sim Planting v0.3 | Kennedy Wong
hi Rumen !
I saw you’r updated version of kinect with ms-sdk, it’s a great job !
I have a question about UserMap, i hope you’ll be able to answer 😉
I want usermap only show the player1, and in updateUserMap() we have :
ushort userMap = (ushort)(usersDepthMap[i] & 7);
How to compare this ushort userMap with uint player1ID ?
i’m sorry for my english 🙁
Hi Renaud,
Yes, I’m able to answer 🙂
ushort userMap is not playerID, but the player index. As you know Kinect is able to track up to 6 players. So, this is the index of the tracked player. The indexes of player1/2 are currently not saved, although you can slightly modify the script and do it. Anyway, if you send me an e-mail, I can send you an updated script, where these indexes are saved for convenience, so you could use them.
Greetings!
Hi, Rumen,
I have question about Gestures.RightHandCursor. More specifically about SetScreenPos. You make this :
Vector3 relHandPos = handPos – gestureData.tagVector;
gestureData.screenPos.x = Mathf.Clamp01(relHandPos.x / gestureData.tagVector2.x);
gestureData.screenPos.y = Mathf.Clamp01(relHandPos.y / gestureData.tagVector2.y);
I don’t understand where you initialise and/or update gestureData.tagVector. Why using relHandPos instead of handPos ?
I tried this gesture, and i feel cursor suddenly move fast when we triy to focus on the center of the screen, or maybe when the right hand come close to the right shoulder …
I have also give a look on KinectExtras, and it’s different. On KinectExtras, screenpos is just a clamp of handPos … Maybe handPos is not the same, it’s bring by KinectUnityWrapper.dll .
Do you advise me to use KinectUnityWrapper.dll for more efficient ?
anyways, i love your work,
Greetings !
Hi Renau,
Look some lines above in the code and you will find it. As far as I remember, tagVector2 contains the size of the interaction box (between shoulders, neck and hips) and tagVector contains the current position of the interaction box. Cursor position is calculated relative to this box. Cursor moves fast, probably because the box is not correctly calculated in some cases. By all means, use Kinect Interaction, if you have the KinectExtras-package. It is far more accurate than my humble cursor-gestures’ calculations.
Greetings!
Hi Rumen,
I did as you advise me, and it work great.
Thank you very much!
I will get the kinect V2, i will keep a look on your work, tell us when you’ll share the plugin for kinect 2 😉
Greetings
Here are the K2 examples: https://www.assetstore.unity3d.com/en/#!/content/18708
They’re based on the Microsoft’s own Unity plugin.
hi,
i’m working with kinect for windows and unity3d, i have my own plugin but i’m thinking on use your plugin because we dont have time to update ours. but i like to know if, do you have plans to make a version for kinect 2 (the xbox one kinect) ?, and if you do, do you have approximate release date for that version ?
Thanx.
Hi there,
There is a MS plugin for Kinect v2. And yes, I’m working on similar Unity package like for ‘Kinect with MS-SDK’. I’m going to submit it to Unity asset store very soon. If you like to get a copy of that package, before it gets officially published, please send me an e-mail.
Greetings!
Hey I’m starting to work with your asset and it looks great, but I have a question, I want to replace the avatar that is in the asset, and Im wondering how to do it.
Hope you can help me!
Thanks!
Hey I’m starting to work with your asset and it looks great, but I have a question, I want to replace the avatar that is in the asset with another that I have, but the skeleton its quite different, so I wonder if there’s a way to restructure the skeleton of the example and put the new model.
Hope you can help me!
Thanks!
Hi there,
First, your avatar needs to be in T-pose. Put the avatar somewhere in the scene and parent it to an empty control object. This object determines the initial position and rotation of your avatar in the scene. Then attach AvatarController-script as a component to the avatar and assign skeleton joints to the corresponding script parameters (Kinect-controlled joints). Also, pay attention to the section ‘Why There Are Two Avatars in the Scene’ of Readme-Kinect2-MsSdk.pdf
Greetings!
Hey!! Thanks for your quick answer!! I will try that! But I have another question, I’m trying to make the 3d model to collide with other things, I have a Capsule in the scene, I’m giving to the Character a Rigid Body and a Mesh Collider, in the Mesh Collider I’m giving the mesh of the Character U_Char, the one that comes with the asset, but when the model hits the capsule, nothing happens, so I wanted to know if there’s a way to make this work.
Thank you!
You have probably forgot a Collider or RigidBody somewhere. The model in the example is a standard Unity humanoid model, nothing special. Physics has nothing to do with the model or the Kinect control, as to me.
Rumen thanks a lot for your help! I try what you told me but, the collider mesh it’s not working, the solution that I found was give the part that I want to collide a cube and eliminate the mesh filter and with scripting make this cube to follow the part that I want.
Thanks a lot for your help!!
Hello, thank you for this package. I’m having a problem when I place forever run of an error warning of unity that goes like this: “Can not initialize Skeleton Date at KinectManager.Start () [0x00040] ..”. I do not know what else to do for the example to work, please you have no idea what it is?
Hi,
Never had an error like this. No idea what it means. ‘Kinect with MS-SDK’ usually works directly out of the box.
Greetings!
Hello, some of you know if is posible to move two avatars with two different kinects on the same Unity scene?
Hi, with the current setup you can only move up to two avatars tracked by one Kinect.
Greetings!
Hi Rumen,
I received Kinect2 recently, i started work on your plugin and it works great!
I wonder when MS will update their SDK, and if you intend to share a new plugin on asset store?
Anyways, your work is great, congrats!
Greetings!
Hi Renaud,
MS should release the official beta this month, as far as I know. Here is a link to the K2-plugin and example scenes on Asset store: https://www.assetstore.unity3d.com/en/#!/content/18708
Greetings!
hi Rumen,
i’m working with kinect2 and your plugin for unity. Your plugin wrok fine, again congrate.
But i have some problem with kinect. When i try it on SDK browser, i have same problem. My frame rate fall unacceptably. But not every time. Some time, i don’t get why or when, it work fine, some time it lag.
Do you get his problem too ? Do you know how to fix it? is-it related to USB3.0 ?
Hi Renaud, no I didn’t have such an issue. I’ve noticed in the forums that FPS may be related to USB3. Which app do you use when this happens?
i tried many apps, some from kinect browser ( coordinates mapping Basic for exemple), some from unity, you’re plugin and MS exemples …
I tried to change SDK, i had publicPreview-1407 and i moved to devPreview-1404. It work fine now !
I read requirements and i have USB 3.0 from INTEL, MS Windows 8.1 Pro 64-bit, AMD Radeon HD 6700 Series. I don’t think it is about hardware …
MS still consider their SDK 2.0 publicPreview-1407 as Beta. Is it any bug ?
If you mean if the SDK still has bugs, the answer would be yes and no at the same time. They fix old bugs and introduce new. But this is normal, and I hope that SDK and the final sensor will be finally out soon.
Ok, ty for quick answer, i hope so ^^
Hi Rumen,
I’m loving your package for Unity! Quick question, though, I see a method in KinectManager.cs to grab the raw depth data into a short[], but is there a method to grab the raw rgb data?
Hi jagwire,
No, there isn’t such method, but you could use GetUsersClrTex() to get the RGB data as texture.
Rumen, thanks! As a debugging measure, I’m rendering the UsersCltTexture to a GUITexture and it’s rendering upside-down and backwards. Is this intended behavior?
No it wasn’t, but this can be easily fixed, if you set XY-Scale (or Pixel Inset’s Width/Height) to negative values. Hope this helps.
Rumen,
I’m actually piping the rgb data to a DLL that does some image recognition. Any idea why it comes out upside down and backward?
jagwirex, I’m on vacation already and can’t look at all the details. But as far as I remember the image from NuiInterface comes like this. You can change it in a function named PollColorData() (or something like this) in KinectScripts/KinectWrapper-script. Just research it a bit, it’s not that hard.
I have a problem about the piece, where am i able to e-mail the author?
My contact info is on the About-page.
Hello is very interesting your project, congratulations, i hava a question, Is posible view the avatar in a project of visual studio, in wpf form?????
thanks
This is a Unity-oriented project, not VS or WPF. I suppose you’d need to adjust the scripts to the WPF requirements.
I need to view the avatar in wpf form
I am having trouble downloading the Kinect with MS-SDK from the Unity Asset Store. Is there anything I should be doing to that I could be doing wrong? Should I just cut my losses and donate so I can access it on git?
This should be an issue at the Asset store. Have you tried to contact their support team? Anyway, if your problem persists, send me an e-mail and I’ll send you back a download link to this package.
bookmarked, excellent website!
Hello,Rumen,
Thanks for sharing this great work.
we use Kinect with MS-SDK in my college project .
But now I have no idea how to code about detecting two skeleton to import two module.
Many thanks.
Joe
Hello Joe. First you need to enable the ‘Two users’-setting of KinectManager-component of the MainCamera. Then you need to manually drag the game objects that contain AvatarController-components to the respective lists of KinectManager – ‘Player1 Avatars’ or ‘Player2 Avatars’. Hope this helps.
Thank you for your help, I”ll try it.
Hi Rumen, a huge thanks for this tool you release
I made a game that base on the Avatar given in Demo , well then I face a problem that I want to replace the Avatar with user map ,make it like a game that user “in” it, is there any way I can reach that goal?
Another question , can I display a User map at the exactly (x,y) I would like it to ?
I’m still trying to figure it out by studying the example “KinectOverlayDemo” , is this the right direction I’m heading?
thanks again
Hi, the user map is a texture, so you can manipulate it in any way you like. For instance, if you enable ‘Compute User Map’ and ‘Compute Color Map’-settings of KinectManager, you will get a user-map texture with opaque pixels there, where the user is and all other pixels will be transparent. Then you can combine it with another (background) texture. The overlay demo can show you, for instance, how to put the texture on the whole screen. This demo is aimed more to show how to align a 3d-object to the color texture.
Thanks , I found the solution in the KinectOverlayDemo , though there’s still some problem in it.
thanks again.
Hi Rumen,
I just notice that if I enable the ComputeUserMap it eats up ALOT of CPU. Do you have any recommendation on how to handle this since we needed the UserMap on our project?
Regards,
Jhay
Hi Jhay.
This settings causes KinectManager.Update() to call UpdateUserMap()-function each time, which is data- and calculation intensive. You can either call this function less frequently, optimize its code for your specific case or move parts of the code to the GPU instead of CPU. I’m not very competent in the last one, but there are some manuals on the topic, I think. Hope this info helps you resolve the issue. Keep in touch.
Hi Rumen,
Many thanks for your work, right now I’ve been working on kinect project. One of your works, that is Kinect V2 with MS-SDK as being my references, it actually works but one again as my references is Kinect and KinectExtras with MS-SDK Playmaker Actions, with this one, I’ve got problems that I could not be able to interact with others object for example I would like to make an object sphere as my cursor using playmaker action Kinect 3D Hand Cursor but that it not works and also other example like Kinect Gestur Kinect get joint position etc. What am I suppose to ?
Thank you so much.
Sincere,
Dandhi
Hello Dandhi, I’m not sure I understood exactly what your problem is, but I suggest it is that you can’t use ‘Kinect and KinectExtras with MS-SDK Playmaker Actions’ together with ‘Kinect V2 with MS-SDK’. First of all, the Playmaker-actions-package is aimed to Kinect v1. The author of this package is my friend Jonathan O’Duffy and his team from HITLab Australia, so if this is the case, please e-mail them. Their support e-mail is listed in Unity Asset Store. Apart of that, there are some Playmaker actions packed into the K2-package. You can use them directly, but they are not many and are a bit limited. Hope this info helps. Greetings!
Hi Rumen, Thanks a lot for your replying, I’ve been trying one by one all of your PlaymakerKinectAction from Detect gesture till Track Hand Cursor, all that things is really great, but on Track hand Cursor I’ve got a little bit problem I could be able to click but for grip and release, these 2 kind of action are not easy to do.
for instance I make new scene of project on that project I would like to make cube change the colour when it’s clicked, then when it was griped it could change the colour and it’s ready to move then when it’s released cube move to pointed place.
On this project I only just did for clicking and others action are failed. would you like to help me ?
many thank to you..
Sincere,
Dandhi
Hi Dandhi, can you please contact me by e-mail. I’ll send you an updated version of the TrackHandCursor action and a demo scene. Greetings and Happy new year!
Sorry on my replying I used K2-package, inside of it there is PlaymakerKinectAction. That package has already worked on my project, that is great for you. and I got a little bit problem on the project when Track Hand Cursor was implemented with playmaker particularly on grip and release action. would like to help me…
Hi Rumens,
Thanks you much for your replying, I’ve already sent you an email.
I am looking forward to receiving the documents as soon as possible.
Once again, many thanks to you for helping me,
Best regards,
Dandhi
Pingback: Kinect SDK 2 FINGER TRACKING (etc) with Mac OS X, Windows, Desktop and Large Screens (VR) | Erik Champion
Hi! I’ve been using your great scripts, and just had an issue: I just upgraded my oculus scripts to the DK2 ones in my (~1 year old) package. When I did this, the User Map would no longer display. It seems to try, but perhaps its now on a lower level? I’m not sure how to fix this… any ideas? Thanks so much for your great work!!!
Hi Keith, is there an error message in the console window or not?
No there’s no error message. Works normally, and the user is puppeting the character, but the “user silhouette” just doesn’t appear. When I run your template scene, it does, but using the DK2 cameras and scripts it doesn’t. Thank you for trying to help!
I see. Maybe the silhouette is out of sight, due to wrongly calculated coordinates in the case of Oculus. I need to test this combination, before saying anything more definitively.
i bought kinect v2, and i work in unity.
the package you provide is the only one in the web that works so great : )
its incredibel, everything works fine, and i´m able to experiment in school projects.
i´m very happy with this amazing tools you created.
thank you very much.
just a little question:
microsoft provide a “green screen” package for unity.
how can i replace the “green” with an image?
they have the solution in visual studio project, but i dont know how to insert for exemple a land scape behind the player.
hope you can help some how.
thank you once again
Please contact me by e-mail and I’ll send you a similar demo, with picture on the back. It will be part of the next release.
Hello.
Thank you so much for the reply again.
No also your work for kinect is the best in the all internet, you respond to everyone.
You really make a diference. For example, microsoft never responds.
If you could please share the “demo, with picture on the back”, i would appreciate very much.
And counting the days to the next release !!!…
ricardosalazar@live.com.pt
or
salazar30segundos@gmail.com
Hi Rumen.
My computer is not working about the Asset Store so I cant get your kinect with MS-SDK
I really get that package.
So coule you send me include that file to e-mail?
My email is sky9081@gmail.com
Thanks.
Hi there, please send me an e-mail to prove this is really your e-mail address and I’ll send you a link to the package.
Hi, Im trying to use your asset to control my avatar and have a controller that controls the arms for IK to be used with the leap motion.
When I apply a controller to my avatar, the Kinect is unable to move the rest of the bones. it moves the root motion but that is it. I have tried using masks to no success. Any suggestions? Thanks
Hi, is there a way of applying animation to part of the avatar and have the Kinect control the rest?
Im having issues. when I add a controller it sets into a stress pose and the Kinect only controls the root movement.
Any suggestions?
Thanks
oops. didn’t know if the first comment worked
Try to use the AvatarControllerClassic instead of AvatarController and assign only the joints that need to be controlled by the Kinect sensor. Also, mask or remove these joints from the model’s rig-configuration, so they could not be controlled by the Animator.
Hi,
I’m completely new to the area so please bear with silly questions, if any:)
I’ve been searching a VR app using Kinect for controlling and it seems so far Unity is my best shot. However, it also looks Unity previously not ‘officially’ support Kinect SDK and there are various scripts wrappers such as Kinect1.7UnityWrapper.zip I’m sure you are aware of.
I’m using Kinect v1 & SDK 1.8, so not sure the above will work (wiki page says it updated to Kinect SDK 1.7 in Unity 4.2.)
Simply judging by the above comments, I think your scripts is best so far – thanks for that!
However I got compilation errors (see below) with your examples, coz I installed Unity 5.
Assets/KinectScripts/Samples/GetJointPositionDemo.cs(34,66): error CS0117: `System.IO.File’ does not contain a definition for `CreateText’
Assets/KinectScripts/Samples/GetJointPositionDemo.cs(68,90): error CS0117: `System.IO.File’ does not contain a definition for `AppendText’
Are they easy to fix, how? Will there be another update to Unity 5?
Many thanks,
If these are the only errors you get, just delete KinectScripts/Samples/GetJointPositionDemo.cs It is just a simple example script. But I’m also using Unity 5 and don’t get these kinds of errors. There is probably something else. Check, if you are not building your game for a limited player, like the Web-player, for instance.
Is there any way to add support for multiple Kinect sensors attached to one PC?
Currently not. The Kinect plugin provides access only to the default sensor. And the scripts are not thought for multi-sensor environment.
hi Rumen how r u? am arun. i am beginner to XBOX and unity 3D.I am using ur kinect plugin..lot of thanks for that. can you please tell me how to integrate GUI button with push and pull..plz reply me..My mail id is sarun_susu@yahoo.com
bye take care…once again thanks
s.Arun
Hi Arun, the easiest way would be to enable the ‘Control Mouse Cursor’-setting of KinectManager (component of MainCamera in the example scenes), and then use the Click-gesture for button presses, i.e. stay still over the button for about 2 seconds.
Thanks .Its Working But I want Like Particular GUIButton Push , Pull , Grip,Grip Released How to Do?..
To make grip/release work for GUI Button, enable ‘Control Mouse Cursor’ and ‘Control Mouse Drag’-settings of the InteractionManager-component.
hello im a beginner and want to ask is it possible to take or record a pose (or get a pose information like its coordinates, etc) so i can make it as a requirement to do something?
thanks in advance!
As far as I understand your question: There are some predefined poses (like T-pose, Psi, RaiseRightHand, etc. you can use as a calibration pose (used to start tracking the respective user). You need to set the ‘Player 1 Calibration Pose’-setting of KinectManager (component of MainCamera in the example scenes). Of course, you can add your own calibration poses in KinectGestures.cs and utilize them later in the same way.
thanks! any explanation how to add my own pose? how to write the code to match my pose? actually my project is a simple dance game using video so video pause at a certain pose and when user match the pose the video will resume and so on. sorry if its too much and thanks again 🙂
In this case, I would create separate scripts for saving and for checking the pose. First off, look at KinectScripts/Samples/GetJointPositionDemo.cs, to see how to get absolute joint positions of the tracked user. Then calculate the differences between joints of interest (elbow – shoulder, wrist – elbow, knee – hip, etc.) Then save these differences for the key-poses in a file and use them later in the script to check, if the current user’s pose is similar. Allow some tolerance along the way. Hope this suggestion helps a bit.
Hi, i tried the Playmaker package and already imported it also unzip it in the same folder but i got these errors:
http://puu.sh/hOhxC/ca28d7f9dd.png
How to fix it? Thanks
Update to the latest version of ‘Kinect with MS-SDK’-asset and try again. Please contact me by e-mail, if the problem persists.
Hi Rumen. I’m using this excelent asset to control an avatar in a theraphy game. I need to measure the angle between booth legs when one gets up, but when I raise one leg, the other moves in the horizontal plane. Is it possible to configure the avatar to the other leg stand still?. ( I still keep one fm my leg´s when I stand in front of the kinect )
Tks. (Please excuse my english)
Disable the ‘Use Bone Orientations Constraint’-setting of the KinectManager-component and try again.
Hi Rumen,
I am back after a year and working on a new project where I have multiple scenes. Basically opening scene does not have any Kinect integration and kinda gateway into other scenes. Each of the other scenes have got exact 1 character which is bound to Kinect user (as Avatar). User can navigate to any scene by going back to opening scene.
Now the issue that happens is when I am loading the scenes one after the other, the overall game slows down a lot. E.g. User’s movements (mapped to Kinect user) are delayed by a few seconds and lag is very clearly visible.
Each of the 4 sub-scenes have 1 character, a couple of image textures, main camera and directional light.
Whenever I load a sub-scene (either using Application.LoadLevel or LoadLevelAsync) CPU usage goes upto 30-40% for this application.
To debug, I left everything in the scene as it is, but removed the KinectManager from Main Camera in all the scenes and CPU spike went away and scenes started loading quickly without any lags.
I am not sure what can be done in-order to fix the issue or somehow use a single instance of KinectManager in whole of the application (all scenes) without loading KinectManager and initializing it in every scene.
Would be great if you can share your thoughts on this.
Regards,
Manpreet
There is a short manual, called ‘Howto-Use-KinectManager-Across-Multiple-Scenes.pdf’ in the Assets-folder. If you still don’t understand how to do it, contact me by e-mail to send you a multi-scene demo (compiled for Unity5).
Thanks for the quick reply Rumen. I went through the pdf and looks straight forward. I do not see Extras folder in KinectScripts. Just downloaded the package again to confirm. Can you please let me know from where to download the Extra scripts?
Sorry, the manual is rather old. I’ve renamed ‘Extras’ to ‘Samples’-folder some time ago. The needed scripts are there.
Ah! Found it 🙂
I’ve got a problem about Application.LoadLevel while using KinectOverlayDemo in sample file.
I make a time counter, when time is get 0, it will reload the scene. It’s work. BUT, when the scene reload, there are duplicated Main Camera and keep showing “There are 2 audio listeners in the scene. Please ensure there is always exactly one audio listener in the scene.”
Is anyone got this problem too?
The KinectManager-component prevents destroying its parent object, which is the MainCamera in this case. That’s why you get two MainCameras after the scene reloads. A workaround would be to destroy the camera explicitly: ‘GameObject.Destroy(Camera.main);’ just before reloading the level.
Hello Rumen,
I want to know, if this kinect asset works in a 2d project the same way it does in 3d? Or i have to change something?
Thank you.
Hi, open KinectScripts/KinectManager.cs and locate ProcessSkeleton()-function. Then: 1. after line ‘player1Pos = skeletonPos;’ add ‘player1Pos.z = 0f;’ 2. after ‘player1JointsPos[j] = …’ add ‘player1JointsPos[j].z = 0f;’. After that all Kinect positions and rotations will be 2D.
Hi Rumen! Thanks again for your good work!
How can I change the grey screen of the user map (when no user is detected), to show a transparent PNG instead?
I have the Kinect with MS-SDK v1.12
Thanks!
Best regards,
Cris.
Open the KinectScripts/KinectManager.cs, find ‘usersMapColors = new Color32[usersMapSize];’ and after that add ‘usersLblTex.SetPixels32(usersMapColors); ‘usersLblTex.Apply();’. I think this should clear the users’ texture.
OK Thanks Rumen for your help.
I made it, and it works!
In my game I’m changing (loading) frequently between 2 scenes. One scene has the KinectManager and the other not. I’ve seen that the KinectManager class has at the end of the Awake() method the “DontDestroyOnLoad(gameObject);”.
Is there any problem if I comment this line, so that the game load every time the KinectManager?
The idea of DontDestroyOnLoad() is to initialize the sensor only once, at the start of the game and stop it, when the game ends. See the MultiSceneDemo for example. If you comment out the DontDestroyOnLoad()-invocation, don’t forget to rename OnApplicationQuit()-function of KM to OnDestroy(), to match the sensor closing to sensor opening, within the same scene.
Thanks Rumen!!!
I renamed the OnApplicationQuit()-function of KinectManager and it works perfect!
As always, you are the best!!
Thanks for your quick answer and usefull help!!!
Best regards,
Cris.
Pingback: KinectExtras with MsSDK | RFilkov.com - Technology, Health and More