Tensorflow Pose Net on Construct 3

Hello, I want to make motion capture from construct 3 using user media (webcam) then get pose human every frame. I use tensorflow and pose net for the plugin (script) and already implement on script but i dont know how to get that video webcam every frame for process the poses.

This my current file

drive.google.com/file/d/1WPekQQy2B0E411cRYCxodUBemzjSF8Cv/view

this my script when i try process poses from user media

Can someone help ?

initialize pose net

Check out this template

As a quick hack test, try this (set project to not use worker mode):

Add a usermedia element to your layout.

In scripting use:

// Get the video element (assuming only 1 video element present)

globalThis.myV = C3UserMedia_GetVideoElement(0)

globalThis.myV should point to the usermedia object, which you use for your tensorflow pose.

If it works, request to Ashley to add an official scripting SDK for Usermedia to get access to the usermedia element to import to other APIs. I recommended only using official scripting/SDK API calls.

Mikal Cool, It works! Thank you.

Okay maybe i will request it to Ashley for add it to official scripting SDK.

But how to request for official SDK ? is there forum for request it?

You could create a new thread to request and also add a link to a request here:

construct3-21h2.ideas.aha.io

Hi! very interesting!

Exist a project for download for this?

Thanks

Sorry i cant give you this project because its already included my work projects.

Maybe later i will remake this project

You can check this github,

github.com/tensorflow/tfjs-models

Just make some script to include the tfjs script. I use posenet and handpose,

Use usermedia for camera then implement template camera average color to convert camera into image data. After you get image data, just process into tfjs to get poses then read it to JSON.