How do I capture a face from the usermedia webcam?

As a novelty, I would like to extract a face from a webcam, put it into a sprite, and pin it to the player's sprite as its head.

I have a User media object and sprite object, and a keyboard object.

At first I thought maybe I could use a green screen, then add an effect to replace the green colour from the image with a transparent alpha colour, but this is not working so well.

Then I figured, how about if I put an oval over the usermedia cam stream, and ask the player to position their head inside the oval? I could then use only the image data inside the oval in my sprite image. I have no idea how to tackle this so looking for any suggestions.

This is for students and high school kids studying at our college campus to play with and possibly implement in their game projects. Some of my previous students had a hoot using cropped side-on headshots of themselves on their platform characters. It would be so cool to integrate the capture process in their games.

All the best,

Al

OK, progress so far. Looking into the use of face-api.js to do this, with a chroma filter applied to green screen the background. my plan is to extract the face area, then make all the green bits transparent.

Work in progress, but getting there. I've hit a bit of an issue. This needs to run from a Linux machine.

I have Linux Mint 21 Xfce installed on an old PC, with a PS2 EyeToy set up as webcam. The webcam works well with other applications such as Cheese, but when I try to access it from my construct 3 game, I get nothing.

Permissions pops up, and I click the button to approve access, but nothing shows in the user media preview, and no pic is captured when I press a button.

This works on Windows 10/11 but on Linux I get no picture back with Chromium or Firefox.

Any advice welcome.

Event sheet is as follows:

Regards,

Al

Quick FYI for those interested. Uninstalling Cheese fixed my ps2 eyetoy webcam issue.

Ok, back at it. I would like to use TensorFlow Body-Pix to segment the webcam image.

I'm picking at a range of tutorials, including this: blog.tensorflow.org/2019/11/updated-bodypix-2.html

Within my main.js script before anything else happens I tried the following two lines, and both failed with the same result:

//import * as TensorFlow from 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.13.3';
import * as TensorFlow from './tf.js'; // local copy in script folder
/*
Uncaught TypeError: Cannot read properties of undefined (reading 'tf')
 at tf.js:2:149
 at tf.js:2:157
*/

If you have a green screen, you could very simply use the drawing canvas object to take a snapshot and loop through each pixel that is in a particular range of green to set the alpha to 0.