Integrates face-api.Js for face detection, landmarks.
An updated version added 1.2.1.0, works in preview, see the example for changes required (weights now included in project files.)
Hi Mikal, this updated version is not working in mobile device now at all. previous one was though slow but working.
Do you have any console logs, so we can see the error? Is this when you build an apk? Does it work if you try remote preview?
I simply exported html5 from construct 3 and hosted in aws. Same thing I did for previous version. Also, want to make a request, I respect your hardwork you put in this to make this feature possible. As of now 0.4 second delay is too much for giving realtime experience, apart from this there is an issue. Frame get repeated sometime, like it auto turns to gif for 3 or 4 seconds and after repeating it, it will be fine.
I am not so expert in construct 3 so wanted to ask following things:
1. Possible to track hand movement also in this ?
2. If i place sprite on any facepoint, for example a glasses sprite for eyes so when I move back and forth, how it will scale automatically to adjust its size with facepoint.
Can you point me to the aws link, so I can try it from my browser and see if I see errors?
Answer to your other questions: no it cannot track hand movements (the model is not trained to recognize a hand.) You can check distance between eyes (in pixels) to get an idea of how close or far the face is from the camera.
This algorithm is pretty GPU compute-intensive, so phones may not have enough of a GPU to have it work well (though I have tried on an iPhone XR and it works ok, but I mainly am using this on PC notebook and desktop for the game I am making.)
Here is the aws link.
vserv-images.s3.amazonaws.com/adsunit/faceapi/index.html
It is working fine on desktop but on mobile, face points not correctly syncing. I m using vivo nex with 8gb ram and latest gpu..so cant be processing power issue. Thanks for your support. Would love to see if you can guide me how Can i use to put glasses on eyes using this template.
Displaying on live video would be amazing and screenshot procedure could be removed from this (might be this is just my assumption) but I tried your suggested solution for live video and nothing worked. Might be something I did in a wrong way. Can you please share a c3 file for the same. It will be really helpful. Thanks in advance.
Try this for the dots(you can do something similar for the box), it scales the location based on the ratio of the display Sprite and the video input:
-> System: Create object Dot on layer 0 at (JSON.Get("._x")×Sprite.Width÷(UserMedia.VideoWidth)+Sprite.X, JSON.Get("._y")×Sprite.Height÷UserMedia.VideoHeight+Sprite.Y)
Another fun thing to do (but it will _lag_) is to display dots on the live video:
-> System: Create object Dot on layer 0 at (JSON.Get("._x")×UserMedia.Width÷(UserMedia.VideoWidth)+UserMedia.X, JSON.Get("._y")×UserMedia.Height÷UserMedia.VideoHeight+UserMedia.Y)
If you still see something is not right, play with the ratio / offset: ×Sprite.Width÷UserMedia.VideoWidth+Sprite.X (I have sometimes seen mobile with 2X the expected resolution.)
Added another example with face points on video. There will be lag between video and facepoints due to processing time of Tensor Flow.