Human: AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition, Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction, Gaze Tracking, Gesture Recognition
 
 
 
 
Go to file
Vladimir Mandic d340ada3fc 1.1.8 2021-03-17 11:40:35 -04:00
.github update 2021-03-15 08:52:28 -04:00
assets added blazepose-upper 2021-03-05 07:39:37 -05:00
demo add experimental nanodet object detection 2021-03-17 11:32:37 -04:00
dist update 2021-03-17 11:40:31 -04:00
models full models signature 2021-03-17 09:01:59 -04:00
server cleanup 2021-03-16 18:20:23 -04:00
src update 2021-03-17 11:40:31 -04:00
typedoc update 2021-03-17 11:40:31 -04:00
types update 2021-03-17 11:40:31 -04:00
wiki@5c012ed4cc add experimental nanodet object detection 2021-03-17 11:32:37 -04:00
.eslintrc.json add typedocs and types 2021-03-13 22:31:09 -05:00
.gitignore add test for face descriptors 2021-03-11 18:26:04 -05:00
.gitmodules updated wiki 2020-11-07 09:42:54 -05:00
.markdownlint.json update badges 2021-03-08 15:06:56 -05:00
CHANGELOG.md update 2021-03-17 11:40:31 -04:00
CODE_OF_CONDUCT update default github docs 2021-02-08 13:20:37 -05:00
CONTRIBUTING update default github docs 2021-02-08 13:20:37 -05:00
LICENSE Initial commit 2020-10-11 19:14:20 -04:00
README.md implement human.match and embedding demo 2021-03-15 12:14:48 -04:00
SECURITY.md Create SECURITY.md 2021-02-08 13:28:32 -05:00
TODO.md fix for seedrandom 2021-03-16 07:16:25 -04:00
config.ts add experimental nanodet object detection 2021-03-17 11:32:37 -04:00
favicon.ico refactored package file layout 2020-10-17 06:30:00 -04:00
human.service update tfjs to 3.1.0 2021-02-17 10:22:38 -05:00
manifest.webmanifest menu fixes 2021-02-08 13:07:49 -05:00
package-lock.json 1.1.8 2021-03-17 11:40:35 -04:00
package.json 1.1.8 2021-03-17 11:40:35 -04:00
tsconfig.json custom typedoc 2021-03-15 12:29:51 -04:00

README.md

Version Last Commit License GitHub Status Checks Vulnerabilities

Human Library

3D Face Detection & Rotation Tracking, Face Embedding & Recognition,
Body Pose Tracking, Hand & Finger Tracking,
Iris Analysis, Age & Gender & Emotion Prediction
& Gesture Recognition


JavaScript module using TensorFlow/JS Machine Learning library

  • Browser:
    Compatible with CPU, WebGL, WASM backends
    Compatible with both desktop and mobile platforms
    Compatible with WebWorker execution
  • NodeJS:
    Compatible with both software tfjs-node and GPU accelerated backends tfjs-node-gpu using CUDA libraries

Check out Live Demo for processing of live WebCam video or static images


Project pages


Wiki pages


Additional notes


Default models

Default models in Human library are:

  • Face Detection: MediaPipe BlazeFace-Back
  • Face Mesh: MediaPipe FaceMesh
  • Face Iris Analysis: MediaPipe Iris
  • Emotion Detection: Oarriaga Emotion
  • Gender Detection: Oarriaga Gender
  • Age Detection: SSR-Net Age IMDB
  • Body Analysis: PoseNet
  • Face Embedding: BecauseofAI MobileFace Embedding

Note that alternative models are provided and can be enabled via configuration
For example, PoseNet model can be switched for BlazePose model depending on the use case

For more info, see Configuration Details and List of Models


See issues and discussions for list of known limitations and planned enhancements

Suggestions are welcome!




Options

As presented in the demo application...

Options visible in demo




Examples


Training image:

Example Training Image

Using static images:

Example Using Image

Live WebCam view:

Example Using WebCam

468-Point Face Mesh Defails:

FaceMesh




Example simple app that uses Human to process video input and
draw output on screen using internal draw helper functions

import Human from '@vladmandic/human';

// create instance of human with simple configuration using default values
const config = { backend: 'webgl' };
const human = new Human(config);

function detectVideo() {
  // select input HTMLVideoElement and output HTMLCanvasElement from page
  const inputVideo = document.getElementById('video-id');
  const outputCanvas = document.getElementById('canvas-id');
  // perform processing using default configuration
  human.detect(inputVideo).then((result) => {
    // result object will contain detected details
    // as well as the processed canvas itself
    // so lets first draw processed frame on canvas
    human.draw.canvas(result.canvas, outputCanvas);
    // then draw results on the same canvas
    human.draw.face(outputCanvas, result.face);
    human.draw.body(outputCanvas, result.body);
    human.draw.hand(outputCanvas, result.hand);
    human.draw.gesture(outputCanvas, result.gesture);
    // loop immediate to next frame
    requestAnimationFrame(detectVideo);
  });
}

detectVideo();




Human library is written in TypeScript 4.3
Conforming to JavaScript ECMAScript version 2020 standard
Build target is JavaScript EMCAScript version 2018


For details see Wiki Pages
and API Specification


Downloads Stars Code Size