Human: AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition, Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction, Gaze Tracking, Gesture Recognition
 
 
 
 
Go to file
Vladimir Mandic eb6c33b7b6 update security policy 2021-05-30 09:41:24 -04:00
.github update github templates 2021-03-18 11:58:46 -04:00
assets fix node build and update model signatures 2021-05-11 07:53:06 -04:00
demo quantize handdetect model 2021-05-29 18:29:57 -04:00
dist quantize handdetect model 2021-05-29 18:29:57 -04:00
models quantize handdetect model 2021-05-29 18:29:57 -04:00
server quantize handdetect model 2021-05-29 18:29:57 -04:00
src quantize handdetect model 2021-05-29 18:29:57 -04:00
test added experimental movenet-lightning and removed blazepose from default dist 2021-05-29 09:20:01 -04:00
typedoc quantize handdetect model 2021-05-29 18:29:57 -04:00
types quantize handdetect model 2021-05-29 18:29:57 -04:00
wiki@317a8fc76c added experimental movenet-lightning and removed blazepose from default dist 2021-05-29 09:20:01 -04:00
.eslintrc.json add tfjs types and remove all instances of any 2021-05-22 21:47:59 -04:00
.gitignore fix box clamping and raw output 2021-03-17 14:35:11 -04:00
.gitmodules updated wiki 2020-11-07 09:42:54 -05:00
.hintrc implement webhint 2021-04-04 09:25:18 -04:00
.markdownlint.json update badges 2021-03-08 15:06:56 -05:00
.npmignore fix linting and tests 2021-04-03 10:49:14 -04:00
.npmrc add npmrc 2021-04-20 08:02:21 -04:00
CHANGELOG.md quantize handdetect model 2021-05-29 18:29:57 -04:00
CODE_OF_CONDUCT update github templates 2021-03-18 11:58:46 -04:00
CONTRIBUTING added node-multiprocess demo 2021-04-16 08:34:16 -04:00
LICENSE update github templates 2021-03-18 11:58:46 -04:00
README.md update security policy 2021-05-30 09:41:24 -04:00
SECURITY.md update security policy 2021-05-30 09:41:24 -04:00
TODO.md update security policy 2021-05-30 09:41:24 -04:00
favicon.ico new icons 2021-03-29 15:01:16 -04:00
human.service fix box clamping and raw output 2021-03-17 14:35:11 -04:00
package.json 1.9.4 2021-05-27 16:05:20 -04:00
tsconfig.json fix typedoc 2021-04-09 21:53:48 -04:00

README.md

Git Version NPM Version Last Commit License GitHub Status Checks Vulnerabilities

Human Library

AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition,
Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis,
Age & Gender & Emotion Prediction, Gesture Recognition


JavaScript module using TensorFlow/JS Machine Learning library

  • Browser:
    Compatible with both desktop and mobile platforms
    Compatible with CPU, WebGL, WASM backends
    Compatible with WebWorker execution
  • NodeJS:
    Compatible with both software tfjs-node and
    GPU accelerated backends tfjs-node-gpu using CUDA libraries

Check out Live Demo for processing of live WebCam video or static images


Demos

Project pages

Wiki pages

Additional notes


See issues and discussions for list of known limitations and planned enhancements

Suggestions are welcome!



Options

As presented in the demo application...

demo/index.html

Options visible in demo


Examples


Training image:

demo/index.html

Example Training Image

Using static images:

demo/index.html

Example Using Image

Live WebCam view:

demo/index.html

Example Using WebCam

Face Similarity Matching:

demo/facematch.html

Face Matching

Face3D OpenGL Rendering:

demo/face3d.html

Face Matching

468-Point Face Mesh Defails:
(view in full resolution to see keypoints)

FaceMesh




Quick Start

Simply load Human (IIFE version) directly from a cloud CDN in your HTML file:
(pick one: jsdelirv, unpkg or cdnjs)

<script src="https://cdn.jsdelivr.net/npm/@vladmandic/human/dist/human.js"></script>
<script src="https://unpkg.dev/@vladmandic/human/dist/human.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/human/1.4.1/human.js"></script>

For details, including how to use Browser ESM version or NodeJS version of Human, see Installation


Inputs

Human library can process all known input types:

  • Image, ImageData, ImageBitmap, Canvas, OffscreenCanvas, Tensor,
  • HTMLImageElement, HTMLCanvasElement, HTMLVideoElement, HTMLMediaElement

Additionally, HTMLVideoElement, HTMLMediaElement can be a standard <video> tag that links to:

  • WebCam on user's system
  • Any supported video type
    For example: .mp4, .avi, etc.
  • Additional video types supported via HTML5 Media Source Extensions
    Live streaming examples:
    • HLS (HTTP Live Streaming) using hls.js
    • DASH (Dynamic Adaptive Streaming over HTTP) using dash.js
  • WebRTC media track

Example

Example simple app that uses Human to process video input and
draw output on screen using internal draw helper functions

// create instance of human with simple configuration using default values
const config = { backend: 'webgl' };
const human = new Human(config);

function detectVideo() {
  // select input HTMLVideoElement and output HTMLCanvasElement from page
  const inputVideo = document.getElementById('video-id');
  const outputCanvas = document.getElementById('canvas-id');
  // perform processing using default configuration
  human.detect(inputVideo).then((result) => {
    // result object will contain detected details
    // as well as the processed canvas itself
    // so lets first draw processed frame on canvas
    human.draw.canvas(result.canvas, outputCanvas);
    // then draw results on the same canvas
    human.draw.face(outputCanvas, result.face);
    human.draw.body(outputCanvas, result.body);
    human.draw.hand(outputCanvas, result.hand);
    human.draw.gesture(outputCanvas, result.gesture);
    // and loop immediate to the next frame
    requestAnimationFrame(detectVideo);
  });
}

detectVideo();

or using async/await:

// create instance of human with simple configuration using default values
const config = { backend: 'webgl' };
const human = new Human(config);

async function detectVideo() {
  const inputVideo = document.getElementById('video-id');
  const outputCanvas = document.getElementById('canvas-id');
  const result = await human.detect(inputVideo);
  human.draw.all(outputCanvas, result);
  requestAnimationFrame(detectVideo);
}

detectVideo();




Default models

Default models in Human library are:

  • Face Detection: MediaPipe BlazeFace (Back version)
  • Face Mesh: MediaPipe FaceMesh
  • Face Description: HSE FaceRes
  • Face Iris Analysis: MediaPipe Iris
  • Emotion Detection: Oarriaga Emotion
  • Body Analysis: PoseNet (AtomicBits version)

Note that alternative models are provided and can be enabled via configuration
For example, PoseNet model can be switched for BlazePose, EfficientPose or MoveNet model depending on the use case

For more info, see Configuration Details and List of Models




Human library is written in TypeScript 4.2
Conforming to JavaScript ECMAScript version 2020 standard
Build target is JavaScript EMCAScript version 2018


For details see Wiki Pages
and API Specification


Stars Forks Code Size CDN
Downloads Downloads Downloads