diff --git a/Backends.md b/Backends.md
index fd9299e..b82b345 100644
--- a/Backends.md
+++ b/Backends.md
@@ -1,10 +1,10 @@
-## Backends
+# Backends
Human library includes pre-bundled `CPU`, `WASM` and `WebGL` backends, there is no need to load them externally.
-### Notes:
+## Notes
- CPU backend usage is not recommended for performance reasons
as TensorFlow implementation in JavaScript is by far the slowest
@@ -16,23 +16,26 @@ Human library includes pre-bundled `CPU`, `WASM` and `WebGL` backends, there is
- Reduce size of input image (e.g. webcam)
as WASM does not have optimized image extraction methods
-#### How to enable WASM SIMD support
+### How to enable WASM SIMD support
Chrome:
+
- Navigate to
- `WebAssembly SIMD support` set to Enabled
Edge:
+
- Navigate to
- `WebAssembly SIMD support` set to Enabled
Firefox:
+
- Navigate to
- `javascript.options.wasm_simd` set to True
-### Known Issues:
+## Known Issues
- **WASM** backend without SIMD has limited processing capabilities before it causes issues
To avoid issues, start `human` with most models disabled and if needed, enable them one by one as each initializes
@@ -42,4 +45,3 @@ Firefox:
- **NodeJS** backend is missing key operations and thus neither `Face` nor `Hand` models are currently supported
-
diff --git a/Build-Process.md b/Build-Process.md
index a086ba3..3c5af89 100644
--- a/Build-Process.md
+++ b/Build-Process.md
@@ -1,4 +1,4 @@
-## Build Process
+# Build Process
If you want to modify the library and perform a full rebuild:
diff --git a/Change-Log.md b/Change-Log.md
index 8028b6b..2f8007a 100644
--- a/Change-Log.md
+++ b/Change-Log.md
@@ -1,7 +1,6 @@
-
# @vladmandic/human
-Version: **0.30.4**
+Version: **0.30.5**
Description: **Human: AI-powered 3D Face Detection, Face Embedding & Recognition, Body Pose Tracking, Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction & Gesture Recognition**
Author: **Vladimir Mandic **
@@ -10,209 +9,271 @@ Repository: ****
## Changelog
-### **HEAD -> main, tag: v0.30.4, origin/main, origin/HEAD** 2021/03/01 mandic00@live.com
+### **HEAD -> main** 2021/03/02 mandic00@live.com
+
+- add debug logging flag
+
+### **0.30.4** 2021/03/01 mandic00@live.com
+
- added skipinitial flag
-### **tag: v0.30.3** 2021/02/28 mandic00@live.com
+### **0.30.3** 2021/02/28 mandic00@live.com
+
- typo
-### **tag: v0.30.2** 2021/02/26 mandic00@live.com
+### **0.30.2** 2021/02/26 mandic00@live.com
+
- rebuild
- fix typo
-### **tag: v0.30.1** 2021/02/25 mandic00@live.com
+### **0.30.1** 2021/02/25 mandic00@live.com
-### **tag: v0.20.11** 2021/02/24 mandic00@live.com
-### **tag: v0.20.10** 2021/02/22 mandic00@live.com
+### **0.20.11** 2021/02/24 mandic00@live.com
+
+
+### **0.20.10** 2021/02/22 mandic00@live.com
+
+
+### **0.20.9** 2021/02/21 mandic00@live.com
-### **tag: v0.20.9** 2021/02/21 mandic00@live.com
- remove extra items
- simmilarity fix
-### **tag: v0.20.8** 2021/02/21 mandic00@live.com
+### **0.20.8** 2021/02/21 mandic00@live.com
+
- embedding fix
-### **tag: v0.20.7** 2021/02/21 mandic00@live.com
+### **0.20.7** 2021/02/21 mandic00@live.com
+
- build fix
-### **tag: v0.20.6** 2021/02/21 mandic00@live.com
+### **0.20.6** 2021/02/21 mandic00@live.com
+
- embedding fix
-### **tag: v0.20.5** 2021/02/21 mandic00@live.com
+### **0.20.5** 2021/02/21 mandic00@live.com
+
- fix imagefx and add dev builds
-### **tag: v0.20.4, tag: 0.3, tag: 0.2, tag: 0.1** 2021/02/19 mandic00@live.com
+### **0.20.4** 2021/02/19 mandic00@live.com
+
- 0.20.4
-### **tag: v0.20.3** 2021/02/17 mandic00@live.com
+### **0.20.3** 2021/02/17 mandic00@live.com
+
- rebuild
-### **tag: v0.20.2** 2021/02/13 mandic00@live.com
+### **0.20.2** 2021/02/13 mandic00@live.com
+
- merge branch 'main' of https://github.com/vladmandic/human into main
- create codeql-analysis.yml
- create security.md
- add templates
-### **tag: v0.20.1** 2021/02/08 mandic00@live.com
+### **0.20.1** 2021/02/08 mandic00@live.com
+
- menu fixes
- convert to typescript
-### **tag: v0.11.5** 2021/02/06 mandic00@live.com
+### **0.11.5** 2021/02/06 mandic00@live.com
+
- added faceboxes alternative model
-### **tag: v0.11.4** 2021/02/06 mandic00@live.com
+### **0.11.4** 2021/02/06 mandic00@live.com
-### **tag: v0.11.3** 2021/02/02 mandic00@live.com
-### **tag: v0.11.2** 2021/01/30 mandic00@live.com
+### **0.11.3** 2021/02/02 mandic00@live.com
+
+
+### **0.11.2** 2021/01/30 mandic00@live.com
+
- added warmup for nodejs
### **update for tfjs 3.0.0** 2021/01/29 mandic00@live.com
-### **tag: v0.11.1** 2021/01/29 mandic00@live.com
-### **tag: v0.10.2** 2021/01/22 mandic00@live.com
+### **0.11.1** 2021/01/29 mandic00@live.com
+
+
+### **0.10.2** 2021/01/22 mandic00@live.com
+
+
+### **0.10.1** 2021/01/20 mandic00@live.com
-### **tag: v0.10.1** 2021/01/20 mandic00@live.com
### **0.9.26** 2021/01/18 mandic00@live.com
+
- fix face detection when mesh is disabled
- added minification notes
- version bump
### **0.9.25** 2021/01/13 mandic00@live.com
+
- added humangl custom backend
- rebuild
- code cleanup and enable minification
- fix safari incopatibility
### **0.9.24** 2021/01/12 mandic00@live.com
+
- work on blazepose
- full rebuild
### **0.9.23** 2021/01/11 mandic00@live.com
+
- added iris gesture
- fix emotion labels
- full rebuild
### **0.9.22** 2021/01/05 mandic00@live.com
+
- remove iris coords if iris is disabled
- web worker fix
### **0.9.21** 2021/01/03 mandic00@live.com
+
### **0.9.20** 2021/01/03 mandic00@live.com
+
- stricter linting, fix face annotations
### **0.9.19** 2020/12/23 mandic00@live.com
+
- added rawbox and rawmesh
- variable name changes, setting .rawcoords only if necessary
- option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream
### **0.9.18** 2020/12/16 mandic00@live.com
+
- add z axis scaling
- major work on body module
- republish due to tfjs 2.8.0 issues
### **0.9.17** 2020/12/15 mandic00@live.com
+
- added custom webgl backend
### **0.9.16** 2020/12/12 mandic00@live.com
+
- change default ports
### **0.9.15** 2020/12/11 mandic00@live.com
+
- improved caching and warmup
- rebuild
### **0.9.14** 2020/12/10 mandic00@live.com
+
- conditional hand rotation
- staggered skipframes
- fix permissions
### **0.9.13** 2020/12/08 mandic00@live.com
+
- implemented face and hand boundary checks
- embedded sample for warmup
- switch to central logger
### **0.9.12** 2020/11/26 mandic00@live.com
+
- minor compatibility fixes
### **0.9.11** 2020/11/23 mandic00@live.com
+
- implement multi-person gestures
- modularize pipeline models
### **0.9.10** 2020/11/21 mandic00@live.com
+
- changed build for optimized node & browser
### **0.9.9** 2020/11/21 mandic00@live.com
+
- new screenshots
- camera exception handling
### **0.9.8** 2020/11/19 mandic00@live.com
+
- force f16 textures
- bugfix embedding check
### **0.9.7** 2020/11/19 mandic00@live.com
+
- ui redesign
### **0.9.6** 2020/11/18 mandic00@live.com
+
- optimize camera resize on mobile
- completed tfjs wrapper
### **0.9.5** 2020/11/17 mandic00@live.com
+
- fix serious performance bug around skipframes
- swtich to custom tfjs bundle
### **0.9.4** 2020/11/17 mandic00@live.com
+
- swtich to tfjs source import
### **0.9.3** 2020/11/16 mandic00@live.com
+
- switched to minified build
-### **tag: 1.2** 2020/11/15 mandic00@live.com
+### **release: 1.2** 2020/11/15 mandic00@live.com
+
- full rebuild
### **0.9.2** 2020/11/14 mandic00@live.com
+
- fix camera restart on resize
### **0.9.1** 2020/11/13 mandic00@live.com
+
- version bump
- full rebuild
- implemented face embedding
- added internal benchmark tool
### **0.8.8** 2020/11/12 mandic00@live.com
+
- reduced bundle size
- implemented buffered processing
- fix for conditional model loading
### **0.8.7** 2020/11/11 mandic00@live.com
+
- added performance notes
- added notes on models
- fix bug in async ops and change imports
- fix wiki links
### **0.8.6** 2020/11/09 mandic00@live.com
+
- add wasm bundle
### **0.8.5** 2020/11/09 mandic00@live.com
+
- reimplemented blazeface processing
### **0.8.4** 2020/11/09 mandic00@live.com
+
- added additional gestures
- implemented blink detection
- fix wasm module
### **0.8.3** 2020/11/08 mandic00@live.com
+
- refresh
- optimizations
### **0.8.2** 2020/11/08 mandic00@live.com
+
- fix typos
- commit
### **0.8.1** 2020/11/07 mandic00@live.com
+
- fix hand detection performance
- optimized model loader
- merge branch 'main' of https://github.com/vladmandic/human into main
@@ -222,52 +283,65 @@ Repository: ****
- fix nms sync call
### **0.7.6** 2020/11/06 mandic00@live.com
+
- fixed memory leaks and updated docs
- model tuning
- cache invalidation improvements
- full async operations
### **0.7.5** 2020/11/05 mandic00@live.com
+
- implemented dev-server
### **0.7.4** 2020/11/05 mandic00@live.com
+
- fix canvas size on different orientation
- switched from es2020 to es2018 build target
### **0.7.3** 2020/11/05 mandic00@live.com
+
- optimized camera and mobile layout
- fixed worker and filter compatibility
- removed test code
### **0.7.2** 2020/11/04 mandic00@live.com
+
- major work on handpose model
### **0.7.1** 2020/11/04 mandic00@live.com
+
- changed demo build process
### **0.6.7** 2020/11/04 mandic00@live.com
+
- implemented simple gesture recognition
### **0.6.6** 2020/11/04 mandic00@live.com
+
- remove debug code
### **0.6.5** 2020/11/04 mandic00@live.com
+
- redo hand detection
### **0.6.4** 2020/11/03 mandic00@live.com
+
- added manifest
### **0.6.3** 2020/11/03 mandic00@live.com
+
- enhanced processing resolution
- fix pause restart
- complete model refactoring
- fixed typo
### **0.6.2** 2020/11/02 mandic00@live.com
+
- optimized demo
- package update
### **0.6.1** 2020/11/02 mandic00@live.com
+
- major performance improvements for all models
- revert "optimized canvas handling"
- optimized canvas handling
@@ -276,61 +350,80 @@ Repository: ****
- added tfjs-vis to distribution
### **0.5.5** 2020/11/01 mandic00@live.com
+
- changed defaults
### **0.5.4** 2020/11/01 mandic00@live.com
+
- implemented memory profiler
### **0.5.3** 2020/10/30 mandic00@live.com
+
- improved debug logging
### **0.5.2** 2020/10/30 mandic00@live.com
+
- added wasm and webgpu backends
### **0.5.1** 2020/10/30 mandic00@live.com
+
- improve demo line continous draws
### **0.4.10** 2020/10/30 mandic00@live.com
+
- fix for seedrandom
### **0.4.9** 2020/10/29 mandic00@live.com
+
### **0.4.8** 2020/10/28 mandic00@live.com
+
- revert "updated menu handler"
- fix webpack compatibility issue
### **0.4.7** 2020/10/27 mandic00@live.com
+
### **0.4.6** 2020/10/27 mandic00@live.com
+
- fix firefox compatibility bug
### **0.4.5** 2020/10/27 mandic00@live.com
+
### **0.4.4** 2020/10/27 mandic00@live.com
+
- implelented input resizing
### **0.4.3** 2020/10/22 mandic00@live.com
+
### **0.4.2** 2020/10/20 mandic00@live.com
+
- log initialization
### **0.4.1** 2020/10/19 mandic00@live.com
+
- breaking change: convert to object class
- compatibility notes
### **0.3.9** 2020/10/18 mandic00@live.com
+
- implemented image filters
- pure tensor pipeline without image converts
- autodetect skipframes
### **0.3.8** 2020/10/17 mandic00@live.com
+
- new menu layout
### **0.3.7** 2020/10/17 mandic00@live.com
+
- added diagnostics output
- parallelized agegender operations
### **0.3.6** 2020/10/17 mandic00@live.com
+
- fixed webcam initialization
- fixed memory leaks and added scoped runs
- modularized draw
@@ -338,17 +431,21 @@ Repository: ****
- refactored package file layout
### **0.3.5** 2020/10/16 mandic00@live.com
+
- added auto-generated changelog
### **0.3.4** 2020/10/16 mandic00@live.com
+
- added camera selection
- optimized blazeface anchors
- added error handling
### **0.3.3** 2020/10/15 mandic00@live.com
+
- added blazeface back and front models
### **0.3.2** 2020/10/15 mandic00@live.com
+
- reduced web worker latency
- added debugging and versioning
- optimized demos and added scoped runs
@@ -356,37 +453,49 @@ Repository: ****
### **0.3.1** 2020/10/14 mandic00@live.com
+
### **0.2.10** 2020/10/14 mandic00@live.com
+
- added emotion backend
- module parametrization and performance monitoring
- implemented multi-hand support
- fixed documentation typos
### **0.2.9** 2020/10/13 mandic00@live.com
+
- added node build and demo
### **0.2.8** 2020/10/13 mandic00@live.com
+
- added example image
### **0.2.7** 2020/10/13 mandic00@live.com
+
- new examples
### **0.2.6** 2020/10/13 mandic00@live.com
+
- enable all models by default
### **0.2.5** 2020/10/12 mandic00@live.com
+
- fixed memory leak
### **0.2.4** 2020/10/12 mandic00@live.com
+
- removed extra files
### **0.2.3** 2020/10/12 mandic00@live.com
+
### **0.2.2** 2020/10/12 mandic00@live.com
-### **tag: 1.0** 2020/10/12 mandic00@live.com
+
+### **release: 1.0** 2020/10/12 mandic00@live.com
+
### **0.2.1** 2020/10/12 mandic00@live.com
+
- added sample image
- initial public commit
- initial commit
diff --git a/Configuration.md b/Configuration.md
index 4f54a15..27eeb1c 100644
--- a/Configuration.md
+++ b/Configuration.md
@@ -1,4 +1,4 @@
-## Configuration
+# Configuration
Detailed configuration options are explained below, but they are best seen in the menus present in the `demo` application:
*note: some advanced configuration options are not exposed in the UI*
@@ -10,12 +10,14 @@ Detailed configuration options are explained below, but they are best seen in th
Main configuration objects are:
+
- **config.filter**: controls image pre-processing
- **config.face**: controls face detection
- **config.body**: controls body pose detection
- **config.hand**: contros hand and finger detection
With **config.face** having several subsections:
+
- **config.face.mesh**: controls facial mesh and landscape detection
- **config.face.iris**: controls iris detection
- **config.face.age**: controls age prediction
@@ -31,7 +33,6 @@ Note that user object and default configuration are merged using deep-merge, so
All configuration details can be changed in real-time!
-
```js
config = {
backend: 'webgl', // select tfjs backend to use
diff --git a/Credits.md b/Credits.md
index 0483d12..d29e503 100644
--- a/Credits.md
+++ b/Credits.md
@@ -1,4 +1,4 @@
-## Credits
+# Credits
- Face Detection: [**MediaPipe BlazeFace**](https://drive.google.com/file/d/1f39lSzU5Oq-j_OXgS67KfN5wNsoeAZ4V/view)
- Face Boxes: [**FaceBoxes**](https://github.com/TropComplique/FaceBoxes-tensorflow/)
diff --git a/Demos.md b/Demos.md
index 7e42acc..0727d15 100644
--- a/Demos.md
+++ b/Demos.md
@@ -1,10 +1,11 @@
-## Demos
+# Demos
Demos are included in `/demo`:
**Browser**:
+
- `index.html`: Full demo using Browser with ESM module, includes selectable backends and webworkers
*You can run browser demo either live from git pages, by serving demo folder from your web server or use
@@ -12,7 +13,7 @@ included micro http2 server with source file monitoring and dynamic rebuild*
-### Changing Demo Target
+## Changing Demo Target
Demo in `demo/index.html` loads `dist/demo-browser-index.js` which is built from sources in `demo`, starting with `demo/browser`
This bundled version is needed since mobile browsers (e.g. Chrome on Android) do not support native modules loading yet
@@ -21,10 +22,10 @@ If your target is desktop, alternatively you can load `demo/browser.js` directly
-### Demo Options
+## Demo Options
Demo implements several ways to use `Human` library,
-all configurable in `browse.js:ui` configuration object and in the UI itself:
+all configurable in `browse.js:ui` configuration object and in the UI itself:
- `ui.buffered`: run detection and screen refresh in a sequence or as separate buffered functions
- `ui.bufferedFPSTarget`: when using buffered execution this target fps for screen refresh
@@ -41,18 +42,20 @@ all configurable in `browse.js:ui` configuration object and in the UI itself:
-### Dev Server
+## Dev Server
To use dev server, you must have an SSL certificate since browsers enforce *https* for webcam access
You can use a provided ones self-signed certificate or configure your own
Self-signed test certificate was generated using:
+
```shell
openssl req -x509 -newkey rsa:4096 -nodes -keyout https.key -out https.crt -days 365 -subj "/C=US/ST=Florida/L=Miami/O=@vladmandic"
```
If you want to use your own certificate, edit `server/dev.js`:
+
```js
const options = {
key: fs.readFileSync('server/https.key'),
@@ -61,9 +64,11 @@ const options = {
```
Once SSL certificates have been provided (defalt test certificate or your own), simply run
+
```shell
npm run dev
```
+
On first start, it will install all development dependencies required to rebuild `Human` library
Dev version of the library runs without code minification for easier debugging while production version is fully minified
@@ -113,9 +118,11 @@ found 0 vulnerabilities
2020-12-12 10:12:55 DATA: GET/2.0 200 application/octet-stream 5502280 /models/handskeleton.bin ::ffff:192.168.0.200
...
```
+
**NodeJS**:
+
- `node.js`: Demo using NodeJS with CommonJS module
This is a very simple demo as althought `Human` library is compatible with NodeJS execution
and is able to load images and models from local filesystem,
diff --git a/Embedding.md b/Embedding.md
index a66423d..8ec9c01 100644
--- a/Embedding.md
+++ b/Embedding.md
@@ -1,4 +1,4 @@
-## Face Feature Embedding and Simmilarity Compare
+# Face Feature Embedding and Simmilarity Compare
diff --git a/Gesture.md b/Gesture.md
index d755c20..c51f9f6 100644
--- a/Gesture.md
+++ b/Gesture.md
@@ -1,4 +1,4 @@
-## Gesture Recognition
+# Gesture Recognition
Gesture recognition is done by looking up coordinates of different detected landmarks
@@ -21,6 +21,7 @@ There are three pre-defined methods:
- "*`` forward `` up*"
Example output of `result.gesture`:
+
```js
[
{face: "0", gesture: "facing camera"}
@@ -31,4 +32,5 @@ Example output of `result.gesture`:
{hand: "0", gesture: "thumb forward middlefinger up"}
]
```
+
Where number after gesture refers to number of person that detection belongs to in scenes with multiple people.
diff --git a/Home.md b/Home.md
index c779a70..299d294 100644
--- a/Home.md
+++ b/Home.md
@@ -4,7 +4,7 @@
-### Project pages:
+### Project pages
- [**Live Demo**](https://vladmandic.github.io/human/demo/index.html)
- [**Code Repository**](https://github.com/vladmandic/human)
@@ -12,7 +12,7 @@
- [**Issues Tracker**](https://github.com/vladmandic/human/issues)
- [**Change Log**](https://github.com/vladmandic/human/wiki/Change-Log)
-### Wiki pages:
+### Wiki pages
- [**Home**](https://github.com/vladmandic/human/wiki)
- [**Demos**](https://github.com/vladmandic/human/wiki/Demos)
@@ -23,7 +23,7 @@
- [**Face Embedding and Recognition**](https://github.com/vladmandic/human/wiki/Embedding)
- [**Gesture Recognition**](https://github.com/vladmandic/human/wiki/Gesture)
-### Additional notes:
+### Additional notes
- [**Notes on Backends**](https://github.com/vladmandic/human/wiki/Backends)
- [**Development Server**](https://github.com/vladmandic/human/wiki/Development-Server)
@@ -35,7 +35,8 @@
-Compatible with *Browser*, *WebWorker* and *NodeJS* execution on both Windows and Linux
+Compatible with *Browser*, *WebWorker* and *NodeJS* execution on both Windows and Linux
+
- Browser/WebWorker: Compatible with *CPU*, *WebGL*, *WASM* and *WebGPU* backends
- NodeJS: Compatible with software *tfjs-node* and CUDA accelerated backends *tfjs-node-gpu*
- (and maybe with React-Native as it doesn't use any DOM objects)
diff --git a/Install.md b/Install.md
index e676592..ba77fba 100644
--- a/Install.md
+++ b/Install.md
@@ -1,19 +1,19 @@
-## Installation
+# Installation
-**Important**
+**Important**
*The packaged version of `Human` includes `TensorFlow/JS (TFJS) 3.2.0` library which can be accessed via `human.tf`*
-*You should NOT manually load another instance of `tfjs` unless you're using specific `.nobudle` edition of `Human,
+*You should NOT manually load another instance of `tfjs` unless you're using specific `.nobudle` edition of `Human,
but if you do, be aware of possible version conflicts*
There are multiple ways to use `Human` library, pick one that suits you:
-### Included
+## Included
- `dist/human.js`: IIFE format bundle with TFJS for Browsers
- `dist/human.esm.js`: ESM format bundle with TFJS for Browsers
@@ -25,6 +25,7 @@ All versions include `sourcemap` *(.map)* and build `manifest` *(.json)*
While `Human` is in pre-release mode, all bundles are non-minified
Defaults:
+
```json
{
"main": "dist/human.node.js",
@@ -35,7 +36,7 @@ Defaults:
-### 1. [IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE) script
+## 1. [IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE) script
*Simplest way for usage within Browser*
@@ -43,10 +44,10 @@ Simply download `dist/human.js`, include it in your `HTML` file & it's ready to
```html