update with markdownlint

master
Vladimir Mandic 2021-03-03 09:59:13 -05:00
parent 4091054e6d
commit 7bb1673bd2
17 changed files with 214 additions and 78 deletions

@ -1,10 +1,10 @@
## Backends # Backends
Human library includes pre-bundled `CPU`, `WASM` and `WebGL` backends, there is no need to load them externally. Human library includes pre-bundled `CPU`, `WASM` and `WebGL` backends, there is no need to load them externally.
<br> <br>
### Notes: ## Notes
- CPU backend usage is not recommended for performance reasons - CPU backend usage is not recommended for performance reasons
as TensorFlow implementation in JavaScript is by far the slowest as TensorFlow implementation in JavaScript is by far the slowest
@ -16,23 +16,26 @@ Human library includes pre-bundled `CPU`, `WASM` and `WebGL` backends, there is
- Reduce size of input image (e.g. webcam) - Reduce size of input image (e.g. webcam)
as WASM does not have optimized image extraction methods as WASM does not have optimized image extraction methods
#### How to enable WASM SIMD support ### How to enable WASM SIMD support
Chrome: Chrome:
- Navigate to <chrome://flags> - Navigate to <chrome://flags>
- `WebAssembly SIMD support` set to Enabled - `WebAssembly SIMD support` set to Enabled
Edge: Edge:
- Navigate to <chrome://flags> - Navigate to <chrome://flags>
- `WebAssembly SIMD support` set to Enabled - `WebAssembly SIMD support` set to Enabled
Firefox: Firefox:
- Navigate to <about:config> - Navigate to <about:config>
- `javascript.options.wasm_simd` set to True - `javascript.options.wasm_simd` set to True
<br> <br>
### Known Issues: ## Known Issues
- **WASM** backend without SIMD has limited processing capabilities before it causes issues - **WASM** backend without SIMD has limited processing capabilities before it causes issues
To avoid issues, start `human` with most models disabled and if needed, enable them one by one as each initializes To avoid issues, start `human` with most models disabled and if needed, enable them one by one as each initializes
@ -42,4 +45,3 @@ Firefox:
- **NodeJS** backend is missing key operations and thus neither `Face` nor `Hand` models are currently supported - **NodeJS** backend is missing key operations and thus neither `Face` nor `Hand` models are currently supported
<br> <br>

@ -1,4 +1,4 @@
## Build Process # Build Process
If you want to modify the library and perform a full rebuild: If you want to modify the library and perform a full rebuild:

@ -1,7 +1,6 @@
# @vladmandic/human # @vladmandic/human
Version: **0.30.4** Version: **0.30.5**
Description: **Human: AI-powered 3D Face Detection, Face Embedding & Recognition, Body Pose Tracking, Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction & Gesture Recognition** Description: **Human: AI-powered 3D Face Detection, Face Embedding & Recognition, Body Pose Tracking, Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction & Gesture Recognition**
Author: **Vladimir Mandic <mandic00@live.com>** Author: **Vladimir Mandic <mandic00@live.com>**
@ -10,209 +9,271 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
## Changelog ## Changelog
### **HEAD -> main, tag: v0.30.4, origin/main, origin/HEAD** 2021/03/01 mandic00@live.com ### **HEAD -> main** 2021/03/02 mandic00@live.com
- add debug logging flag
### **0.30.4** 2021/03/01 mandic00@live.com
- added skipinitial flag - added skipinitial flag
### **tag: v0.30.3** 2021/02/28 mandic00@live.com ### **0.30.3** 2021/02/28 mandic00@live.com
- typo - typo
### **tag: v0.30.2** 2021/02/26 mandic00@live.com ### **0.30.2** 2021/02/26 mandic00@live.com
- rebuild - rebuild
- fix typo - fix typo
### **tag: v0.30.1** 2021/02/25 mandic00@live.com ### **0.30.1** 2021/02/25 mandic00@live.com
### **tag: v0.20.11** 2021/02/24 mandic00@live.com
### **tag: v0.20.10** 2021/02/22 mandic00@live.com ### **0.20.11** 2021/02/24 mandic00@live.com
### **0.20.10** 2021/02/22 mandic00@live.com
### **0.20.9** 2021/02/21 mandic00@live.com
### **tag: v0.20.9** 2021/02/21 mandic00@live.com
- remove extra items - remove extra items
- simmilarity fix - simmilarity fix
### **tag: v0.20.8** 2021/02/21 mandic00@live.com ### **0.20.8** 2021/02/21 mandic00@live.com
- embedding fix - embedding fix
### **tag: v0.20.7** 2021/02/21 mandic00@live.com ### **0.20.7** 2021/02/21 mandic00@live.com
- build fix - build fix
### **tag: v0.20.6** 2021/02/21 mandic00@live.com ### **0.20.6** 2021/02/21 mandic00@live.com
- embedding fix - embedding fix
### **tag: v0.20.5** 2021/02/21 mandic00@live.com ### **0.20.5** 2021/02/21 mandic00@live.com
- fix imagefx and add dev builds - fix imagefx and add dev builds
### **tag: v0.20.4, tag: 0.3, tag: 0.2, tag: 0.1** 2021/02/19 mandic00@live.com ### **0.20.4** 2021/02/19 mandic00@live.com
- 0.20.4 - 0.20.4
### **tag: v0.20.3** 2021/02/17 mandic00@live.com ### **0.20.3** 2021/02/17 mandic00@live.com
- rebuild - rebuild
### **tag: v0.20.2** 2021/02/13 mandic00@live.com ### **0.20.2** 2021/02/13 mandic00@live.com
- merge branch 'main' of https://github.com/vladmandic/human into main - merge branch 'main' of https://github.com/vladmandic/human into main
- create codeql-analysis.yml - create codeql-analysis.yml
- create security.md - create security.md
- add templates - add templates
### **tag: v0.20.1** 2021/02/08 mandic00@live.com ### **0.20.1** 2021/02/08 mandic00@live.com
- menu fixes - menu fixes
- convert to typescript - convert to typescript
### **tag: v0.11.5** 2021/02/06 mandic00@live.com ### **0.11.5** 2021/02/06 mandic00@live.com
- added faceboxes alternative model - added faceboxes alternative model
### **tag: v0.11.4** 2021/02/06 mandic00@live.com ### **0.11.4** 2021/02/06 mandic00@live.com
### **tag: v0.11.3** 2021/02/02 mandic00@live.com
### **tag: v0.11.2** 2021/01/30 mandic00@live.com ### **0.11.3** 2021/02/02 mandic00@live.com
### **0.11.2** 2021/01/30 mandic00@live.com
- added warmup for nodejs - added warmup for nodejs
### **update for tfjs 3.0.0** 2021/01/29 mandic00@live.com ### **update for tfjs 3.0.0** 2021/01/29 mandic00@live.com
### **tag: v0.11.1** 2021/01/29 mandic00@live.com
### **tag: v0.10.2** 2021/01/22 mandic00@live.com ### **0.11.1** 2021/01/29 mandic00@live.com
### **0.10.2** 2021/01/22 mandic00@live.com
### **0.10.1** 2021/01/20 mandic00@live.com
### **tag: v0.10.1** 2021/01/20 mandic00@live.com
### **0.9.26** 2021/01/18 mandic00@live.com ### **0.9.26** 2021/01/18 mandic00@live.com
- fix face detection when mesh is disabled - fix face detection when mesh is disabled
- added minification notes - added minification notes
- version bump - version bump
### **0.9.25** 2021/01/13 mandic00@live.com ### **0.9.25** 2021/01/13 mandic00@live.com
- added humangl custom backend - added humangl custom backend
- rebuild - rebuild
- code cleanup and enable minification - code cleanup and enable minification
- fix safari incopatibility - fix safari incopatibility
### **0.9.24** 2021/01/12 mandic00@live.com ### **0.9.24** 2021/01/12 mandic00@live.com
- work on blazepose - work on blazepose
- full rebuild - full rebuild
### **0.9.23** 2021/01/11 mandic00@live.com ### **0.9.23** 2021/01/11 mandic00@live.com
- added iris gesture - added iris gesture
- fix emotion labels - fix emotion labels
- full rebuild - full rebuild
### **0.9.22** 2021/01/05 mandic00@live.com ### **0.9.22** 2021/01/05 mandic00@live.com
- remove iris coords if iris is disabled - remove iris coords if iris is disabled
- web worker fix - web worker fix
### **0.9.21** 2021/01/03 mandic00@live.com ### **0.9.21** 2021/01/03 mandic00@live.com
### **0.9.20** 2021/01/03 mandic00@live.com ### **0.9.20** 2021/01/03 mandic00@live.com
- stricter linting, fix face annotations - stricter linting, fix face annotations
### **0.9.19** 2020/12/23 mandic00@live.com ### **0.9.19** 2020/12/23 mandic00@live.com
- added rawbox and rawmesh - added rawbox and rawmesh
- variable name changes, setting .rawcoords only if necessary - variable name changes, setting .rawcoords only if necessary
- option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream - option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream
### **0.9.18** 2020/12/16 mandic00@live.com ### **0.9.18** 2020/12/16 mandic00@live.com
- add z axis scaling - add z axis scaling
- major work on body module - major work on body module
- republish due to tfjs 2.8.0 issues - republish due to tfjs 2.8.0 issues
### **0.9.17** 2020/12/15 mandic00@live.com ### **0.9.17** 2020/12/15 mandic00@live.com
- added custom webgl backend - added custom webgl backend
### **0.9.16** 2020/12/12 mandic00@live.com ### **0.9.16** 2020/12/12 mandic00@live.com
- change default ports - change default ports
### **0.9.15** 2020/12/11 mandic00@live.com ### **0.9.15** 2020/12/11 mandic00@live.com
- improved caching and warmup - improved caching and warmup
- rebuild - rebuild
### **0.9.14** 2020/12/10 mandic00@live.com ### **0.9.14** 2020/12/10 mandic00@live.com
- conditional hand rotation - conditional hand rotation
- staggered skipframes - staggered skipframes
- fix permissions - fix permissions
### **0.9.13** 2020/12/08 mandic00@live.com ### **0.9.13** 2020/12/08 mandic00@live.com
- implemented face and hand boundary checks - implemented face and hand boundary checks
- embedded sample for warmup - embedded sample for warmup
- switch to central logger - switch to central logger
### **0.9.12** 2020/11/26 mandic00@live.com ### **0.9.12** 2020/11/26 mandic00@live.com
- minor compatibility fixes - minor compatibility fixes
### **0.9.11** 2020/11/23 mandic00@live.com ### **0.9.11** 2020/11/23 mandic00@live.com
- implement multi-person gestures - implement multi-person gestures
- modularize pipeline models - modularize pipeline models
### **0.9.10** 2020/11/21 mandic00@live.com ### **0.9.10** 2020/11/21 mandic00@live.com
- changed build for optimized node & browser - changed build for optimized node & browser
### **0.9.9** 2020/11/21 mandic00@live.com ### **0.9.9** 2020/11/21 mandic00@live.com
- new screenshots - new screenshots
- camera exception handling - camera exception handling
### **0.9.8** 2020/11/19 mandic00@live.com ### **0.9.8** 2020/11/19 mandic00@live.com
- force f16 textures - force f16 textures
- bugfix embedding check - bugfix embedding check
### **0.9.7** 2020/11/19 mandic00@live.com ### **0.9.7** 2020/11/19 mandic00@live.com
- ui redesign - ui redesign
### **0.9.6** 2020/11/18 mandic00@live.com ### **0.9.6** 2020/11/18 mandic00@live.com
- optimize camera resize on mobile - optimize camera resize on mobile
- completed tfjs wrapper - completed tfjs wrapper
### **0.9.5** 2020/11/17 mandic00@live.com ### **0.9.5** 2020/11/17 mandic00@live.com
- fix serious performance bug around skipframes - fix serious performance bug around skipframes
- swtich to custom tfjs bundle - swtich to custom tfjs bundle
### **0.9.4** 2020/11/17 mandic00@live.com ### **0.9.4** 2020/11/17 mandic00@live.com
- swtich to tfjs source import - swtich to tfjs source import
### **0.9.3** 2020/11/16 mandic00@live.com ### **0.9.3** 2020/11/16 mandic00@live.com
- switched to minified build - switched to minified build
### **tag: 1.2** 2020/11/15 mandic00@live.com ### **release: 1.2** 2020/11/15 mandic00@live.com
- full rebuild - full rebuild
### **0.9.2** 2020/11/14 mandic00@live.com ### **0.9.2** 2020/11/14 mandic00@live.com
- fix camera restart on resize - fix camera restart on resize
### **0.9.1** 2020/11/13 mandic00@live.com ### **0.9.1** 2020/11/13 mandic00@live.com
- version bump - version bump
- full rebuild - full rebuild
- implemented face embedding - implemented face embedding
- added internal benchmark tool - added internal benchmark tool
### **0.8.8** 2020/11/12 mandic00@live.com ### **0.8.8** 2020/11/12 mandic00@live.com
- reduced bundle size - reduced bundle size
- implemented buffered processing - implemented buffered processing
- fix for conditional model loading - fix for conditional model loading
### **0.8.7** 2020/11/11 mandic00@live.com ### **0.8.7** 2020/11/11 mandic00@live.com
- added performance notes - added performance notes
- added notes on models - added notes on models
- fix bug in async ops and change imports - fix bug in async ops and change imports
- fix wiki links - fix wiki links
### **0.8.6** 2020/11/09 mandic00@live.com ### **0.8.6** 2020/11/09 mandic00@live.com
- add wasm bundle - add wasm bundle
### **0.8.5** 2020/11/09 mandic00@live.com ### **0.8.5** 2020/11/09 mandic00@live.com
- reimplemented blazeface processing - reimplemented blazeface processing
### **0.8.4** 2020/11/09 mandic00@live.com ### **0.8.4** 2020/11/09 mandic00@live.com
- added additional gestures - added additional gestures
- implemented blink detection - implemented blink detection
- fix wasm module - fix wasm module
### **0.8.3** 2020/11/08 mandic00@live.com ### **0.8.3** 2020/11/08 mandic00@live.com
- refresh - refresh
- optimizations - optimizations
### **0.8.2** 2020/11/08 mandic00@live.com ### **0.8.2** 2020/11/08 mandic00@live.com
- fix typos - fix typos
- commit - commit
### **0.8.1** 2020/11/07 mandic00@live.com ### **0.8.1** 2020/11/07 mandic00@live.com
- fix hand detection performance - fix hand detection performance
- optimized model loader - optimized model loader
- merge branch 'main' of https://github.com/vladmandic/human into main - merge branch 'main' of https://github.com/vladmandic/human into main
@ -222,52 +283,65 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
- fix nms sync call - fix nms sync call
### **0.7.6** 2020/11/06 mandic00@live.com ### **0.7.6** 2020/11/06 mandic00@live.com
- fixed memory leaks and updated docs - fixed memory leaks and updated docs
- model tuning - model tuning
- cache invalidation improvements - cache invalidation improvements
- full async operations - full async operations
### **0.7.5** 2020/11/05 mandic00@live.com ### **0.7.5** 2020/11/05 mandic00@live.com
- implemented dev-server - implemented dev-server
### **0.7.4** 2020/11/05 mandic00@live.com ### **0.7.4** 2020/11/05 mandic00@live.com
- fix canvas size on different orientation - fix canvas size on different orientation
- switched from es2020 to es2018 build target - switched from es2020 to es2018 build target
### **0.7.3** 2020/11/05 mandic00@live.com ### **0.7.3** 2020/11/05 mandic00@live.com
- optimized camera and mobile layout - optimized camera and mobile layout
- fixed worker and filter compatibility - fixed worker and filter compatibility
- removed test code - removed test code
### **0.7.2** 2020/11/04 mandic00@live.com ### **0.7.2** 2020/11/04 mandic00@live.com
- major work on handpose model - major work on handpose model
### **0.7.1** 2020/11/04 mandic00@live.com ### **0.7.1** 2020/11/04 mandic00@live.com
- changed demo build process - changed demo build process
### **0.6.7** 2020/11/04 mandic00@live.com ### **0.6.7** 2020/11/04 mandic00@live.com
- implemented simple gesture recognition - implemented simple gesture recognition
### **0.6.6** 2020/11/04 mandic00@live.com ### **0.6.6** 2020/11/04 mandic00@live.com
- remove debug code - remove debug code
### **0.6.5** 2020/11/04 mandic00@live.com ### **0.6.5** 2020/11/04 mandic00@live.com
- redo hand detection - redo hand detection
### **0.6.4** 2020/11/03 mandic00@live.com ### **0.6.4** 2020/11/03 mandic00@live.com
- added manifest - added manifest
### **0.6.3** 2020/11/03 mandic00@live.com ### **0.6.3** 2020/11/03 mandic00@live.com
- enhanced processing resolution - enhanced processing resolution
- fix pause restart - fix pause restart
- complete model refactoring - complete model refactoring
- fixed typo - fixed typo
### **0.6.2** 2020/11/02 mandic00@live.com ### **0.6.2** 2020/11/02 mandic00@live.com
- optimized demo - optimized demo
- package update - package update
### **0.6.1** 2020/11/02 mandic00@live.com ### **0.6.1** 2020/11/02 mandic00@live.com
- major performance improvements for all models - major performance improvements for all models
- revert "optimized canvas handling" - revert "optimized canvas handling"
- optimized canvas handling - optimized canvas handling
@ -276,61 +350,80 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
- added tfjs-vis to distribution - added tfjs-vis to distribution
### **0.5.5** 2020/11/01 mandic00@live.com ### **0.5.5** 2020/11/01 mandic00@live.com
- changed defaults - changed defaults
### **0.5.4** 2020/11/01 mandic00@live.com ### **0.5.4** 2020/11/01 mandic00@live.com
- implemented memory profiler - implemented memory profiler
### **0.5.3** 2020/10/30 mandic00@live.com ### **0.5.3** 2020/10/30 mandic00@live.com
- improved debug logging - improved debug logging
### **0.5.2** 2020/10/30 mandic00@live.com ### **0.5.2** 2020/10/30 mandic00@live.com
- added wasm and webgpu backends - added wasm and webgpu backends
### **0.5.1** 2020/10/30 mandic00@live.com ### **0.5.1** 2020/10/30 mandic00@live.com
- improve demo line continous draws - improve demo line continous draws
### **0.4.10** 2020/10/30 mandic00@live.com ### **0.4.10** 2020/10/30 mandic00@live.com
- fix for seedrandom - fix for seedrandom
### **0.4.9** 2020/10/29 mandic00@live.com ### **0.4.9** 2020/10/29 mandic00@live.com
### **0.4.8** 2020/10/28 mandic00@live.com ### **0.4.8** 2020/10/28 mandic00@live.com
- revert "updated menu handler" - revert "updated menu handler"
- fix webpack compatibility issue - fix webpack compatibility issue
### **0.4.7** 2020/10/27 mandic00@live.com ### **0.4.7** 2020/10/27 mandic00@live.com
### **0.4.6** 2020/10/27 mandic00@live.com ### **0.4.6** 2020/10/27 mandic00@live.com
- fix firefox compatibility bug - fix firefox compatibility bug
### **0.4.5** 2020/10/27 mandic00@live.com ### **0.4.5** 2020/10/27 mandic00@live.com
### **0.4.4** 2020/10/27 mandic00@live.com ### **0.4.4** 2020/10/27 mandic00@live.com
- implelented input resizing - implelented input resizing
### **0.4.3** 2020/10/22 mandic00@live.com ### **0.4.3** 2020/10/22 mandic00@live.com
### **0.4.2** 2020/10/20 mandic00@live.com ### **0.4.2** 2020/10/20 mandic00@live.com
- log initialization - log initialization
### **0.4.1** 2020/10/19 mandic00@live.com ### **0.4.1** 2020/10/19 mandic00@live.com
- breaking change: convert to object class - breaking change: convert to object class
- compatibility notes - compatibility notes
### **0.3.9** 2020/10/18 mandic00@live.com ### **0.3.9** 2020/10/18 mandic00@live.com
- implemented image filters - implemented image filters
- pure tensor pipeline without image converts - pure tensor pipeline without image converts
- autodetect skipframes - autodetect skipframes
### **0.3.8** 2020/10/17 mandic00@live.com ### **0.3.8** 2020/10/17 mandic00@live.com
- new menu layout - new menu layout
### **0.3.7** 2020/10/17 mandic00@live.com ### **0.3.7** 2020/10/17 mandic00@live.com
- added diagnostics output - added diagnostics output
- parallelized agegender operations - parallelized agegender operations
### **0.3.6** 2020/10/17 mandic00@live.com ### **0.3.6** 2020/10/17 mandic00@live.com
- fixed webcam initialization - fixed webcam initialization
- fixed memory leaks and added scoped runs - fixed memory leaks and added scoped runs
- modularized draw - modularized draw
@ -338,17 +431,21 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
- refactored package file layout - refactored package file layout
### **0.3.5** 2020/10/16 mandic00@live.com ### **0.3.5** 2020/10/16 mandic00@live.com
- added auto-generated changelog - added auto-generated changelog
### **0.3.4** 2020/10/16 mandic00@live.com ### **0.3.4** 2020/10/16 mandic00@live.com
- added camera selection - added camera selection
- optimized blazeface anchors - optimized blazeface anchors
- added error handling - added error handling
### **0.3.3** 2020/10/15 mandic00@live.com ### **0.3.3** 2020/10/15 mandic00@live.com
- added blazeface back and front models - added blazeface back and front models
### **0.3.2** 2020/10/15 mandic00@live.com ### **0.3.2** 2020/10/15 mandic00@live.com
- reduced web worker latency - reduced web worker latency
- added debugging and versioning - added debugging and versioning
- optimized demos and added scoped runs - optimized demos and added scoped runs
@ -356,37 +453,49 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
### **0.3.1** 2020/10/14 mandic00@live.com ### **0.3.1** 2020/10/14 mandic00@live.com
### **0.2.10** 2020/10/14 mandic00@live.com ### **0.2.10** 2020/10/14 mandic00@live.com
- added emotion backend - added emotion backend
- module parametrization and performance monitoring - module parametrization and performance monitoring
- implemented multi-hand support - implemented multi-hand support
- fixed documentation typos - fixed documentation typos
### **0.2.9** 2020/10/13 mandic00@live.com ### **0.2.9** 2020/10/13 mandic00@live.com
- added node build and demo - added node build and demo
### **0.2.8** 2020/10/13 mandic00@live.com ### **0.2.8** 2020/10/13 mandic00@live.com
- added example image - added example image
### **0.2.7** 2020/10/13 mandic00@live.com ### **0.2.7** 2020/10/13 mandic00@live.com
- new examples - new examples
### **0.2.6** 2020/10/13 mandic00@live.com ### **0.2.6** 2020/10/13 mandic00@live.com
- enable all models by default - enable all models by default
### **0.2.5** 2020/10/12 mandic00@live.com ### **0.2.5** 2020/10/12 mandic00@live.com
- fixed memory leak - fixed memory leak
### **0.2.4** 2020/10/12 mandic00@live.com ### **0.2.4** 2020/10/12 mandic00@live.com
- removed extra files - removed extra files
### **0.2.3** 2020/10/12 mandic00@live.com ### **0.2.3** 2020/10/12 mandic00@live.com
### **0.2.2** 2020/10/12 mandic00@live.com ### **0.2.2** 2020/10/12 mandic00@live.com
### **tag: 1.0** 2020/10/12 mandic00@live.com
### **release: 1.0** 2020/10/12 mandic00@live.com
### **0.2.1** 2020/10/12 mandic00@live.com ### **0.2.1** 2020/10/12 mandic00@live.com
- added sample image - added sample image
- initial public commit - initial public commit
- initial commit - initial commit

@ -1,4 +1,4 @@
## Configuration # Configuration
Detailed configuration options are explained below, but they are best seen in the menus present in the `demo` application: Detailed configuration options are explained below, but they are best seen in the menus present in the `demo` application:
*note: some advanced configuration options are not exposed in the UI* *note: some advanced configuration options are not exposed in the UI*
@ -10,12 +10,14 @@ Detailed configuration options are explained below, but they are best seen in th
<br> <br>
Main configuration objects are: Main configuration objects are:
- **config.filter**: controls image pre-processing - **config.filter**: controls image pre-processing
- **config.face**: controls face detection - **config.face**: controls face detection
- **config.body**: controls body pose detection - **config.body**: controls body pose detection
- **config.hand**: contros hand and finger detection - **config.hand**: contros hand and finger detection
With **config.face** having several subsections: With **config.face** having several subsections:
- **config.face.mesh**: controls facial mesh and landscape detection - **config.face.mesh**: controls facial mesh and landscape detection
- **config.face.iris**: controls iris detection - **config.face.iris**: controls iris detection
- **config.face.age**: controls age prediction - **config.face.age**: controls age prediction
@ -31,7 +33,6 @@ Note that user object and default configuration are merged using deep-merge, so
All configuration details can be changed in real-time! All configuration details can be changed in real-time!
```js ```js
config = { config = {
backend: 'webgl', // select tfjs backend to use backend: 'webgl', // select tfjs backend to use

@ -1,4 +1,4 @@
## Credits # Credits
- Face Detection: [**MediaPipe BlazeFace**](https://drive.google.com/file/d/1f39lSzU5Oq-j_OXgS67KfN5wNsoeAZ4V/view) - Face Detection: [**MediaPipe BlazeFace**](https://drive.google.com/file/d/1f39lSzU5Oq-j_OXgS67KfN5wNsoeAZ4V/view)
- Face Boxes: [**FaceBoxes**](https://github.com/TropComplique/FaceBoxes-tensorflow/) - Face Boxes: [**FaceBoxes**](https://github.com/TropComplique/FaceBoxes-tensorflow/)

@ -1,10 +1,11 @@
## Demos # Demos
Demos are included in `/demo`: Demos are included in `/demo`:
<br> <br>
**Browser**: **Browser**:
- `index.html`: Full demo using Browser with ESM module, includes selectable backends and webworkers - `index.html`: Full demo using Browser with ESM module, includes selectable backends and webworkers
*You can run browser demo either live from git pages, by serving demo folder from your web server or use *You can run browser demo either live from git pages, by serving demo folder from your web server or use
@ -12,7 +13,7 @@ included micro http2 server with source file monitoring and dynamic rebuild*
<br> <br>
### Changing Demo Target ## Changing Demo Target
Demo in `demo/index.html` loads `dist/demo-browser-index.js` which is built from sources in `demo`, starting with `demo/browser` Demo in `demo/index.html` loads `dist/demo-browser-index.js` which is built from sources in `demo`, starting with `demo/browser`
This bundled version is needed since mobile browsers (e.g. Chrome on Android) do not support native modules loading yet This bundled version is needed since mobile browsers (e.g. Chrome on Android) do not support native modules loading yet
@ -21,10 +22,10 @@ If your target is desktop, alternatively you can load `demo/browser.js` directly
<br> <br>
### Demo Options ## Demo Options
Demo implements several ways to use `Human` library, Demo implements several ways to use `Human` library,
all configurable in `browse.js:ui` configuration object and in the UI itself: all configurable in `browse.js:ui` configuration object and in the UI itself:
- `ui.buffered`: run detection and screen refresh in a sequence or as separate buffered functions - `ui.buffered`: run detection and screen refresh in a sequence or as separate buffered functions
- `ui.bufferedFPSTarget`: when using buffered execution this target fps for screen refresh - `ui.bufferedFPSTarget`: when using buffered execution this target fps for screen refresh
@ -41,18 +42,20 @@ all configurable in `browse.js:ui` configuration object and in the UI itself:
<br> <br>
### Dev Server ## Dev Server
To use dev server, you must have an SSL certificate since browsers enforce *https* for webcam access To use dev server, you must have an SSL certificate since browsers enforce *https* for webcam access
You can use a provided ones self-signed certificate or configure your own You can use a provided ones self-signed certificate or configure your own
Self-signed test certificate was generated using: Self-signed test certificate was generated using:
```shell ```shell
openssl req -x509 -newkey rsa:4096 -nodes -keyout https.key -out https.crt -days 365 -subj "/C=US/ST=Florida/L=Miami/O=@vladmandic" openssl req -x509 -newkey rsa:4096 -nodes -keyout https.key -out https.crt -days 365 -subj "/C=US/ST=Florida/L=Miami/O=@vladmandic"
``` ```
If you want to use your own certificate, edit `server/dev.js`: If you want to use your own certificate, edit `server/dev.js`:
```js ```js
const options = { const options = {
key: fs.readFileSync('server/https.key'), key: fs.readFileSync('server/https.key'),
@ -61,9 +64,11 @@ const options = {
``` ```
Once SSL certificates have been provided (defalt test certificate or your own), simply run Once SSL certificates have been provided (defalt test certificate or your own), simply run
```shell ```shell
npm run dev npm run dev
``` ```
On first start, it will install all development dependencies required to rebuild `Human` library On first start, it will install all development dependencies required to rebuild `Human` library
Dev version of the library runs without code minification for easier debugging while production version is fully minified Dev version of the library runs without code minification for easier debugging while production version is fully minified
@ -113,9 +118,11 @@ found 0 vulnerabilities
2020-12-12 10:12:55 DATA: GET/2.0 200 application/octet-stream 5502280 /models/handskeleton.bin ::ffff:192.168.0.200 2020-12-12 10:12:55 DATA: GET/2.0 200 application/octet-stream 5502280 /models/handskeleton.bin ::ffff:192.168.0.200
... ...
``` ```
<br> <br>
**NodeJS**: **NodeJS**:
- `node.js`: Demo using NodeJS with CommonJS module - `node.js`: Demo using NodeJS with CommonJS module
This is a very simple demo as althought `Human` library is compatible with NodeJS execution This is a very simple demo as althought `Human` library is compatible with NodeJS execution
and is able to load images and models from local filesystem, and is able to load images and models from local filesystem,

@ -1,4 +1,4 @@
## Face Feature Embedding and Simmilarity Compare # Face Feature Embedding and Simmilarity Compare
<br> <br>

@ -1,4 +1,4 @@
## Gesture Recognition # Gesture Recognition
Gesture recognition is done by looking up coordinates of different detected landmarks Gesture recognition is done by looking up coordinates of different detected landmarks
@ -21,6 +21,7 @@ There are three pre-defined methods:
- "*`<finger>` forward `<finger>` up*" - "*`<finger>` forward `<finger>` up*"
Example output of `result.gesture`: Example output of `result.gesture`:
```js ```js
[ [
{face: "0", gesture: "facing camera"} {face: "0", gesture: "facing camera"}
@ -31,4 +32,5 @@ Example output of `result.gesture`:
{hand: "0", gesture: "thumb forward middlefinger up"} {hand: "0", gesture: "thumb forward middlefinger up"}
] ]
``` ```
Where number after gesture refers to number of person that detection belongs to in scenes with multiple people. Where number after gesture refers to number of person that detection belongs to in scenes with multiple people.

@ -4,7 +4,7 @@
<br> <br>
### Project pages: ### Project pages
- [**Live Demo**](https://vladmandic.github.io/human/demo/index.html) - [**Live Demo**](https://vladmandic.github.io/human/demo/index.html)
- [**Code Repository**](https://github.com/vladmandic/human) - [**Code Repository**](https://github.com/vladmandic/human)
@ -12,7 +12,7 @@
- [**Issues Tracker**](https://github.com/vladmandic/human/issues) - [**Issues Tracker**](https://github.com/vladmandic/human/issues)
- [**Change Log**](https://github.com/vladmandic/human/wiki/Change-Log) - [**Change Log**](https://github.com/vladmandic/human/wiki/Change-Log)
### Wiki pages: ### Wiki pages
- [**Home**](https://github.com/vladmandic/human/wiki) - [**Home**](https://github.com/vladmandic/human/wiki)
- [**Demos**](https://github.com/vladmandic/human/wiki/Demos) - [**Demos**](https://github.com/vladmandic/human/wiki/Demos)
@ -23,7 +23,7 @@
- [**Face Embedding and Recognition**](https://github.com/vladmandic/human/wiki/Embedding) - [**Face Embedding and Recognition**](https://github.com/vladmandic/human/wiki/Embedding)
- [**Gesture Recognition**](https://github.com/vladmandic/human/wiki/Gesture) - [**Gesture Recognition**](https://github.com/vladmandic/human/wiki/Gesture)
### Additional notes: ### Additional notes
- [**Notes on Backends**](https://github.com/vladmandic/human/wiki/Backends) - [**Notes on Backends**](https://github.com/vladmandic/human/wiki/Backends)
- [**Development Server**](https://github.com/vladmandic/human/wiki/Development-Server) - [**Development Server**](https://github.com/vladmandic/human/wiki/Development-Server)
@ -35,7 +35,8 @@
<br> <br>
Compatible with *Browser*, *WebWorker* and *NodeJS* execution on both Windows and Linux Compatible with *Browser*, *WebWorker* and *NodeJS* execution on both Windows and Linux
- Browser/WebWorker: Compatible with *CPU*, *WebGL*, *WASM* and *WebGPU* backends - Browser/WebWorker: Compatible with *CPU*, *WebGL*, *WASM* and *WebGPU* backends
- NodeJS: Compatible with software *tfjs-node* and CUDA accelerated backends *tfjs-node-gpu* - NodeJS: Compatible with software *tfjs-node* and CUDA accelerated backends *tfjs-node-gpu*
- (and maybe with React-Native as it doesn't use any DOM objects) - (and maybe with React-Native as it doesn't use any DOM objects)

@ -1,19 +1,19 @@
## Installation # Installation
<br> <br>
**Important** **Important**
*The packaged version of `Human` includes `TensorFlow/JS (TFJS) 3.2.0` library which can be accessed via `human.tf`* *The packaged version of `Human` includes `TensorFlow/JS (TFJS) 3.2.0` library which can be accessed via `human.tf`*
*You should NOT manually load another instance of `tfjs` unless you're using specific `.nobudle` edition of `Human, *You should NOT manually load another instance of `tfjs` unless you're using specific `.nobudle` edition of `Human,
but if you do, be aware of possible version conflicts* but if you do, be aware of possible version conflicts*
<br> <br>
There are multiple ways to use `Human` library, pick one that suits you: There are multiple ways to use `Human` library, pick one that suits you:
### Included ## Included
- `dist/human.js`: IIFE format bundle with TFJS for Browsers - `dist/human.js`: IIFE format bundle with TFJS for Browsers
- `dist/human.esm.js`: ESM format bundle with TFJS for Browsers - `dist/human.esm.js`: ESM format bundle with TFJS for Browsers
@ -25,6 +25,7 @@ All versions include `sourcemap` *(.map)* and build `manifest` *(.json)*
While `Human` is in pre-release mode, all bundles are non-minified While `Human` is in pre-release mode, all bundles are non-minified
Defaults: Defaults:
```json ```json
{ {
"main": "dist/human.node.js", "main": "dist/human.node.js",
@ -35,7 +36,7 @@ Defaults:
<br> <br>
### 1. [IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE) script ## 1. [IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE) script
*Simplest way for usage within Browser* *Simplest way for usage within Browser*
@ -43,10 +44,10 @@ Simply download `dist/human.js`, include it in your `HTML` file & it's ready to
```html ```html
<script src="dist/human.js"><script> <script src="dist/human.js"><script>
``` ```
IIFE script auto-registers global namespace `Human` within global `Window` object IIFE script auto-registers global namespace `Human` within global `Window` object
Which you can use to create instance of `human` library: Which you can use to create instance of `human` library:
```js ```js
const human = new Human(); const human = new Human();
@ -56,16 +57,18 @@ This way you can also use `Human` library within embbedded `<script>` tag within
<br> <br>
### 2. [ESM](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) module ## 2. [ESM](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) module
*Recommended for usage within `Browser`* *Recommended for usage within `Browser`*
#### **2.1 Using Script Module** ### **2.1 Using Script Module**
You could use same syntax within your main `JS` file if it's imported with `<script type="module">` You could use same syntax within your main `JS` file if it's imported with `<script type="module">`
```html ```html
<script src="./index.js" type="module"> <script src="./index.js" type="module">
``` ```
and then in your `index.js` and then in your `index.js`
```js ```js
@ -73,15 +76,17 @@ and then in your `index.js`
const human = new Human(); const human = new Human();
``` ```
#### **2.2 With Bundler** ### **2.2 With Bundler**
If you're using bundler *(such as rollup, webpack, parcel, browserify, esbuild)* to package your client application, If you're using bundler *(such as rollup, webpack, parcel, browserify, esbuild)* to package your client application,
you can import ESM version of `Human` library which supports full tree shaking you can import ESM version of `Human` library which supports full tree shaking
Install with: Install with:
```shell ```shell
npm install @vladmandic/human npm install @vladmandic/human
``` ```
```js ```js
import Human from '@vladmandic/human'; // points to @vladmandic/human/dist/human.esm.js import Human from '@vladmandic/human'; // points to @vladmandic/human/dist/human.esm.js
// you can also force-load specific version // you can also force-load specific version
@ -92,9 +97,11 @@ Install with:
Or if you prefer to package your version of `tfjs`, you can use `nobundle` version Or if you prefer to package your version of `tfjs`, you can use `nobundle` version
Install with: Install with:
```shell ```shell
npm install @vladmandic/human @tensorflow/tfjs npm install @vladmandic/human @tensorflow/tfjs
``` ```
```js ```js
import tf from '@tensorflow/tfjs' import tf from '@tensorflow/tfjs'
import Human from '@vladmandic/human/dist/human.esm-nobundle.js'; // same functionality as default import, but without tfjs bundled import Human from '@vladmandic/human/dist/human.esm-nobundle.js'; // same functionality as default import, but without tfjs bundled
@ -103,7 +110,7 @@ Install with:
<br> <br>
### 3. [NPM](https://www.npmjs.com/) module ## 3. [NPM](https://www.npmjs.com/) module
*Recommended for `NodeJS` projects that will execute in the backend* *Recommended for `NodeJS` projects that will execute in the backend*
@ -111,10 +118,13 @@ Install with:
Entry point are bundles in CommonJS format `dist/human.node.js` and `dist/human.node-gpu.js` Entry point are bundles in CommonJS format `dist/human.node.js` and `dist/human.node-gpu.js`
Install with: Install with:
```shell ```shell
npm install @vladmandic/human @tensorflow/tfjs-node npm install @vladmandic/human @tensorflow/tfjs-node
``` ```
And then use with: And then use with:
```js ```js
const tf = require('@tensorflow/tfjs-node'); // can also use '@tensorflow/tfjs-node-gpu' if you have environment with CUDA extensions const tf = require('@tensorflow/tfjs-node'); // can also use '@tensorflow/tfjs-node-gpu' if you have environment with CUDA extensions
const Human = require('@vladmandic/human').default; // points to @vladmandic/human/dist/human.node.js const Human = require('@vladmandic/human').default; // points to @vladmandic/human/dist/human.node.js
@ -122,20 +132,23 @@ And then use with:
``` ```
Or for CUDA accelerated NodeJS backend: Or for CUDA accelerated NodeJS backend:
```shell ```shell
npm install @vladmandic/human @tensorflow/tfjs-node-gpu npm install @vladmandic/human @tensorflow/tfjs-node-gpu
``` ```
And then use with: And then use with:
```js ```js
const tf = require('@tensorflow/tfjs-node-gpu'); // can also use '@tensorflow/tfjs-node-gpu' if you have environment with CUDA extensions const tf = require('@tensorflow/tfjs-node-gpu'); // can also use '@tensorflow/tfjs-node-gpu' if you have environment with CUDA extensions
const Human = require('@vladmandic/human/dist/human.node-gpu.js').default; // points to @vladmandic/human/dist/human.node.js const Human = require('@vladmandic/human/dist/human.node-gpu.js').default; // points to @vladmandic/human/dist/human.node.js
const human = new Human(); const human = new Human();
``` ```
Since NodeJS projects load `weights` from local filesystem instead of using `http` calls, you must modify default configuration to include correct paths with `file://` prefix Since NodeJS projects load `weights` from local filesystem instead of using `http` calls, you must modify default configuration to include correct paths with `file://` prefix
For example: For example:
```js ```js
const config = { const config = {
body: { enabled: true, modelPath: 'file://models/posenet/model.json' }, body: { enabled: true, modelPath: 'file://models/posenet/model.json' },
@ -144,7 +157,7 @@ const config = {
<br> <br>
### Weights ## Weights
Pretrained model weights are includes in `./models` Pretrained model weights are includes in `./models`
Default configuration uses relative paths to you entry script pointing to `../models` Default configuration uses relative paths to you entry script pointing to `../models`

@ -1,5 +0,0 @@
# math
distance = \sqrt{ \sum_{i=1}^{192} (first_i - second_i)^2 }
confidence = 2 * (0.5 - distance)
simmilarity = 1 - confidence

@ -1,4 +1,4 @@
## List of all models included in Human library: # List of all models included in Human library
<br> <br>
@ -24,7 +24,7 @@
*Note: All model definitions JSON files are parsed for human readability* *Note: All model definitions JSON files are parsed for human readability*
<br> <br>
## Credits ## Credits

@ -1,4 +1,4 @@
## Outputs # Outputs
Result of `humand.detect()` is a single object that includes data for all enabled modules and all detected objects: Result of `humand.detect()` is a single object that includes data for all enabled modules and all detected objects:
@ -8,7 +8,9 @@ result = {
face: // <array of detected objects> face: // <array of detected objects>
[ [
{ {
confidence, // <number> confidence, // <number> returns faceConfidence if exists, otherwise boxConfidence
faceConfidence // <number> confidence in detection box after running mesh
boxConfidence // <number> confidence in detection box before running mesh
box, // <array [x, y, width, height]> box, // <array [x, y, width, height]>
rawBox, // normalized values for box, only set if returnRawData is set rawBox, // normalized values for box, only set if returnRawData is set
mesh, // <array of 3D points [x, y, z]> 468 base points & 10 iris points mesh, // <array of 3D points [x, y, z]> 468 base points & 10 iris points

@ -1,4 +1,4 @@
## Performance # Performance
Performance will vary depending on your hardware, but also on number of resolution of input video/image, enabled modules as well as their parameters Performance will vary depending on your hardware, but also on number of resolution of input video/image, enabled modules as well as their parameters
@ -6,7 +6,7 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~2
<br> <br>
### Performance per module on a **notebook** with nVidia GTX1050 GPU on a FullHD input: ## Performance per module on a **notebook** with nVidia GTX1050 GPU on a FullHD input
- Enabled all: 20 FPS - Enabled all: 20 FPS
- Image filters: 80 FPS (standalone) - Image filters: 80 FPS (standalone)
@ -20,7 +20,7 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~2
- Hand: 40 FPS (standalone) - Hand: 40 FPS (standalone)
- Body: 50 FPS (standalone) - Body: 50 FPS (standalone)
### Performance per module on a **smartphone** with Snapdragon 855 on a FullHD input: ## Performance per module on a **smartphone** with Snapdragon 855 on a FullHD input
- Enabled all: 10 FPS - Enabled all: 10 FPS
- Image filters: 30 FPS (standalone) - Image filters: 30 FPS (standalone)
@ -37,4 +37,3 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~2
<br> <br>
For performance details, see output of `result.performance` object during after running inference For performance details, see output of `result.performance` object during after running inference

@ -2,13 +2,14 @@
## Browser ## Browser
### Desktop: ### Desktop
- **Chrome**: *Supported* - **Chrome**: *Supported*
- Minimum version Chrome 60 from 2017 - Minimum version Chrome 60 from 2017
- WASM: For performance reasons it is recommended to enable `WebAssembly SIMD support` in `chrome://flags` - WASM: For performance reasons it is recommended to enable `WebAssembly SIMD support` in `chrome://flags`
- **Firefox**: *Supported* - **Firefox**: *Supported*
- Minimum version Firefox 55 from 2017 - Minimum version Firefox 55 from 2017
- WebGL: Lower performance due to Firefox handing of WebGL memory garbage collection - WebGL: Lower performance due to Firefox handing of WebGL memory garbage collection
- WASM: For performance reasons it is recommended to enable `javascript.options.wasm.simd` in `about:config` - WASM: For performance reasons it is recommended to enable `javascript.options.wasm.simd` in `about:config`
- WASM: Async operations are not supported (running multiple detections or models in parallel) Due to missing threaded WASM support in Firefox - WASM: Async operations are not supported (running multiple detections or models in parallel) Due to missing threaded WASM support in Firefox
- Web Workers: Not supported due to Firefox missing implementation for `OffscreenCanvas` - Web Workers: Not supported due to Firefox missing implementation for `OffscreenCanvas`
@ -22,7 +23,8 @@
- **IE 11** or older: *Not supported* - **IE 11** or older: *Not supported*
- Human requires ES2018 support in browsers - Human requires ES2018 support in browsers
### Mobile: ### Mobile
- **Chrome**: *Supported* - **Chrome**: *Supported*
- **Firefox**: *Supported* - **Firefox**: *Supported*
- **Safari**: *Limited support* - **Safari**: *Limited support*
@ -33,5 +35,6 @@
- Human requires ES2018 support in browsers - Human requires ES2018 support in browsers
## NodeJS ## NodeJS
- Minimum version 12.0 - Minimum version 12.0
- WIP: Face models are not supported in NodeJS as TensorFlow/JS for Node (`tfjs-node`) does not implement required function - WIP: Face models are not supported in NodeJS as TensorFlow/JS for Node (`tfjs-node`) does not implement required function

@ -1,8 +1,9 @@
## Profiling # Profiling
If `config.profile` is enabled, call to `human.profile()` will return detailed profiling data from the last detect invokation. If `config.profile` is enabled, call to `human.profile()` will return detailed profiling data from the last detect invokation.
example: example:
```js ```js
result = { result = {
{age: {…}, gender: {…}, emotion: {…}} {age: {…}, gender: {…}, emotion: {…}}

@ -1,4 +1,4 @@
## Usage # Usage
`Human` library does not require special initialization `Human` library does not require special initialization
All configuration is done in a single JSON object and all model weights are dynamically loaded upon their first usage All configuration is done in a single JSON object and all model weights are dynamically loaded upon their first usage
@ -45,9 +45,10 @@ Additionally, `Human` library exposes several objects and methods:
<br> <br>
Note that when using `Human` library in `NodeJS`, you must load and parse the image *before* you pass it for detection and dispose it afterwards Note that when using `Human` library in `NodeJS`, you must load and parse the image *before* you pass it for detection and dispose it afterwards
Input format is `Tensor4D[1, width, height, 3]` of type `float32` Input format is `Tensor4D[1, width, height, 3]` of type `float32`
For example: For example:
```js ```js
const imageFile = '../assets/sample1.jpg'; const imageFile = '../assets/sample1.jpg';
const buffer = fs.readFileSync(imageFile); const buffer = fs.readFileSync(imageFile);