mirror of https://github.com/vladmandic/human
update with markdownlint
Vladimir Mandic
parent
4091054e6d
commit
7bb1673bd2
12
Backends.md
12
Backends.md
|
|
@ -1,10 +1,10 @@
|
|||
## Backends
|
||||
# Backends
|
||||
|
||||
Human library includes pre-bundled `CPU`, `WASM` and `WebGL` backends, there is no need to load them externally.
|
||||
|
||||
<br>
|
||||
|
||||
### Notes:
|
||||
## Notes
|
||||
|
||||
- CPU backend usage is not recommended for performance reasons
|
||||
as TensorFlow implementation in JavaScript is by far the slowest
|
||||
|
|
@ -16,23 +16,26 @@ Human library includes pre-bundled `CPU`, `WASM` and `WebGL` backends, there is
|
|||
- Reduce size of input image (e.g. webcam)
|
||||
as WASM does not have optimized image extraction methods
|
||||
|
||||
#### How to enable WASM SIMD support
|
||||
### How to enable WASM SIMD support
|
||||
|
||||
Chrome:
|
||||
|
||||
- Navigate to <chrome://flags>
|
||||
- `WebAssembly SIMD support` set to Enabled
|
||||
|
||||
Edge:
|
||||
|
||||
- Navigate to <chrome://flags>
|
||||
- `WebAssembly SIMD support` set to Enabled
|
||||
|
||||
Firefox:
|
||||
|
||||
- Navigate to <about:config>
|
||||
- `javascript.options.wasm_simd` set to True
|
||||
|
||||
<br>
|
||||
|
||||
### Known Issues:
|
||||
## Known Issues
|
||||
|
||||
- **WASM** backend without SIMD has limited processing capabilities before it causes issues
|
||||
To avoid issues, start `human` with most models disabled and if needed, enable them one by one as each initializes
|
||||
|
|
@ -42,4 +45,3 @@ Firefox:
|
|||
- **NodeJS** backend is missing key operations and thus neither `Face` nor `Hand` models are currently supported
|
||||
|
||||
<br>
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
## Build Process
|
||||
# Build Process
|
||||
|
||||
If you want to modify the library and perform a full rebuild:
|
||||
|
||||
|
|
|
|||
161
Change-Log.md
161
Change-Log.md
|
|
@ -1,7 +1,6 @@
|
|||
|
||||
# @vladmandic/human
|
||||
|
||||
Version: **0.30.4**
|
||||
Version: **0.30.5**
|
||||
Description: **Human: AI-powered 3D Face Detection, Face Embedding & Recognition, Body Pose Tracking, Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction & Gesture Recognition**
|
||||
|
||||
Author: **Vladimir Mandic <mandic00@live.com>**
|
||||
|
|
@ -10,209 +9,271 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
|
|||
|
||||
## Changelog
|
||||
|
||||
### **HEAD -> main, tag: v0.30.4, origin/main, origin/HEAD** 2021/03/01 mandic00@live.com
|
||||
### **HEAD -> main** 2021/03/02 mandic00@live.com
|
||||
|
||||
- add debug logging flag
|
||||
|
||||
### **0.30.4** 2021/03/01 mandic00@live.com
|
||||
|
||||
- added skipinitial flag
|
||||
|
||||
### **tag: v0.30.3** 2021/02/28 mandic00@live.com
|
||||
### **0.30.3** 2021/02/28 mandic00@live.com
|
||||
|
||||
- typo
|
||||
|
||||
### **tag: v0.30.2** 2021/02/26 mandic00@live.com
|
||||
### **0.30.2** 2021/02/26 mandic00@live.com
|
||||
|
||||
- rebuild
|
||||
- fix typo
|
||||
|
||||
### **tag: v0.30.1** 2021/02/25 mandic00@live.com
|
||||
### **0.30.1** 2021/02/25 mandic00@live.com
|
||||
|
||||
### **tag: v0.20.11** 2021/02/24 mandic00@live.com
|
||||
|
||||
### **tag: v0.20.10** 2021/02/22 mandic00@live.com
|
||||
### **0.20.11** 2021/02/24 mandic00@live.com
|
||||
|
||||
|
||||
### **0.20.10** 2021/02/22 mandic00@live.com
|
||||
|
||||
|
||||
### **0.20.9** 2021/02/21 mandic00@live.com
|
||||
|
||||
### **tag: v0.20.9** 2021/02/21 mandic00@live.com
|
||||
- remove extra items
|
||||
- simmilarity fix
|
||||
|
||||
### **tag: v0.20.8** 2021/02/21 mandic00@live.com
|
||||
### **0.20.8** 2021/02/21 mandic00@live.com
|
||||
|
||||
- embedding fix
|
||||
|
||||
### **tag: v0.20.7** 2021/02/21 mandic00@live.com
|
||||
### **0.20.7** 2021/02/21 mandic00@live.com
|
||||
|
||||
- build fix
|
||||
|
||||
### **tag: v0.20.6** 2021/02/21 mandic00@live.com
|
||||
### **0.20.6** 2021/02/21 mandic00@live.com
|
||||
|
||||
- embedding fix
|
||||
|
||||
### **tag: v0.20.5** 2021/02/21 mandic00@live.com
|
||||
### **0.20.5** 2021/02/21 mandic00@live.com
|
||||
|
||||
- fix imagefx and add dev builds
|
||||
|
||||
### **tag: v0.20.4, tag: 0.3, tag: 0.2, tag: 0.1** 2021/02/19 mandic00@live.com
|
||||
### **0.20.4** 2021/02/19 mandic00@live.com
|
||||
|
||||
- 0.20.4
|
||||
|
||||
### **tag: v0.20.3** 2021/02/17 mandic00@live.com
|
||||
### **0.20.3** 2021/02/17 mandic00@live.com
|
||||
|
||||
- rebuild
|
||||
|
||||
### **tag: v0.20.2** 2021/02/13 mandic00@live.com
|
||||
### **0.20.2** 2021/02/13 mandic00@live.com
|
||||
|
||||
- merge branch 'main' of https://github.com/vladmandic/human into main
|
||||
- create codeql-analysis.yml
|
||||
- create security.md
|
||||
- add templates
|
||||
|
||||
### **tag: v0.20.1** 2021/02/08 mandic00@live.com
|
||||
### **0.20.1** 2021/02/08 mandic00@live.com
|
||||
|
||||
- menu fixes
|
||||
- convert to typescript
|
||||
|
||||
### **tag: v0.11.5** 2021/02/06 mandic00@live.com
|
||||
### **0.11.5** 2021/02/06 mandic00@live.com
|
||||
|
||||
- added faceboxes alternative model
|
||||
|
||||
### **tag: v0.11.4** 2021/02/06 mandic00@live.com
|
||||
### **0.11.4** 2021/02/06 mandic00@live.com
|
||||
|
||||
### **tag: v0.11.3** 2021/02/02 mandic00@live.com
|
||||
|
||||
### **tag: v0.11.2** 2021/01/30 mandic00@live.com
|
||||
### **0.11.3** 2021/02/02 mandic00@live.com
|
||||
|
||||
|
||||
### **0.11.2** 2021/01/30 mandic00@live.com
|
||||
|
||||
- added warmup for nodejs
|
||||
|
||||
### **update for tfjs 3.0.0** 2021/01/29 mandic00@live.com
|
||||
|
||||
### **tag: v0.11.1** 2021/01/29 mandic00@live.com
|
||||
|
||||
### **tag: v0.10.2** 2021/01/22 mandic00@live.com
|
||||
### **0.11.1** 2021/01/29 mandic00@live.com
|
||||
|
||||
|
||||
### **0.10.2** 2021/01/22 mandic00@live.com
|
||||
|
||||
|
||||
### **0.10.1** 2021/01/20 mandic00@live.com
|
||||
|
||||
### **tag: v0.10.1** 2021/01/20 mandic00@live.com
|
||||
|
||||
### **0.9.26** 2021/01/18 mandic00@live.com
|
||||
|
||||
- fix face detection when mesh is disabled
|
||||
- added minification notes
|
||||
- version bump
|
||||
|
||||
### **0.9.25** 2021/01/13 mandic00@live.com
|
||||
|
||||
- added humangl custom backend
|
||||
- rebuild
|
||||
- code cleanup and enable minification
|
||||
- fix safari incopatibility
|
||||
|
||||
### **0.9.24** 2021/01/12 mandic00@live.com
|
||||
|
||||
- work on blazepose
|
||||
- full rebuild
|
||||
|
||||
### **0.9.23** 2021/01/11 mandic00@live.com
|
||||
|
||||
- added iris gesture
|
||||
- fix emotion labels
|
||||
- full rebuild
|
||||
|
||||
### **0.9.22** 2021/01/05 mandic00@live.com
|
||||
|
||||
- remove iris coords if iris is disabled
|
||||
- web worker fix
|
||||
|
||||
### **0.9.21** 2021/01/03 mandic00@live.com
|
||||
|
||||
|
||||
### **0.9.20** 2021/01/03 mandic00@live.com
|
||||
|
||||
- stricter linting, fix face annotations
|
||||
|
||||
### **0.9.19** 2020/12/23 mandic00@live.com
|
||||
|
||||
- added rawbox and rawmesh
|
||||
- variable name changes, setting .rawcoords only if necessary
|
||||
- option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream
|
||||
|
||||
### **0.9.18** 2020/12/16 mandic00@live.com
|
||||
|
||||
- add z axis scaling
|
||||
- major work on body module
|
||||
- republish due to tfjs 2.8.0 issues
|
||||
|
||||
### **0.9.17** 2020/12/15 mandic00@live.com
|
||||
|
||||
- added custom webgl backend
|
||||
|
||||
### **0.9.16** 2020/12/12 mandic00@live.com
|
||||
|
||||
- change default ports
|
||||
|
||||
### **0.9.15** 2020/12/11 mandic00@live.com
|
||||
|
||||
- improved caching and warmup
|
||||
- rebuild
|
||||
|
||||
### **0.9.14** 2020/12/10 mandic00@live.com
|
||||
|
||||
- conditional hand rotation
|
||||
- staggered skipframes
|
||||
- fix permissions
|
||||
|
||||
### **0.9.13** 2020/12/08 mandic00@live.com
|
||||
|
||||
- implemented face and hand boundary checks
|
||||
- embedded sample for warmup
|
||||
- switch to central logger
|
||||
|
||||
### **0.9.12** 2020/11/26 mandic00@live.com
|
||||
|
||||
- minor compatibility fixes
|
||||
|
||||
### **0.9.11** 2020/11/23 mandic00@live.com
|
||||
|
||||
- implement multi-person gestures
|
||||
- modularize pipeline models
|
||||
|
||||
### **0.9.10** 2020/11/21 mandic00@live.com
|
||||
|
||||
- changed build for optimized node & browser
|
||||
|
||||
### **0.9.9** 2020/11/21 mandic00@live.com
|
||||
|
||||
- new screenshots
|
||||
- camera exception handling
|
||||
|
||||
### **0.9.8** 2020/11/19 mandic00@live.com
|
||||
|
||||
- force f16 textures
|
||||
- bugfix embedding check
|
||||
|
||||
### **0.9.7** 2020/11/19 mandic00@live.com
|
||||
|
||||
- ui redesign
|
||||
|
||||
### **0.9.6** 2020/11/18 mandic00@live.com
|
||||
|
||||
- optimize camera resize on mobile
|
||||
- completed tfjs wrapper
|
||||
|
||||
### **0.9.5** 2020/11/17 mandic00@live.com
|
||||
|
||||
- fix serious performance bug around skipframes
|
||||
- swtich to custom tfjs bundle
|
||||
|
||||
### **0.9.4** 2020/11/17 mandic00@live.com
|
||||
|
||||
- swtich to tfjs source import
|
||||
|
||||
### **0.9.3** 2020/11/16 mandic00@live.com
|
||||
|
||||
- switched to minified build
|
||||
|
||||
### **tag: 1.2** 2020/11/15 mandic00@live.com
|
||||
### **release: 1.2** 2020/11/15 mandic00@live.com
|
||||
|
||||
- full rebuild
|
||||
|
||||
### **0.9.2** 2020/11/14 mandic00@live.com
|
||||
|
||||
- fix camera restart on resize
|
||||
|
||||
### **0.9.1** 2020/11/13 mandic00@live.com
|
||||
|
||||
- version bump
|
||||
- full rebuild
|
||||
- implemented face embedding
|
||||
- added internal benchmark tool
|
||||
|
||||
### **0.8.8** 2020/11/12 mandic00@live.com
|
||||
|
||||
- reduced bundle size
|
||||
- implemented buffered processing
|
||||
- fix for conditional model loading
|
||||
|
||||
### **0.8.7** 2020/11/11 mandic00@live.com
|
||||
|
||||
- added performance notes
|
||||
- added notes on models
|
||||
- fix bug in async ops and change imports
|
||||
- fix wiki links
|
||||
|
||||
### **0.8.6** 2020/11/09 mandic00@live.com
|
||||
|
||||
- add wasm bundle
|
||||
|
||||
### **0.8.5** 2020/11/09 mandic00@live.com
|
||||
|
||||
- reimplemented blazeface processing
|
||||
|
||||
### **0.8.4** 2020/11/09 mandic00@live.com
|
||||
|
||||
- added additional gestures
|
||||
- implemented blink detection
|
||||
- fix wasm module
|
||||
|
||||
### **0.8.3** 2020/11/08 mandic00@live.com
|
||||
|
||||
- refresh
|
||||
- optimizations
|
||||
|
||||
### **0.8.2** 2020/11/08 mandic00@live.com
|
||||
|
||||
- fix typos
|
||||
- commit
|
||||
|
||||
### **0.8.1** 2020/11/07 mandic00@live.com
|
||||
|
||||
- fix hand detection performance
|
||||
- optimized model loader
|
||||
- merge branch 'main' of https://github.com/vladmandic/human into main
|
||||
|
|
@ -222,52 +283,65 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
|
|||
- fix nms sync call
|
||||
|
||||
### **0.7.6** 2020/11/06 mandic00@live.com
|
||||
|
||||
- fixed memory leaks and updated docs
|
||||
- model tuning
|
||||
- cache invalidation improvements
|
||||
- full async operations
|
||||
|
||||
### **0.7.5** 2020/11/05 mandic00@live.com
|
||||
|
||||
- implemented dev-server
|
||||
|
||||
### **0.7.4** 2020/11/05 mandic00@live.com
|
||||
|
||||
- fix canvas size on different orientation
|
||||
- switched from es2020 to es2018 build target
|
||||
|
||||
### **0.7.3** 2020/11/05 mandic00@live.com
|
||||
|
||||
- optimized camera and mobile layout
|
||||
- fixed worker and filter compatibility
|
||||
- removed test code
|
||||
|
||||
### **0.7.2** 2020/11/04 mandic00@live.com
|
||||
|
||||
- major work on handpose model
|
||||
|
||||
### **0.7.1** 2020/11/04 mandic00@live.com
|
||||
|
||||
- changed demo build process
|
||||
|
||||
### **0.6.7** 2020/11/04 mandic00@live.com
|
||||
|
||||
- implemented simple gesture recognition
|
||||
|
||||
### **0.6.6** 2020/11/04 mandic00@live.com
|
||||
|
||||
- remove debug code
|
||||
|
||||
### **0.6.5** 2020/11/04 mandic00@live.com
|
||||
|
||||
- redo hand detection
|
||||
|
||||
### **0.6.4** 2020/11/03 mandic00@live.com
|
||||
|
||||
- added manifest
|
||||
|
||||
### **0.6.3** 2020/11/03 mandic00@live.com
|
||||
|
||||
- enhanced processing resolution
|
||||
- fix pause restart
|
||||
- complete model refactoring
|
||||
- fixed typo
|
||||
|
||||
### **0.6.2** 2020/11/02 mandic00@live.com
|
||||
|
||||
- optimized demo
|
||||
- package update
|
||||
|
||||
### **0.6.1** 2020/11/02 mandic00@live.com
|
||||
|
||||
- major performance improvements for all models
|
||||
- revert "optimized canvas handling"
|
||||
- optimized canvas handling
|
||||
|
|
@ -276,61 +350,80 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
|
|||
- added tfjs-vis to distribution
|
||||
|
||||
### **0.5.5** 2020/11/01 mandic00@live.com
|
||||
|
||||
- changed defaults
|
||||
|
||||
### **0.5.4** 2020/11/01 mandic00@live.com
|
||||
|
||||
- implemented memory profiler
|
||||
|
||||
### **0.5.3** 2020/10/30 mandic00@live.com
|
||||
|
||||
- improved debug logging
|
||||
|
||||
### **0.5.2** 2020/10/30 mandic00@live.com
|
||||
|
||||
- added wasm and webgpu backends
|
||||
|
||||
### **0.5.1** 2020/10/30 mandic00@live.com
|
||||
|
||||
- improve demo line continous draws
|
||||
|
||||
### **0.4.10** 2020/10/30 mandic00@live.com
|
||||
|
||||
- fix for seedrandom
|
||||
|
||||
### **0.4.9** 2020/10/29 mandic00@live.com
|
||||
|
||||
|
||||
### **0.4.8** 2020/10/28 mandic00@live.com
|
||||
|
||||
- revert "updated menu handler"
|
||||
- fix webpack compatibility issue
|
||||
|
||||
### **0.4.7** 2020/10/27 mandic00@live.com
|
||||
|
||||
|
||||
### **0.4.6** 2020/10/27 mandic00@live.com
|
||||
|
||||
- fix firefox compatibility bug
|
||||
|
||||
### **0.4.5** 2020/10/27 mandic00@live.com
|
||||
|
||||
|
||||
### **0.4.4** 2020/10/27 mandic00@live.com
|
||||
|
||||
- implelented input resizing
|
||||
|
||||
### **0.4.3** 2020/10/22 mandic00@live.com
|
||||
|
||||
|
||||
### **0.4.2** 2020/10/20 mandic00@live.com
|
||||
|
||||
- log initialization
|
||||
|
||||
### **0.4.1** 2020/10/19 mandic00@live.com
|
||||
|
||||
- breaking change: convert to object class
|
||||
- compatibility notes
|
||||
|
||||
### **0.3.9** 2020/10/18 mandic00@live.com
|
||||
|
||||
- implemented image filters
|
||||
- pure tensor pipeline without image converts
|
||||
- autodetect skipframes
|
||||
|
||||
### **0.3.8** 2020/10/17 mandic00@live.com
|
||||
|
||||
- new menu layout
|
||||
|
||||
### **0.3.7** 2020/10/17 mandic00@live.com
|
||||
|
||||
- added diagnostics output
|
||||
- parallelized agegender operations
|
||||
|
||||
### **0.3.6** 2020/10/17 mandic00@live.com
|
||||
|
||||
- fixed webcam initialization
|
||||
- fixed memory leaks and added scoped runs
|
||||
- modularized draw
|
||||
|
|
@ -338,17 +431,21 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
|
|||
- refactored package file layout
|
||||
|
||||
### **0.3.5** 2020/10/16 mandic00@live.com
|
||||
|
||||
- added auto-generated changelog
|
||||
|
||||
### **0.3.4** 2020/10/16 mandic00@live.com
|
||||
|
||||
- added camera selection
|
||||
- optimized blazeface anchors
|
||||
- added error handling
|
||||
|
||||
### **0.3.3** 2020/10/15 mandic00@live.com
|
||||
|
||||
- added blazeface back and front models
|
||||
|
||||
### **0.3.2** 2020/10/15 mandic00@live.com
|
||||
|
||||
- reduced web worker latency
|
||||
- added debugging and versioning
|
||||
- optimized demos and added scoped runs
|
||||
|
|
@ -356,37 +453,49 @@ Repository: **<git+https://github.com/vladmandic/human.git>**
|
|||
|
||||
### **0.3.1** 2020/10/14 mandic00@live.com
|
||||
|
||||
|
||||
### **0.2.10** 2020/10/14 mandic00@live.com
|
||||
|
||||
- added emotion backend
|
||||
- module parametrization and performance monitoring
|
||||
- implemented multi-hand support
|
||||
- fixed documentation typos
|
||||
|
||||
### **0.2.9** 2020/10/13 mandic00@live.com
|
||||
|
||||
- added node build and demo
|
||||
|
||||
### **0.2.8** 2020/10/13 mandic00@live.com
|
||||
|
||||
- added example image
|
||||
|
||||
### **0.2.7** 2020/10/13 mandic00@live.com
|
||||
|
||||
- new examples
|
||||
|
||||
### **0.2.6** 2020/10/13 mandic00@live.com
|
||||
|
||||
- enable all models by default
|
||||
|
||||
### **0.2.5** 2020/10/12 mandic00@live.com
|
||||
|
||||
- fixed memory leak
|
||||
|
||||
### **0.2.4** 2020/10/12 mandic00@live.com
|
||||
|
||||
- removed extra files
|
||||
|
||||
### **0.2.3** 2020/10/12 mandic00@live.com
|
||||
|
||||
|
||||
### **0.2.2** 2020/10/12 mandic00@live.com
|
||||
|
||||
### **tag: 1.0** 2020/10/12 mandic00@live.com
|
||||
|
||||
### **release: 1.0** 2020/10/12 mandic00@live.com
|
||||
|
||||
|
||||
### **0.2.1** 2020/10/12 mandic00@live.com
|
||||
|
||||
- added sample image
|
||||
- initial public commit
|
||||
- initial commit
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
## Configuration
|
||||
# Configuration
|
||||
|
||||
Detailed configuration options are explained below, but they are best seen in the menus present in the `demo` application:
|
||||
*note: some advanced configuration options are not exposed in the UI*
|
||||
|
|
@ -10,12 +10,14 @@ Detailed configuration options are explained below, but they are best seen in th
|
|||
<br>
|
||||
|
||||
Main configuration objects are:
|
||||
|
||||
- **config.filter**: controls image pre-processing
|
||||
- **config.face**: controls face detection
|
||||
- **config.body**: controls body pose detection
|
||||
- **config.hand**: contros hand and finger detection
|
||||
|
||||
With **config.face** having several subsections:
|
||||
|
||||
- **config.face.mesh**: controls facial mesh and landscape detection
|
||||
- **config.face.iris**: controls iris detection
|
||||
- **config.face.age**: controls age prediction
|
||||
|
|
@ -31,7 +33,6 @@ Note that user object and default configuration are merged using deep-merge, so
|
|||
|
||||
All configuration details can be changed in real-time!
|
||||
|
||||
|
||||
```js
|
||||
config = {
|
||||
backend: 'webgl', // select tfjs backend to use
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
## Credits
|
||||
# Credits
|
||||
|
||||
- Face Detection: [**MediaPipe BlazeFace**](https://drive.google.com/file/d/1f39lSzU5Oq-j_OXgS67KfN5wNsoeAZ4V/view)
|
||||
- Face Boxes: [**FaceBoxes**](https://github.com/TropComplique/FaceBoxes-tensorflow/)
|
||||
|
|
|
|||
17
Demos.md
17
Demos.md
|
|
@ -1,10 +1,11 @@
|
|||
## Demos
|
||||
# Demos
|
||||
|
||||
Demos are included in `/demo`:
|
||||
|
||||
<br>
|
||||
|
||||
**Browser**:
|
||||
|
||||
- `index.html`: Full demo using Browser with ESM module, includes selectable backends and webworkers
|
||||
|
||||
*You can run browser demo either live from git pages, by serving demo folder from your web server or use
|
||||
|
|
@ -12,7 +13,7 @@ included micro http2 server with source file monitoring and dynamic rebuild*
|
|||
|
||||
<br>
|
||||
|
||||
### Changing Demo Target
|
||||
## Changing Demo Target
|
||||
|
||||
Demo in `demo/index.html` loads `dist/demo-browser-index.js` which is built from sources in `demo`, starting with `demo/browser`
|
||||
This bundled version is needed since mobile browsers (e.g. Chrome on Android) do not support native modules loading yet
|
||||
|
|
@ -21,10 +22,10 @@ If your target is desktop, alternatively you can load `demo/browser.js` directly
|
|||
|
||||
<br>
|
||||
|
||||
### Demo Options
|
||||
## Demo Options
|
||||
|
||||
Demo implements several ways to use `Human` library,
|
||||
all configurable in `browse.js:ui` configuration object and in the UI itself:
|
||||
all configurable in `browse.js:ui` configuration object and in the UI itself:
|
||||
|
||||
- `ui.buffered`: run detection and screen refresh in a sequence or as separate buffered functions
|
||||
- `ui.bufferedFPSTarget`: when using buffered execution this target fps for screen refresh
|
||||
|
|
@ -41,18 +42,20 @@ all configurable in `browse.js:ui` configuration object and in the UI itself:
|
|||
|
||||
<br>
|
||||
|
||||
### Dev Server
|
||||
## Dev Server
|
||||
|
||||
To use dev server, you must have an SSL certificate since browsers enforce *https* for webcam access
|
||||
|
||||
You can use a provided ones self-signed certificate or configure your own
|
||||
|
||||
Self-signed test certificate was generated using:
|
||||
|
||||
```shell
|
||||
openssl req -x509 -newkey rsa:4096 -nodes -keyout https.key -out https.crt -days 365 -subj "/C=US/ST=Florida/L=Miami/O=@vladmandic"
|
||||
```
|
||||
|
||||
If you want to use your own certificate, edit `server/dev.js`:
|
||||
|
||||
```js
|
||||
const options = {
|
||||
key: fs.readFileSync('server/https.key'),
|
||||
|
|
@ -61,9 +64,11 @@ const options = {
|
|||
```
|
||||
|
||||
Once SSL certificates have been provided (defalt test certificate or your own), simply run
|
||||
|
||||
```shell
|
||||
npm run dev
|
||||
```
|
||||
|
||||
On first start, it will install all development dependencies required to rebuild `Human` library
|
||||
Dev version of the library runs without code minification for easier debugging while production version is fully minified
|
||||
|
||||
|
|
@ -113,9 +118,11 @@ found 0 vulnerabilities
|
|||
2020-12-12 10:12:55 DATA: GET/2.0 200 application/octet-stream 5502280 /models/handskeleton.bin ::ffff:192.168.0.200
|
||||
...
|
||||
```
|
||||
|
||||
<br>
|
||||
|
||||
**NodeJS**:
|
||||
|
||||
- `node.js`: Demo using NodeJS with CommonJS module
|
||||
This is a very simple demo as althought `Human` library is compatible with NodeJS execution
|
||||
and is able to load images and models from local filesystem,
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
## Face Feature Embedding and Simmilarity Compare
|
||||
# Face Feature Embedding and Simmilarity Compare
|
||||
|
||||
<br>
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
## Gesture Recognition
|
||||
# Gesture Recognition
|
||||
|
||||
Gesture recognition is done by looking up coordinates of different detected landmarks
|
||||
|
||||
|
|
@ -21,6 +21,7 @@ There are three pre-defined methods:
|
|||
- "*`<finger>` forward `<finger>` up*"
|
||||
|
||||
Example output of `result.gesture`:
|
||||
|
||||
```js
|
||||
[
|
||||
{face: "0", gesture: "facing camera"}
|
||||
|
|
@ -31,4 +32,5 @@ Example output of `result.gesture`:
|
|||
{hand: "0", gesture: "thumb forward middlefinger up"}
|
||||
]
|
||||
```
|
||||
|
||||
Where number after gesture refers to number of person that detection belongs to in scenes with multiple people.
|
||||
|
|
|
|||
9
Home.md
9
Home.md
|
|
@ -4,7 +4,7 @@
|
|||
|
||||
<br>
|
||||
|
||||
### Project pages:
|
||||
### Project pages
|
||||
|
||||
- [**Live Demo**](https://vladmandic.github.io/human/demo/index.html)
|
||||
- [**Code Repository**](https://github.com/vladmandic/human)
|
||||
|
|
@ -12,7 +12,7 @@
|
|||
- [**Issues Tracker**](https://github.com/vladmandic/human/issues)
|
||||
- [**Change Log**](https://github.com/vladmandic/human/wiki/Change-Log)
|
||||
|
||||
### Wiki pages:
|
||||
### Wiki pages
|
||||
|
||||
- [**Home**](https://github.com/vladmandic/human/wiki)
|
||||
- [**Demos**](https://github.com/vladmandic/human/wiki/Demos)
|
||||
|
|
@ -23,7 +23,7 @@
|
|||
- [**Face Embedding and Recognition**](https://github.com/vladmandic/human/wiki/Embedding)
|
||||
- [**Gesture Recognition**](https://github.com/vladmandic/human/wiki/Gesture)
|
||||
|
||||
### Additional notes:
|
||||
### Additional notes
|
||||
|
||||
- [**Notes on Backends**](https://github.com/vladmandic/human/wiki/Backends)
|
||||
- [**Development Server**](https://github.com/vladmandic/human/wiki/Development-Server)
|
||||
|
|
@ -35,7 +35,8 @@
|
|||
|
||||
<br>
|
||||
|
||||
Compatible with *Browser*, *WebWorker* and *NodeJS* execution on both Windows and Linux
|
||||
Compatible with *Browser*, *WebWorker* and *NodeJS* execution on both Windows and Linux
|
||||
|
||||
- Browser/WebWorker: Compatible with *CPU*, *WebGL*, *WASM* and *WebGPU* backends
|
||||
- NodeJS: Compatible with software *tfjs-node* and CUDA accelerated backends *tfjs-node-gpu*
|
||||
- (and maybe with React-Native as it doesn't use any DOM objects)
|
||||
|
|
|
|||
39
Install.md
39
Install.md
|
|
@ -1,19 +1,19 @@
|
|||
## Installation
|
||||
# Installation
|
||||
|
||||
<br>
|
||||
|
||||
**Important**
|
||||
**Important**
|
||||
|
||||
*The packaged version of `Human` includes `TensorFlow/JS (TFJS) 3.2.0` library which can be accessed via `human.tf`*
|
||||
|
||||
*You should NOT manually load another instance of `tfjs` unless you're using specific `.nobudle` edition of `Human,
|
||||
*You should NOT manually load another instance of `tfjs` unless you're using specific `.nobudle` edition of `Human,
|
||||
but if you do, be aware of possible version conflicts*
|
||||
|
||||
<br>
|
||||
|
||||
There are multiple ways to use `Human` library, pick one that suits you:
|
||||
|
||||
### Included
|
||||
## Included
|
||||
|
||||
- `dist/human.js`: IIFE format bundle with TFJS for Browsers
|
||||
- `dist/human.esm.js`: ESM format bundle with TFJS for Browsers
|
||||
|
|
@ -25,6 +25,7 @@ All versions include `sourcemap` *(.map)* and build `manifest` *(.json)*
|
|||
While `Human` is in pre-release mode, all bundles are non-minified
|
||||
|
||||
Defaults:
|
||||
|
||||
```json
|
||||
{
|
||||
"main": "dist/human.node.js",
|
||||
|
|
@ -35,7 +36,7 @@ Defaults:
|
|||
|
||||
<br>
|
||||
|
||||
### 1. [IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE) script
|
||||
## 1. [IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE) script
|
||||
|
||||
*Simplest way for usage within Browser*
|
||||
|
||||
|
|
@ -43,10 +44,10 @@ Simply download `dist/human.js`, include it in your `HTML` file & it's ready to
|
|||
|
||||
```html
|
||||
<script src="dist/human.js"><script>
|
||||
```
|
||||
```
|
||||
|
||||
IIFE script auto-registers global namespace `Human` within global `Window` object
|
||||
Which you can use to create instance of `human` library:
|
||||
Which you can use to create instance of `human` library:
|
||||
|
||||
```js
|
||||
const human = new Human();
|
||||
|
|
@ -56,16 +57,18 @@ This way you can also use `Human` library within embbedded `<script>` tag within
|
|||
|
||||
<br>
|
||||
|
||||
### 2. [ESM](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) module
|
||||
## 2. [ESM](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) module
|
||||
|
||||
*Recommended for usage within `Browser`*
|
||||
|
||||
#### **2.1 Using Script Module**
|
||||
### **2.1 Using Script Module**
|
||||
|
||||
You could use same syntax within your main `JS` file if it's imported with `<script type="module">`
|
||||
|
||||
```html
|
||||
<script src="./index.js" type="module">
|
||||
```
|
||||
|
||||
and then in your `index.js`
|
||||
|
||||
```js
|
||||
|
|
@ -73,15 +76,17 @@ and then in your `index.js`
|
|||
const human = new Human();
|
||||
```
|
||||
|
||||
#### **2.2 With Bundler**
|
||||
### **2.2 With Bundler**
|
||||
|
||||
If you're using bundler *(such as rollup, webpack, parcel, browserify, esbuild)* to package your client application,
|
||||
you can import ESM version of `Human` library which supports full tree shaking
|
||||
|
||||
Install with:
|
||||
|
||||
```shell
|
||||
npm install @vladmandic/human
|
||||
```
|
||||
|
||||
```js
|
||||
import Human from '@vladmandic/human'; // points to @vladmandic/human/dist/human.esm.js
|
||||
// you can also force-load specific version
|
||||
|
|
@ -92,9 +97,11 @@ Install with:
|
|||
Or if you prefer to package your version of `tfjs`, you can use `nobundle` version
|
||||
|
||||
Install with:
|
||||
|
||||
```shell
|
||||
npm install @vladmandic/human @tensorflow/tfjs
|
||||
```
|
||||
|
||||
```js
|
||||
import tf from '@tensorflow/tfjs'
|
||||
import Human from '@vladmandic/human/dist/human.esm-nobundle.js'; // same functionality as default import, but without tfjs bundled
|
||||
|
|
@ -103,7 +110,7 @@ Install with:
|
|||
|
||||
<br>
|
||||
|
||||
### 3. [NPM](https://www.npmjs.com/) module
|
||||
## 3. [NPM](https://www.npmjs.com/) module
|
||||
|
||||
*Recommended for `NodeJS` projects that will execute in the backend*
|
||||
|
||||
|
|
@ -111,10 +118,13 @@ Install with:
|
|||
Entry point are bundles in CommonJS format `dist/human.node.js` and `dist/human.node-gpu.js`
|
||||
|
||||
Install with:
|
||||
|
||||
```shell
|
||||
npm install @vladmandic/human @tensorflow/tfjs-node
|
||||
```
|
||||
|
||||
And then use with:
|
||||
|
||||
```js
|
||||
const tf = require('@tensorflow/tfjs-node'); // can also use '@tensorflow/tfjs-node-gpu' if you have environment with CUDA extensions
|
||||
const Human = require('@vladmandic/human').default; // points to @vladmandic/human/dist/human.node.js
|
||||
|
|
@ -122,20 +132,23 @@ And then use with:
|
|||
```
|
||||
|
||||
Or for CUDA accelerated NodeJS backend:
|
||||
|
||||
```shell
|
||||
npm install @vladmandic/human @tensorflow/tfjs-node-gpu
|
||||
```
|
||||
|
||||
And then use with:
|
||||
|
||||
```js
|
||||
const tf = require('@tensorflow/tfjs-node-gpu'); // can also use '@tensorflow/tfjs-node-gpu' if you have environment with CUDA extensions
|
||||
const Human = require('@vladmandic/human/dist/human.node-gpu.js').default; // points to @vladmandic/human/dist/human.node.js
|
||||
const human = new Human();
|
||||
```
|
||||
|
||||
|
||||
Since NodeJS projects load `weights` from local filesystem instead of using `http` calls, you must modify default configuration to include correct paths with `file://` prefix
|
||||
|
||||
For example:
|
||||
|
||||
```js
|
||||
const config = {
|
||||
body: { enabled: true, modelPath: 'file://models/posenet/model.json' },
|
||||
|
|
@ -144,7 +157,7 @@ const config = {
|
|||
|
||||
<br>
|
||||
|
||||
### Weights
|
||||
## Weights
|
||||
|
||||
Pretrained model weights are includes in `./models`
|
||||
Default configuration uses relative paths to you entry script pointing to `../models`
|
||||
|
|
|
|||
5
Math.md
5
Math.md
|
|
@ -1,5 +0,0 @@
|
|||
# math
|
||||
|
||||
distance = \sqrt{ \sum_{i=1}^{192} (first_i - second_i)^2 }
|
||||
confidence = 2 * (0.5 - distance)
|
||||
simmilarity = 1 - confidence
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
## List of all models included in Human library:
|
||||
# List of all models included in Human library
|
||||
|
||||
<br>
|
||||
|
||||
|
|
@ -24,7 +24,7 @@
|
|||
|
||||
*Note: All model definitions JSON files are parsed for human readability*
|
||||
|
||||
<br>
|
||||
<br>
|
||||
|
||||
## Credits
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
## Outputs
|
||||
# Outputs
|
||||
|
||||
Result of `humand.detect()` is a single object that includes data for all enabled modules and all detected objects:
|
||||
|
||||
|
|
@ -8,7 +8,9 @@ result = {
|
|||
face: // <array of detected objects>
|
||||
[
|
||||
{
|
||||
confidence, // <number>
|
||||
confidence, // <number> returns faceConfidence if exists, otherwise boxConfidence
|
||||
faceConfidence // <number> confidence in detection box after running mesh
|
||||
boxConfidence // <number> confidence in detection box before running mesh
|
||||
box, // <array [x, y, width, height]>
|
||||
rawBox, // normalized values for box, only set if returnRawData is set
|
||||
mesh, // <array of 3D points [x, y, z]> 468 base points & 10 iris points
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
## Performance
|
||||
# Performance
|
||||
|
||||
Performance will vary depending on your hardware, but also on number of resolution of input video/image, enabled modules as well as their parameters
|
||||
|
||||
|
|
@ -6,7 +6,7 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~2
|
|||
|
||||
<br>
|
||||
|
||||
### Performance per module on a **notebook** with nVidia GTX1050 GPU on a FullHD input:
|
||||
## Performance per module on a **notebook** with nVidia GTX1050 GPU on a FullHD input
|
||||
|
||||
- Enabled all: 20 FPS
|
||||
- Image filters: 80 FPS (standalone)
|
||||
|
|
@ -20,7 +20,7 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~2
|
|||
- Hand: 40 FPS (standalone)
|
||||
- Body: 50 FPS (standalone)
|
||||
|
||||
### Performance per module on a **smartphone** with Snapdragon 855 on a FullHD input:
|
||||
## Performance per module on a **smartphone** with Snapdragon 855 on a FullHD input
|
||||
|
||||
- Enabled all: 10 FPS
|
||||
- Image filters: 30 FPS (standalone)
|
||||
|
|
@ -37,4 +37,3 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~2
|
|||
<br>
|
||||
|
||||
For performance details, see output of `result.performance` object during after running inference
|
||||
|
||||
|
|
|
|||
|
|
@ -2,13 +2,14 @@
|
|||
|
||||
## Browser
|
||||
|
||||
### Desktop:
|
||||
### Desktop
|
||||
|
||||
- **Chrome**: *Supported*
|
||||
- Minimum version Chrome 60 from 2017
|
||||
- WASM: For performance reasons it is recommended to enable `WebAssembly SIMD support` in `chrome://flags`
|
||||
- **Firefox**: *Supported*
|
||||
- Minimum version Firefox 55 from 2017
|
||||
- WebGL: Lower performance due to Firefox handing of WebGL memory garbage collection
|
||||
- WebGL: Lower performance due to Firefox handing of WebGL memory garbage collection
|
||||
- WASM: For performance reasons it is recommended to enable `javascript.options.wasm.simd` in `about:config`
|
||||
- WASM: Async operations are not supported (running multiple detections or models in parallel) Due to missing threaded WASM support in Firefox
|
||||
- Web Workers: Not supported due to Firefox missing implementation for `OffscreenCanvas`
|
||||
|
|
@ -22,7 +23,8 @@
|
|||
- **IE 11** or older: *Not supported*
|
||||
- Human requires ES2018 support in browsers
|
||||
|
||||
### Mobile:
|
||||
### Mobile
|
||||
|
||||
- **Chrome**: *Supported*
|
||||
- **Firefox**: *Supported*
|
||||
- **Safari**: *Limited support*
|
||||
|
|
@ -33,5 +35,6 @@
|
|||
- Human requires ES2018 support in browsers
|
||||
|
||||
## NodeJS
|
||||
|
||||
- Minimum version 12.0
|
||||
- WIP: Face models are not supported in NodeJS as TensorFlow/JS for Node (`tfjs-node`) does not implement required function
|
||||
|
|
|
|||
|
|
@ -1,8 +1,9 @@
|
|||
## Profiling
|
||||
# Profiling
|
||||
|
||||
If `config.profile` is enabled, call to `human.profile()` will return detailed profiling data from the last detect invokation.
|
||||
|
||||
example:
|
||||
|
||||
```js
|
||||
result = {
|
||||
{age: {…}, gender: {…}, emotion: {…}}
|
||||
|
|
|
|||
5
Usage.md
5
Usage.md
|
|
@ -1,4 +1,4 @@
|
|||
## Usage
|
||||
# Usage
|
||||
|
||||
`Human` library does not require special initialization
|
||||
All configuration is done in a single JSON object and all model weights are dynamically loaded upon their first usage
|
||||
|
|
@ -45,9 +45,10 @@ Additionally, `Human` library exposes several objects and methods:
|
|||
<br>
|
||||
|
||||
Note that when using `Human` library in `NodeJS`, you must load and parse the image *before* you pass it for detection and dispose it afterwards
|
||||
Input format is `Tensor4D[1, width, height, 3]` of type `float32`
|
||||
Input format is `Tensor4D[1, width, height, 3]` of type `float32`
|
||||
|
||||
For example:
|
||||
|
||||
```js
|
||||
const imageFile = '../assets/sample1.jpg';
|
||||
const buffer = fs.readFileSync(imageFile);
|
||||
|
|
|
|||
Loading…
Reference in New Issue