mirror of https://github.com/vladmandic/human
initial
Vladimir Mandic
parent
ecd2eb1730
commit
26e765023c
|
|
@ -0,0 +1,22 @@
|
|||
## Build Process
|
||||
|
||||
If you want to modify the library and perform a full rebuild:
|
||||
|
||||
*clone repository, install dependencies, check for errors and run full rebuild from which creates bundles from `/src` into `/dist`:*
|
||||
|
||||
```shell
|
||||
git clone https://github.com/vladmandic/human
|
||||
cd human
|
||||
npm install # installs all project dependencies
|
||||
npm run lint
|
||||
npm run build
|
||||
```
|
||||
|
||||
This will rebuild library itself (all variations) as well as demo
|
||||
|
||||
Project is written in pure `JavaScript` [ECMAScript version 2020](https://www.ecma-international.org/ecma-262/11.0/index.html)
|
||||
Build target is `JavaScript` **EMCAScript version 2018**
|
||||
|
||||
Only project depdendency is [@tensorflow/tfjs](https://github.com/tensorflow/tfjs)
|
||||
Development dependencies are [eslint](https://github.com/eslint) used for code linting and [esbuild](https://github.com/evanw/esbuild) used for IIFE and ESM script bundling
|
||||
|
||||
|
|
@ -0,0 +1,170 @@
|
|||
|
||||
# @vladmandic/human
|
||||
|
||||
Version: **0.7.3**
|
||||
Description: **human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition**
|
||||
|
||||
Author: **Vladimir Mandic <mandic00@live.com>**
|
||||
License: **MIT** </LICENSE>
|
||||
Repository: **<git+https://github.com/vladmandic/human.git>**
|
||||
|
||||
## Changelog
|
||||
|
||||
### **HEAD -> main** 2020/11/05 mandic00@live.com
|
||||
|
||||
### **0.7.3** 2020/11/05 mandic00@live.com
|
||||
- optimized camera and mobile layout
|
||||
- fixed worker and filter compatibility
|
||||
- removed test code
|
||||
|
||||
### **0.7.2** 2020/11/04 mandic00@live.com
|
||||
- major work on handpose model
|
||||
|
||||
### **0.7.1** 2020/11/04 mandic00@live.com
|
||||
- changed demo build process
|
||||
|
||||
### **0.6.7** 2020/11/04 mandic00@live.com
|
||||
- implemented simple gesture recognition
|
||||
|
||||
### **0.6.6** 2020/11/04 mandic00@live.com
|
||||
- remove debug code
|
||||
|
||||
### **0.6.5** 2020/11/04 mandic00@live.com
|
||||
- redo hand detection
|
||||
|
||||
### **0.6.4** 2020/11/03 mandic00@live.com
|
||||
- added manifest
|
||||
|
||||
### **0.6.3** 2020/11/03 mandic00@live.com
|
||||
- enhanced processing resolution
|
||||
- fix pause restart
|
||||
- complete model refactoring
|
||||
- fixed typo
|
||||
|
||||
### **0.6.2** 2020/11/02 mandic00@live.com
|
||||
- optimized demo
|
||||
- package update
|
||||
|
||||
### **0.6.1** 2020/11/02 mandic00@live.com
|
||||
- major performance improvements for all models
|
||||
- revert "optimized canvas handling"
|
||||
- optimized canvas handling
|
||||
- minor optimization to imagefx
|
||||
- fix demo image sample
|
||||
- added tfjs-vis to distribution
|
||||
|
||||
### **0.5.5** 2020/11/01 mandic00@live.com
|
||||
- changed defaults
|
||||
|
||||
### **0.5.4** 2020/11/01 mandic00@live.com
|
||||
- implemented memory profiler
|
||||
|
||||
### **0.5.3** 2020/10/30 mandic00@live.com
|
||||
- improved debug logging
|
||||
|
||||
### **0.5.2** 2020/10/30 mandic00@live.com
|
||||
- added wasm and webgpu backends
|
||||
|
||||
### **0.5.1** 2020/10/30 mandic00@live.com
|
||||
- improve demo line continous draws
|
||||
|
||||
### **0.4.10** 2020/10/30 mandic00@live.com
|
||||
- fix for seedrandom
|
||||
|
||||
### **0.4.9** 2020/10/29 mandic00@live.com
|
||||
|
||||
### **0.4.8** 2020/10/28 mandic00@live.com
|
||||
- revert "updated menu handler"
|
||||
- fix webpack compatibility issue
|
||||
|
||||
### **0.4.7** 2020/10/27 mandic00@live.com
|
||||
|
||||
### **0.4.6** 2020/10/27 mandic00@live.com
|
||||
- fix firefox compatibility bug
|
||||
|
||||
### **0.4.5** 2020/10/27 mandic00@live.com
|
||||
|
||||
### **0.4.4** 2020/10/27 mandic00@live.com
|
||||
- implelented input resizing
|
||||
|
||||
### **0.4.3** 2020/10/22 mandic00@live.com
|
||||
|
||||
### **0.4.2** 2020/10/20 mandic00@live.com
|
||||
- log initialization
|
||||
|
||||
### **0.4.1** 2020/10/19 mandic00@live.com
|
||||
- breaking change: convert to object class
|
||||
- compatibility notes
|
||||
|
||||
### **0.3.9** 2020/10/18 mandic00@live.com
|
||||
- implemented image filters
|
||||
- pure tensor pipeline without image converts
|
||||
- autodetect skipframes
|
||||
|
||||
### **0.3.8** 2020/10/17 mandic00@live.com
|
||||
- new menu layout
|
||||
|
||||
### **0.3.7** 2020/10/17 mandic00@live.com
|
||||
- added diagnostics output
|
||||
- parallelized agegender operations
|
||||
|
||||
### **0.3.6** 2020/10/17 mandic00@live.com
|
||||
- fixed webcam initialization
|
||||
- fixed memory leaks and added scoped runs
|
||||
- modularized draw
|
||||
- added state handling
|
||||
- refactored package file layout
|
||||
|
||||
### **0.3.5** 2020/10/16 mandic00@live.com
|
||||
- added auto-generated changelog
|
||||
|
||||
### **0.3.4** 2020/10/16 mandic00@live.com
|
||||
- added camera selection
|
||||
- optimized blazeface anchors
|
||||
- added error handling
|
||||
|
||||
### **0.3.3** 2020/10/15 mandic00@live.com
|
||||
- added blazeface back and front models
|
||||
|
||||
### **0.3.2** 2020/10/15 mandic00@live.com
|
||||
- reduced web worker latency
|
||||
- added debugging and versioning
|
||||
- optimized demos and added scoped runs
|
||||
- added multi backend support
|
||||
|
||||
### **0.3.1** 2020/10/14 mandic00@live.com
|
||||
|
||||
### **0.2.10** 2020/10/14 mandic00@live.com
|
||||
- added emotion backend
|
||||
- module parametrization and performance monitoring
|
||||
- implemented multi-hand support
|
||||
- fixed documentation typos
|
||||
|
||||
### **0.2.9** 2020/10/13 mandic00@live.com
|
||||
- added node build and demo
|
||||
|
||||
### **0.2.8** 2020/10/13 mandic00@live.com
|
||||
- added example image
|
||||
|
||||
### **0.2.7** 2020/10/13 mandic00@live.com
|
||||
- new examples
|
||||
|
||||
### **0.2.6** 2020/10/13 mandic00@live.com
|
||||
- enable all models by default
|
||||
|
||||
### **0.2.5** 2020/10/12 mandic00@live.com
|
||||
- fixed memory leak
|
||||
|
||||
### **0.2.4** 2020/10/12 mandic00@live.com
|
||||
- removed extra files
|
||||
|
||||
### **0.2.3** 2020/10/12 mandic00@live.com
|
||||
|
||||
### **0.2.2** 2020/10/12 mandic00@live.com
|
||||
|
||||
### **tag: 1.0** 2020/10/12 mandic00@live.com
|
||||
|
||||
### **0.2.1** 2020/10/12 mandic00@live.com
|
||||
- added sample image
|
||||
- initial public commit
|
||||
- initial commit
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
## Credits
|
||||
|
||||
- Face Detection: [**MediaPipe BlazeFace**](https://drive.google.com/file/d/1f39lSzU5Oq-j_OXgS67KfN5wNsoeAZ4V/view)
|
||||
- Facial Spacial Geometry: [**MediaPipe FaceMesh**](https://drive.google.com/file/d/1VFC_wIpw4O7xBOiTgUldl79d9LA-LsnA/view)
|
||||
- Eye Iris Details: [**MediaPipe Iris**](https://drive.google.com/file/d/1bsWbokp9AklH2ANjCfmjqEzzxO1CNbMu/view)
|
||||
- Hand Detection & Skeleton: [**MediaPipe HandPose**](https://drive.google.com/file/d/1sv4sSb9BSNVZhLzxXJ0jBv9DqD-4jnAz/view)
|
||||
- Body Pose Detection: [**PoseNet**](https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5)
|
||||
- Age & Gender Prediction: [**SSR-Net**](https://github.com/shamangary/SSR-Net)
|
||||
- Emotion Prediction: [**Oarriaga**](https://github.com/oarriaga/face_classification)
|
||||
- Image Filters: [**WebGLImageFilter**](https://github.com/phoboslab/WebGLImageFilter)
|
||||
- Pinto Model Zoo: [**Pinto**](https://github.com/PINTO0309/PINTO_model_zoo)
|
||||
|
||||
*Included models are included under license inherited from the original model source*
|
||||
*Model code has substantially changed from source that it is considered a derivative work and not simple re-publishing*
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
## Performance
|
||||
|
||||
Performance will vary depending on your hardware, but also on number of resolution of input video/image, enabled modules as well as their parameters
|
||||
|
||||
For example, it can perform multiple face detections at 60+ FPS, but drops to ~15 FPS on a medium complex images if all modules are enabled
|
||||
|
||||
### Performance per module on a **notebook** with nVidia GTX1050 GPU on a FullHD input:
|
||||
|
||||
- Enabled all: 15 FPS
|
||||
- Image filters: 80 FPS (standalone)
|
||||
- Gesture: 80 FPS (standalone)
|
||||
- Face Detect: 80 FPS (standalone)
|
||||
- Face Geometry: 30 FPS (includes face detect)
|
||||
- Face Iris: 30 FPS (includes face detect and face geometry)
|
||||
- Age: 60 FPS (includes face detect)
|
||||
- Gender: 60 FPS (includes face detect)
|
||||
- Emotion: 60 FPS (includes face detect)
|
||||
- Hand: 40 FPS (standalone)
|
||||
- Body: 50 FPS (standalone)
|
||||
|
||||
### Performance per module on a **smartphone** with Snapdragon 855 on a FullHD input:
|
||||
|
||||
- Enabled all: 5 FPS
|
||||
- Image filters: 30 FPS (standalone)
|
||||
- Gesture: 30 FPS (standalone)
|
||||
- Face Detect: 20 FPS (standalone)
|
||||
- Face Geometry: 10 FPS (includes face detect)
|
||||
- Face Iris: 5 FPS (includes face detect and face geometry)
|
||||
- Age: 20 FPS (includes face detect)
|
||||
- Gender: 20 FPS (includes face detect)
|
||||
- Emotion: 20 FPS (includes face detect)
|
||||
- Hand: 40 FPS (standalone)
|
||||
- Body: 10 FPS (standalone)
|
||||
|
||||
For performance details, see output of `result.performance` object during after running inference
|
||||
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
## Profiling
|
||||
|
||||
If `config.profile` is enabled, call to `human.profile()` will return detailed profiling data from the last detect invokation.
|
||||
|
||||
example:
|
||||
```js
|
||||
result = {
|
||||
{age: {…}, gender: {…}, emotion: {…}}
|
||||
age:
|
||||
timeKernelOps: 53.78892800000002
|
||||
newBytes: 4
|
||||
newTensors: 1
|
||||
numKernelOps: 341
|
||||
peakBytes: 46033948
|
||||
largestKernelOps: Array(5)
|
||||
0: {name: "Reshape", bytesAdded: 107648, totalBytesSnapshot: 46033948, tensorsAdded: 1, totalTensorsSnapshot: 1149, …}
|
||||
1: {name: "Reshape", bytesAdded: 0, totalBytesSnapshot: 45818652, tensorsAdded: 1, totalTensorsSnapshot: 1147, …}
|
||||
2: {name: "Reshape", bytesAdded: 0, totalBytesSnapshot: 45633996, tensorsAdded: 1, totalTensorsSnapshot: 1148, …}
|
||||
3: {name: "Reshape", bytesAdded: 0, totalBytesSnapshot: 45389376, tensorsAdded: 1, totalTensorsSnapshot: 1154, …}
|
||||
4: {name: "Reshape", bytesAdded: 53824, totalBytesSnapshot: 45381776, tensorsAdded: 1, totalTensorsSnapshot: 1155, …}
|
||||
slowestKernelOps: Array(5)
|
||||
0: {name: "_FusedMatMul", bytesAdded: 12, totalBytesSnapshot: 44802280, tensorsAdded: 1, totalTensorsSnapshot: 1156, …}
|
||||
1: {name: "_FusedMatMul", bytesAdded: 4, totalBytesSnapshot: 44727564, tensorsAdded: 1, totalTensorsSnapshot: 1152, …}
|
||||
2: {name: "_FusedMatMul", bytesAdded: 12, totalBytesSnapshot: 44789100, tensorsAdded: 1, totalTensorsSnapshot: 1157, …}
|
||||
3: {name: "Add", bytesAdded: 4, totalBytesSnapshot: 44788748, tensorsAdded: 1, totalTensorsSnapshot: 1158, …}
|
||||
4: {name: "Add", bytesAdded: 4, totalBytesSnapshot: 44788748, tensorsAdded: 1, totalTensorsSnapshot: 1158, …}
|
||||
}
|
||||
```
|
||||
Loading…
Reference in New Issue