From 79cd119c0c5714324a0dae82477ce36e1d5f40a1 Mon Sep 17 00:00:00 2001 From: Vladimir Mandic Date: Sat, 14 Nov 2020 07:01:34 -0500 Subject: [PATCH] detailed model info --- Home.md | 3 +-- Models.md | 51 ++++++++++++++++++++++++++++++++++---------------- Performance.md | 8 ++++---- 3 files changed, 40 insertions(+), 22 deletions(-) diff --git a/Home.md b/Home.md index 1a47c2b..2461c98 100644 --- a/Home.md +++ b/Home.md @@ -28,10 +28,9 @@ - [**Notes on Backends**](https://github.com/vladmandic/human/wiki/Backends) - [**Development Server**](https://github.com/vladmandic/human/wiki/Development-Server) - [**Build Process**](https://github.com/vladmandic/human/wiki/Build-Process) -- [**List of Models**](https://github.com/vladmandic/human/wiki/Models) - [**Performance Notes**](https://github.com/vladmandic/human/wiki/Performance) - [**Performance Profiling**](https://github.com/vladmandic/human/wiki/Profiling) -- [**Credits**](https://github.com/vladmandic/human/wiki/Credits) +- [**List of Models && Credits**](https://github.com/vladmandic/human/wiki/Models)
diff --git a/Models.md b/Models.md index ed1b132..8e1a8cd 100644 --- a/Models.md +++ b/Models.md @@ -2,23 +2,42 @@
-| Model Definition | Definition Size | Weights Size | Weights Name | -| ---------------- | --------------- | ------------ | ------------ | -| 51K | blazeface-front.json | 393K | blazeface-front.bin | -| 78K | blazeface-back.json | 527K | blazeface-back.bin | -| 88K | facemesh.json | 2.9M | facemesh.bin | -| 120K | iris.json | 2.5M | iris.bin | -| 30K | emotion-mini.json | 220K | emotion-mini.bin | -| 18K | emotion-large.json | 802K | emotion-large.bin | -| 93K | age-ssrnet-imdb.json | 158K | age-ssrnet-imdb.bin | -| 93K | age-ssrnet-wiki.json | 158K | age-ssrnet-wiki.bin | -| 92K | gender-ssrnet-imdb.json | 158K | gender-ssrnet-imdb.bin | -| 92K | gender-ssrnet-wiki.json | 158K | gender-ssrnet-wiki.bin | -| 30K | gender.json | 198K | gender.bin | -| 47K | posenet.json | 4.8M | posenet.bin | -| 126K | handdetect.json | 6.8M | handdetect.bin | -| 127K | handskeleton.json | 5.3M | handskeleton.bin | +| Model Name | Model Definition Size | Model Definition | Weights Size | Weights Name | Num Tensors | +| ---------- | --------------------- | ---------------- | ------------ | ------------ | ----------- | +| MediaPipe MediaPipe BlazeFace (Front) | 51K | blazeface-front.json | 393K | blazeface-front.bin | 73 | +| MediaPipe BlazeFace (Back) | 78K | blazeface-back.json | 527K | blazeface-back.bin | 112 | +| MediaPipe FaceMesh | 88K | facemesh.json | 2.9M | facemesh.bin | 118 | +| MediaPipe Iris | 120K | iris.json | 2.5M | iris.bin | 191 | +| Oarriaga Emotion (Mini) | 30K | emotion-mini.json | 220K | emotion-mini.bin | 39 | +| Oarriaga Emotion (Large) | 18K | emotion-large.json | 802K | emotion-large.bin | 23 | +| SSR-Net Age (IMDB) | 93K | age-ssrnet-imdb.json | 158K | age-ssrnet-imdb.bin | 158 | +| SSR-Net Age (Wiki) | 93K | age-ssrnet-wiki.json | 158K | age-ssrnet-wiki.bin | 158 | +| SSR-Net Gender (IMDB) | 92K | gender-ssrnet-imdb.json | 158K | gender-ssrnet-imdb.bin | 157 | +| SSR-Net Gender (Wiki) | 92K | gender-ssrnet-wiki.json | 158K | gender-ssrnet-wiki.bin | 157 | +| Oarriaga Gender | 30K | gender.json | 198K | gender.bin | 39 | +| PoseNet | 47K | posenet.json | 4.8M | posenet.bin | 62 | +| MediaPipe HandPose (HandDetect) | 126K | handdetect.json | 6.8M | handdetect.bin | 152 | +| MediaPipe HandPose (HandSkeleton) | 127K | handskeleton.json | 5.3M | handskeleton.bin | 145 | +| Sirius-AI MobileFaceNet | 125K | mobilefacenet.json | 5.0M | mobilefacenet.bin | 139 |
*Note: All model definitions JSON files are parsed for human readability* + +
+ +## Credits + +- Face Detection: [**MediaPipe BlazeFace**](https://drive.google.com/file/d/1f39lSzU5Oq-j_OXgS67KfN5wNsoeAZ4V/view) +- Facial Spacial Geometry: [**MediaPipe FaceMesh**](https://drive.google.com/file/d/1VFC_wIpw4O7xBOiTgUldl79d9LA-LsnA/view) +- Eye Iris Details: [**MediaPipe Iris**](https://drive.google.com/file/d/1bsWbokp9AklH2ANjCfmjqEzzxO1CNbMu/view) +- Hand Detection & Skeleton: [**MediaPipe HandPose**](https://drive.google.com/file/d/1sv4sSb9BSNVZhLzxXJ0jBv9DqD-4jnAz/view) +- Body Pose Detection: [**PoseNet**](https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5) +- Age & Gender Prediction: [**SSR-Net**](https://github.com/shamangary/SSR-Net) +- Emotion Prediction: [**Oarriaga**](https://github.com/oarriaga/face_classification) +- Face Embedding: [**Sirius-AI MobileFaceNet**](https://github.com/sirius-ai/MobileFaceNet_TF) +- Image Filters: [**WebGLImageFilter**](https://github.com/phoboslab/WebGLImageFilter) +- Pinto Model Zoo: [**Pinto**](https://github.com/PINTO0309/PINTO_model_zoo) + +*Included models are included under license inherited from the original model source* +*Model code has substantially changed from source that it is considered a derivative work and not simple re-publishing* diff --git a/Performance.md b/Performance.md index b0667ff..1dea97c 100644 --- a/Performance.md +++ b/Performance.md @@ -2,13 +2,13 @@ Performance will vary depending on your hardware, but also on number of resolution of input video/image, enabled modules as well as their parameters -For example, it can perform multiple face detections at 60+ FPS, but drops to ~15 FPS on a medium complex images if all modules are enabled +For example, it can perform multiple face detections at 60+ FPS, but drops to ~20 FPS on a medium complex images if all modules are enabled
### Performance per module on a **notebook** with nVidia GTX1050 GPU on a FullHD input: -- Enabled all: 15 FPS +- Enabled all: 20 FPS - Image filters: 80 FPS (standalone) - Gesture: 80 FPS (standalone) - Face Detect: 80 FPS (standalone) @@ -22,7 +22,7 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~1 ### Performance per module on a **smartphone** with Snapdragon 855 on a FullHD input: -- Enabled all: 5 FPS +- Enabled all: 10 FPS - Image filters: 30 FPS (standalone) - Gesture: 30 FPS (standalone) - Face Detect: 20 FPS (standalone) @@ -32,7 +32,7 @@ For example, it can perform multiple face detections at 60+ FPS, but drops to ~1 - Gender: 20 FPS (includes face detect) - Emotion: 20 FPS (includes face detect) - Hand: 40 FPS (standalone) -- Body: 10 FPS (standalone) +- Body: 15 FPS (standalone)