diff --git a/Configuration.md b/Configuration.md
index 564e8d0..29f995e 100644
--- a/Configuration.md
+++ b/Configuration.md
@@ -25,7 +25,7 @@ With **config.face** having several subsections:
- **config.face.age**: controls age prediction
- **config.face.gender**: controls gender prediction
- **config.face.emotion**: controls emotion prediction
-- **config.face.embedding**: controls generation of face embedding data used for face simmilarity checks
+- **config.face.embedding**: controls generation of face embedding data used for face similarity checks
@@ -36,7 +36,7 @@ Note that user object and default configuration are merged using deep-merge, so
All configuration details can be changed in real-time!
```js
-config = {
+const config: Config = {
backend: 'webgl', // select tfjs backend to use
// can be 'webgl', 'wasm', 'cpu', or 'humangl' which is a custom version of webgl
// leave as empty string to continue using default backend
@@ -133,21 +133,14 @@ config = {
modelPath: '../models/iris.json',
},
- age: {
- enabled: true,
- modelPath: '../models/age.json',
+ description: {
+ enabled: true, // to improve accuracy of face embedding extraction it is
+ // recommended to enable detector.rotation and mesh.enabled
+ modelPath: '../models/faceres.json',
skipFrames: 31, // how many frames to go without re-running the detector
// only used for video inputs
},
- gender: {
- enabled: true,
- minConfidence: 0.1, // threshold for discarding a prediction
- modelPath: '../models/gender.json',
- skipFrames: 32, // how many frames to go without re-running the detector
- // only used for video inputs
- },
-
emotion: {
enabled: true,
minConfidence: 0.1, // threshold for discarding a prediction
@@ -155,9 +148,23 @@ config = {
modelPath: '../models/emotion.json',
},
+ age: {
+ enabled: false, // obsolete, replaced by description module
+ modelPath: '../models/age.json',
+ skipFrames: 31, // how many frames to go without re-running the detector
+ // only used for video inputs
+ },
+
+ gender: {
+ enabled: false, // obsolete, replaced by description module
+ minConfidence: 0.1, // threshold for discarding a prediction
+ modelPath: '../models/gender.json',
+ skipFrames: 32, // how many frames to go without re-running the detector
+ // only used for video inputs
+ },
+
embedding: {
- enabled: false, // to improve accuracy of face embedding extraction it is
- // highly recommended to enable detector.rotation and mesh.enabled
+ enabled: false, // obsolete, replaced by description module
modelPath: '../models/mobileface.json',
},
},
diff --git a/Demos.md b/Demos.md
index 64a9942..9e2bae0 100644
--- a/Demos.md
+++ b/Demos.md
@@ -55,7 +55,7 @@ It highlights functionality such as:
- Loading images
- Extracting faces from images
- Calculating face embedding descriptors
-- Finding face simmilarity and sorting them by simmilarity
+- Finding face similarity and sorting them by similarity
- Finding best face match based on a known list of faces and printing matches
diff --git a/Embedding.md b/Embedding.md
index 417ffeb..d6539bc 100644
--- a/Embedding.md
+++ b/Embedding.md
@@ -1,16 +1,14 @@
-# Face Feature Embedding and Simmilarity Compare
+# Face Feature Embedding and Similarity Compare
-## Demo
-
To see a demo of all all face embedding features, see `/demo/embedding.js`
It highlights functionality such as:
- Loading images
- Extracting faces from images
- Calculating face embedding descriptors
-- Finding face simmilarity and sorting them by simmilarity
+- Finding face similarity and sorting them by similarity
- Finding best face match based on a known list of faces and printing matches
@@ -23,7 +21,7 @@ and calculate embedding vectors for both first and second image you want to comp
To achieve quality results, it is also highly recommended to have `face.mesh` and `face.detection.rotation`
enabled as calculating feature vectors on non-quality inputs can lead to false results
-Simmilarity match above 75% is considered good match while simmilarity match above 60% is considered best-guess
+Similarity match above 65% is considered good match while similarity match above 55% is considered best-guess
For example,
@@ -33,7 +31,8 @@ const myConfig = {
enabled: true,
detector: { rotation: true, return: true },
mesh: { enabled: true },
- embedding: { enabled: true },
+ description: { enabled: true },
+ // embedding: { enabled: true }, // alternative you can use embedding module instead of description
},
};
@@ -42,9 +41,9 @@ const human = new Human(myConfig);
const firstResult = await human.detect(firstImage);
const secondResult = await human.detect(secondImage);
-const simmilarity = human.simmilarity(firstResult.face[0].embedding, secondResult.face[0].embedding);
+const similarity = human.similarity(firstResult.face[0].embedding, secondResult.face[0].embedding);
-console.log(`faces are ${100 * simmilarity}% simmilar`);
+console.log(`faces are ${100 * similarity}% simmilar`);
```
If the image or video frame have multiple faces and you want to match all of them, simply loop through all `results.face`
@@ -52,8 +51,8 @@ If the image or video frame have multiple faces and you want to match all of the
```js
for (let i = 0; i < secondResult.face.length; i++) {
const secondEmbedding = secondResult.face[i].embedding;
- const simmilarity = human.simmilarity(firstEmbedding, secondEmbedding);
- console.log(`face ${i} is ${100 * simmilarity}% simmilar`);
+ const similarity = human.similarity(firstEmbedding, secondEmbedding);
+ console.log(`face ${i} is ${100 * similarity}% simmilar`);
}
```
@@ -76,28 +75,28 @@ They can be stored as normal arrays and reused as needed
-## Face Simmilarity
+## Face Similarity
-Simmilarity function is based on general *Minkowski distance* between all points in vector
+Similarity function is based on general *Minkowski distance* between all points in vector
*[Minkowski distance](https://en.wikipedia.org/wiki/Minkowski_distance) is a nth root of sum of nth powers of distances between each point (each value in 192-member array)*
*Default is Eucliean distance which is a limited case of Minkowski distance with order of 2*
-Changing `order` can make simmilarity matching more or less sensitive (default order is 2nd order)
+Changing `order` can make similarity matching more or less sensitive (default order is 2nd order)
For example, those will produce slighly different results:
```js
- const simmilarity2ndOrder = human.simmilarity(firstEmbedding, secondEmbedding, 2);
- const simmilarity3rdOrder = human.simmilarity(firstEmbedding, secondEmbedding, 3);
+ const similarity2ndOrder = human.similarity(firstEmbedding, secondEmbedding, 2);
+ const similarity3rdOrder = human.similarity(firstEmbedding, secondEmbedding, 3);
```
-How simmilarity is calculated:
+How similarity is calculated:
```js
const distance = ((firstEmbedding.map((val, i) => (val - secondEmbedding[i])).reduce((dist, diff) => dist + (diff ** order), 0) ** (1 / order)));
```
-*Once embedding values are calculated and stored, if you want to use stored embedding values without requiring `Human` library you can use above formula to calculate simmilarity on the fly*
+*Once embedding values are calculated and stored, if you want to use stored embedding values without requiring `Human` library you can use above formula to calculate similarity on the fly*
@@ -125,12 +124,12 @@ Last parameter is optional and notes a minimal threshold for a match
```js
const best = human.match(current.embedding, db, 0)
- // return is object: { name: 'person a', simmilarity '0.99', source 'some-image-file' }
+ // return is object: { name: 'person a', similarity '0.99', source 'some-image-file' }
```
Database can be further stored in a JS or JSON file and retrieved when needed to have
a permanent database of faces that can be expanded over time to cover any number of known faces
-For example, see `/demo/embedding.js`:
+For example, see `/demo/embedding.js` and example database `/demo/faces.json`:
```js
// download db with known faces
@@ -158,6 +157,6 @@ To achieve optimal result, `Human` performs following operations on an image bef
`Human` contains a demo that enumerates number of images,
extracts all faces from them, processed them and then allows
-for a selection of any face which sorts faces by simmilarity
+for a selection of any face which sorts faces by similarity
Demo is available in `demo/embedding.html` which uses `demo/embedding.js` as JavaSript module
diff --git a/Models.md b/Models.md
index 9ae968f..53a9d3b 100644
--- a/Models.md
+++ b/Models.md
@@ -7,11 +7,9 @@ Default models in Human library are:
- **Face Detection**: MediaPipe BlazeFace-Back
- **Face Mesh**: MediaPipe FaceMesh
- **Face Iris Analysis**: MediaPipe Iris
+- **Face Description**: HSE FaceRes
- **Emotion Detection**: Oarriaga Emotion
-- **Gender Detection**: Oarriaga Gender
-- **Age Detection**: SSR-Net Age IMDB
- **Body Analysis**: PoseNet
-- **Face Embedding**: BecauseofAI MobileFace Embedding
- **Object Detection**: NanoDet
## Notes
@@ -24,9 +22,11 @@ Default models in Human library are:
But if conditions are met, it returns far more details (39 vs 17 keypoints) and is far more accurate
Furthermore, it returns 3D approximation of each point instead of 2D
-
+**Face description** can be switched from default combined model `FaceRes` to individual models
-**Gender detection** can be switched from the default model to `SSR-Net` trained on IMDB dataset
+- `Gender Detection`: Oarriaga Gender
+- `Age Detection`: SSR-Net Age IMDB
+- `Face Embedding`: BecauseofAI MobileFace Embedding
@@ -52,6 +52,7 @@ Default models in Human library are:
| BecauseofAI MobileFace | 33K | mobileface.json | 2.1M | mobileface.bin | 75 |
| FaceBoxes | 212K | faceboxes.json | 2.0M | faceboxes.bin | N/A |
| NanoDet | 255K | nanodet.json | 1.9M | nanodet.bin | 524 |
+| FaceRes | 70K | faceres.json | 6.7M | faceres.bin | 524 |
@@ -64,6 +65,7 @@ Default models in Human library are:
- Face Detection: [**MediaPipe BlazeFace**](https://drive.google.com/file/d/1f39lSzU5Oq-j_OXgS67KfN5wNsoeAZ4V/view)
- Facial Spacial Geometry: [**MediaPipe FaceMesh**](https://drive.google.com/file/d/1VFC_wIpw4O7xBOiTgUldl79d9LA-LsnA/view)
- Eye Iris Details: [**MediaPipe Iris**](https://drive.google.com/file/d/1bsWbokp9AklH2ANjCfmjqEzzxO1CNbMu/view)
+- Face Description: [**HSE-FaceRes**](https://github.com/HSE-asavchenko/HSE_FaceRec_tf)
- Hand Detection & Skeleton: [**MediaPipe HandPose**](https://drive.google.com/file/d/1sv4sSb9BSNVZhLzxXJ0jBv9DqD-4jnAz/view)
- Body Pose Detection: [**BlazePose**](https://drive.google.com/file/d/10IU-DRP2ioSNjKFdiGbmmQX81xAYj88s/view)
- Body Pose Detection: [**PoseNet**](https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5)
diff --git a/Outputs.md b/Outputs.md
index 5e8b564..8131b0c 100644
--- a/Outputs.md
+++ b/Outputs.md
@@ -21,7 +21,7 @@ result = {
iris, // relative distance of iris to camera, multiple by focal lenght to get actual distance
age, // estimated age
gender, // 'male', 'female'
- embedding, // [float] vector of 192 values used for face simmilarity compare
+ embedding, // [float] vector of 192 values used for face similarity compare
angle: // 3d face rotation values in radians in range of -pi/2 to pi/2 which is -90 to +90 degrees
{
roll, // roll is face lean left/right, value of 0 means center
diff --git a/Usage.md b/Usage.md
index 68fbe83..e5c330b 100644
--- a/Usage.md
+++ b/Usage.md
@@ -39,7 +39,7 @@ Additionally, `Human` library exposes several objects and methods:
human.image(image, config?) // runs image processing without detection and returns canvas
human.warmup(config, image? // warms up human library for faster initial execution after loading
// if image is not provided, it will generate internal sample
- human.simmilarity(embedding1, embedding2) // runs simmilarity calculation between two provided embedding vectors
+ human.similarity(embedding1, embedding2) // runs similarity calculation between two provided embedding vectors
// vectors for source and target must be previously detected using
// face.embedding module
human.enhance(face) // returns enhanced tensor of a previously detected face that can be used for visualizations
@@ -50,7 +50,7 @@ For details, see [embedding documentation](https://github.com/vladmandic/human/w
```js
- human.simmilarity(embedding1, embedding2) // runs simmilarity calculation between two provided embedding vectors
+ human.similarity(embedding1, embedding2) // runs similarity calculation between two provided embedding vectors
// vectors for source and target must be previously detected using
// face.embedding module
human.match(embedding, db, threshold) // finds best match for current face in a provided list of faces