ambiser
/
human
mirror of https://github.com/vladmandic/human
9
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

npm default install should be prod only

pull/280/head
Vladimir Mandic 2022-07-07 12:11:05 +02:00
parent 1837f5bd6c
commit 4f69e85e1f
14 changed files with 792 additions and 788 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.npmrc
View File

@ -1,2 +1,3 @@
force=true force=true
legacy-peer-deps=true legacy-peer-deps=true
production=true

41
CHANGELOG.md
View File

@ -9,8 +9,9 @@
## Changelog ## Changelog
### **HEAD -> main** 2022/07/02 mandic00@live.com ### **HEAD -> main** 2022/07/05 mandic00@live.com
- add getmodelstats method
- rebuild - rebuild
### **release: 2.8.1** 2022/06/08 mandic00@live.com ### **release: 2.8.1** 2022/06/08 mandic00@live.com
@ -437,7 +438,7 @@
- remove efficientpose - remove efficientpose
- major version rebuild - major version rebuild
### **1.6.1** 2021/04/22 mandic00@live.com ### **1.6.1** 2021/04/23 mandic00@live.com
- add npmrc - add npmrc
- added filter.flip feature - added filter.flip feature
@ -576,7 +577,7 @@
- add typedocs and types - add typedocs and types
- strong typings - strong typings
### **1.1.2** 2021/03/12 mandic00@live.com ### **1.1.2** 2021/03/13 mandic00@live.com
- distance based on minkowski space and limited euclidean space - distance based on minkowski space and limited euclidean space
- guard against invalid input images - guard against invalid input images
@ -585,7 +586,7 @@
- switched face embedding to mobileface - switched face embedding to mobileface
### **1.0.4** 2021/03/11 mandic00@live.com ### **1.0.4** 2021/03/12 mandic00@live.com
- add face return tensor - add face return tensor
- add test for face descriptors - add test for face descriptors
@ -600,7 +601,7 @@
- enhanced age, gender, emotion detection - enhanced age, gender, emotion detection
- full rebuild - full rebuild
### **1.0.2** 2021/03/09 mandic00@live.com ### **1.0.2** 2021/03/10 mandic00@live.com
- remove blazeface-front, blazepose-upper, faceboxes - remove blazeface-front, blazepose-upper, faceboxes
- remove blazeface-front and faceboxes - remove blazeface-front and faceboxes
@ -710,7 +711,7 @@
- variable name changes, setting .rawcoords only if necessary - variable name changes, setting .rawcoords only if necessary
- option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream - option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream
### **0.9.18** 2020/12/16 mandic00@live.com ### **0.9.18** 2020/12/17 mandic00@live.com
- add z axis scaling - add z axis scaling
- major work on body module - major work on body module
@ -744,7 +745,7 @@
- minor compatibility fixes - minor compatibility fixes
### **0.9.11** 2020/11/23 mandic00@live.com ### **0.9.11** 2020/11/24 mandic00@live.com
- implement multi-person gestures - implement multi-person gestures
- modularize pipeline models - modularize pipeline models
@ -781,7 +782,7 @@
- swtich to tfjs source import - swtich to tfjs source import
### **0.9.3** 2020/11/16 mandic00@live.com ### **0.9.3** 2020/11/17 mandic00@live.com
- switched to minified build - switched to minified build
- web worker fixes - web worker fixes
@ -811,7 +812,7 @@
- fix bug in async ops and change imports - fix bug in async ops and change imports
- fix wiki links - fix wiki links
### **0.8.6** 2020/11/09 mandic00@live.com ### **0.8.6** 2020/11/10 mandic00@live.com
- add wasm bundle - add wasm bundle
@ -851,7 +852,7 @@
- cache invalidation improvements - cache invalidation improvements
- full async operations - full async operations
### **0.7.5** 2020/11/05 mandic00@live.com ### **0.7.5** 2020/11/06 mandic00@live.com
- implemented dev-server - implemented dev-server
@ -896,12 +897,12 @@
- complete model refactoring - complete model refactoring
- fixed typo - fixed typo
### **0.6.2** 2020/11/02 mandic00@live.com ### **0.6.2** 2020/11/03 mandic00@live.com
- optimized demo - optimized demo
- package update - package update
### **0.6.1** 2020/11/02 mandic00@live.com ### **0.6.1** 2020/11/03 mandic00@live.com
- major performance improvements for all models - major performance improvements for all models
- revert "optimized canvas handling" - revert "optimized canvas handling"
@ -955,7 +956,7 @@
- implelented input resizing - implelented input resizing
### **0.4.3** 2020/10/22 mandic00@live.com ### **0.4.3** 2020/10/23 mandic00@live.com
### **0.4.2** 2020/10/20 mandic00@live.com ### **0.4.2** 2020/10/20 mandic00@live.com
@ -973,7 +974,7 @@
- pure tensor pipeline without image converts - pure tensor pipeline without image converts
- autodetect skipframes - autodetect skipframes
### **0.3.8** 2020/10/17 mandic00@live.com ### **0.3.8** 2020/10/18 mandic00@live.com
- new menu layout - new menu layout
@ -1000,28 +1001,28 @@
- optimized blazeface anchors - optimized blazeface anchors
- added error handling - added error handling
### **0.3.3** 2020/10/15 mandic00@live.com ### **0.3.3** 2020/10/16 mandic00@live.com
- added blazeface back and front models - added blazeface back and front models
### **0.3.2** 2020/10/15 mandic00@live.com ### **0.3.2** 2020/10/16 mandic00@live.com
- reduced web worker latency - reduced web worker latency
- added debugging and versioning - added debugging and versioning
- optimized demos and added scoped runs - optimized demos and added scoped runs
- added multi backend support - added multi backend support
### **0.3.1** 2020/10/14 mandic00@live.com ### **0.3.1** 2020/10/15 mandic00@live.com
### **0.2.10** 2020/10/14 mandic00@live.com ### **0.2.10** 2020/10/15 mandic00@live.com
- added emotion backend - added emotion backend
- module parametrization and performance monitoring - module parametrization and performance monitoring
- implemented multi-hand support - implemented multi-hand support
- fixed documentation typos - fixed documentation typos
### **0.2.9** 2020/10/13 mandic00@live.com ### **0.2.9** 2020/10/14 mandic00@live.com
- added node build and demo - added node build and demo
@ -1036,7 +1037,7 @@
- enable all models by default - enable all models by default
### **0.2.5** 2020/10/12 mandic00@live.com ### **0.2.5** 2020/10/13 mandic00@live.com
- fixed memory leak - fixed memory leak

8
dist/human.esm-nobundle.js vendored
View File

File diff suppressed because one or more lines are too long

4
dist/human.esm-nobundle.js.map vendored
View File

File diff suppressed because one or more lines are too long

9
dist/human.esm.js vendored
View File

@ -44347,11 +44347,8 @@ async function predict10(input, config3) {
} else { } else {
const results = model11.execute(face4.tensor); const results = model11.execute(face4.tensor);
const confidenceT = results.find((t) => t.shape[t.shape.length - 1] === 1); const confidenceT = results.find((t) => t.shape[t.shape.length - 1] === 1);
const meshT = results.find((t) => t.shape[t.shape.length - 1] === 1404);
const faceConfidence = await confidenceT.data(); const faceConfidence = await confidenceT.data();
face4.faceScore = Math.round(100 * faceConfidence[0]) / 100; face4.faceScore = Math.round(100 * faceConfidence[0]) / 100;
const coordsReshaped = U(meshT, [-1, 3]);
let rawCoords = await coordsReshaped.array();
if (face4.faceScore < (((_g2 = config3.face.detector) == null ? void 0 : _g2.minConfidence) || 1)) { if (face4.faceScore < (((_g2 = config3.face.detector) == null ? void 0 : _g2.minConfidence) || 1)) {
box.confidence = face4.faceScore; box.confidence = face4.faceScore;
if ((_h = config3.face.mesh) == null ? void 0 : _h.keepInvalid) { if ((_h = config3.face.mesh) == null ? void 0 : _h.keepInvalid) {
@ -44368,6 +44365,10 @@ async function predict10(input, config3) {
} }
} }
} else { } else {
const meshT = results.find((t) => t.shape[t.shape.length - 1] === 1404);
const coordsReshaped = U(meshT, [-1, 3]);
let rawCoords = await coordsReshaped.array();
De(coordsReshaped);
if ((_i2 = config3.face.attention) == null ? void 0 : _i2.enabled) { if ((_i2 = config3.face.attention) == null ? void 0 : _i2.enabled) {
rawCoords = await augment(rawCoords, results); rawCoords = await augment(rawCoords, results);
} else if ((_j2 = config3.face.iris) == null ? void 0 : _j2.enabled) { } else if ((_j2 = config3.face.iris) == null ? void 0 : _j2.enabled) {
@ -44383,7 +44384,7 @@ async function predict10(input, config3) {
face4.boxRaw = getRawBox(calculatedBox, input); face4.boxRaw = getRawBox(calculatedBox, input);
newCache.push(calculatedBox); newCache.push(calculatedBox);
} }
De([...results, coordsReshaped]); De(results);
} }
if (face4.score > (((_k2 = config3.face.detector) == null ? void 0 : _k2.minConfidence) || 1)) if (face4.score > (((_k2 = config3.face.detector) == null ? void 0 : _k2.minConfidence) || 1))
faces.push(face4); faces.push(face4);

4
dist/human.esm.js.map vendored
View File

File diff suppressed because one or more lines are too long

62
dist/human.js vendored
View File

File diff suppressed because one or more lines are too long

8
dist/human.node-gpu.js vendored
View File

File diff suppressed because one or more lines are too long

8
dist/human.node-wasm.js vendored
View File

File diff suppressed because one or more lines are too long

8
dist/human.node.js vendored
View File

File diff suppressed because one or more lines are too long

2
package.json
View File

@ -65,7 +65,7 @@
"@tensorflow/tfjs-layers": "^3.18.0", "@tensorflow/tfjs-layers": "^3.18.0",
"@tensorflow/tfjs-node": "^3.18.0", "@tensorflow/tfjs-node": "^3.18.0",
"@tensorflow/tfjs-node-gpu": "^3.18.0", "@tensorflow/tfjs-node-gpu": "^3.18.0",
"@types/node": "^18.0.1", "@types/node": "^18.0.3",
"@types/offscreencanvas": "^2019.7.0", "@types/offscreencanvas": "^2019.7.0",
"@typescript-eslint/eslint-plugin": "^5.30.5", "@typescript-eslint/eslint-plugin": "^5.30.5",
"@typescript-eslint/parser": "^5.30.5", "@typescript-eslint/parser": "^5.30.5",

9
src/face/facemesh.ts
View File

@ -87,11 +87,8 @@ export async function predict(input: Tensor, config: Config): Promise<FaceResult
} else { // mesh enabled } else { // mesh enabled
const results = model.execute(face.tensor as Tensor) as Array<Tensor>; const results = model.execute(face.tensor as Tensor) as Array<Tensor>;
const confidenceT = results.find((t) => t.shape[t.shape.length - 1] === 1) as Tensor; const confidenceT = results.find((t) => t.shape[t.shape.length - 1] === 1) as Tensor;
const meshT = results.find((t) => t.shape[t.shape.length - 1] === 1404) as Tensor;
const faceConfidence = await confidenceT.data(); const faceConfidence = await confidenceT.data();
face.faceScore = Math.round(100 * faceConfidence[0]) / 100; face.faceScore = Math.round(100 * faceConfidence[0]) / 100;
const coordsReshaped = tf.reshape(meshT, [-1, 3]);
let rawCoords = await coordsReshaped.array();
if (face.faceScore < (config.face.detector?.minConfidence || 1)) { // low confidence in detected mesh if (face.faceScore < (config.face.detector?.minConfidence || 1)) { // low confidence in detected mesh
box.confidence = face.faceScore; // reset confidence of cached box box.confidence = face.faceScore; // reset confidence of cached box
if (config.face.mesh?.keepInvalid) { if (config.face.mesh?.keepInvalid) {
@ -108,6 +105,10 @@ export async function predict(input: Tensor, config: Config): Promise<FaceResult
} }
} }
} else { } else {
const meshT = results.find((t) => t.shape[t.shape.length - 1] === 1404) as Tensor;
const coordsReshaped = tf.reshape(meshT, [-1, 3]);
let rawCoords = await coordsReshaped.array();
tf.dispose(coordsReshaped);
if (config.face.attention?.enabled) { if (config.face.attention?.enabled) {
rawCoords = await attention.augment(rawCoords, results); // augment iris results using attention model results rawCoords = await attention.augment(rawCoords, results); // augment iris results using attention model results
} else if (config.face.iris?.enabled) { } else if (config.face.iris?.enabled) {
@ -122,7 +123,7 @@ export async function predict(input: Tensor, config: Config): Promise<FaceResult
face.boxRaw = util.getRawBox(calculatedBox, input); face.boxRaw = util.getRawBox(calculatedBox, input);
newCache.push(calculatedBox); newCache.push(calculatedBox);
} }
tf.dispose([...results, coordsReshaped]); tf.dispose(results);
} }
if (face.score > (config.face.detector?.minConfidence || 1)) faces.push(face); if (face.score > (config.face.detector?.minConfidence || 1)) faces.push(face);
else tf.dispose(face.tensor); else tf.dispose(face.tensor);

48
test/build.log
View File

@ -1,24 +1,24 @@
2022-07-05 05:01:38 INFO:  Application: {"name":"@vladmandic/human","version":"2.8.1"} 2022-07-07 12:07:40 INFO:  Application: {"name":"@vladmandic/human","version":"2.8.1"}
2022-07-05 05:01:38 INFO:  Environment: {"profile":"production","config":".build.json","package":"package.json","tsconfig":true,"eslintrc":true,"git":true} 2022-07-07 12:07:40 INFO:  Environment: {"profile":"production","config":".build.json","package":"package.json","tsconfig":true,"eslintrc":true,"git":true}
2022-07-05 05:01:38 INFO:  Toolchain: {"build":"0.7.3","esbuild":"0.14.48","typescript":"4.7.4","typedoc":"0.22.17","eslint":"8.19.0"} 2022-07-07 12:07:40 INFO:  Toolchain: {"build":"0.7.3","esbuild":"0.14.48","typescript":"4.7.4","typedoc":"0.22.18","eslint":"8.19.0"}
2022-07-05 05:01:38 INFO:  Build: {"profile":"production","steps":["clean","compile","typings","typedoc","lint","changelog"]} 2022-07-07 12:07:40 INFO:  Build: {"profile":"production","steps":["clean","compile","typings","typedoc","lint","changelog"]}
2022-07-05 05:01:38 STATE: Clean: {"locations":["dist/*","types/lib/*","typedoc/*"]} 2022-07-07 12:07:40 STATE: Clean: {"locations":["dist/*","types/lib/*","typedoc/*"]}
2022-07-05 05:01:38 STATE: Compile: {"name":"tfjs/nodejs/cpu","format":"cjs","platform":"node","input":"tfjs/tf-node.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":102,"outputBytes":608} 2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/nodejs/cpu","format":"cjs","platform":"node","input":"tfjs/tf-node.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":102,"outputBytes":608}
2022-07-05 05:01:38 STATE: Compile: {"name":"human/nodejs/cpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node.js","files":73,"inputBytes":643890,"outputBytes":301229} 2022-07-07 12:07:40 STATE: Compile: {"name":"human/nodejs/cpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node.js","files":73,"inputBytes":643911,"outputBytes":301226}
2022-07-05 05:01:38 STATE: Compile: {"name":"tfjs/nodejs/gpu","format":"cjs","platform":"node","input":"tfjs/tf-node-gpu.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":612} 2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/nodejs/gpu","format":"cjs","platform":"node","input":"tfjs/tf-node-gpu.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":612}
2022-07-05 05:01:38 STATE: Compile: {"name":"human/nodejs/gpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-gpu.js","files":73,"inputBytes":643894,"outputBytes":301233} 2022-07-07 12:07:40 STATE: Compile: {"name":"human/nodejs/gpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-gpu.js","files":73,"inputBytes":643915,"outputBytes":301230}
2022-07-05 05:01:38 STATE: Compile: {"name":"tfjs/nodejs/wasm","format":"cjs","platform":"node","input":"tfjs/tf-node-wasm.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":149,"outputBytes":664} 2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/nodejs/wasm","format":"cjs","platform":"node","input":"tfjs/tf-node-wasm.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":149,"outputBytes":664}
2022-07-05 05:01:38 STATE: Compile: {"name":"human/nodejs/wasm","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-wasm.js","files":73,"inputBytes":643946,"outputBytes":301283} 2022-07-07 12:07:40 STATE: Compile: {"name":"human/nodejs/wasm","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-wasm.js","files":73,"inputBytes":643967,"outputBytes":301280}
2022-07-05 05:01:39 STATE: Compile: {"name":"tfjs/browser/version","format":"esm","platform":"browser","input":"tfjs/tf-version.ts","output":"dist/tfjs.version.js","files":1,"inputBytes":1069,"outputBytes":358} 2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/browser/version","format":"esm","platform":"browser","input":"tfjs/tf-version.ts","output":"dist/tfjs.version.js","files":1,"inputBytes":1069,"outputBytes":358}
2022-07-05 05:01:39 STATE: Compile: {"name":"tfjs/browser/esm/nobundle","format":"esm","platform":"browser","input":"tfjs/tf-browser.ts","output":"dist/tfjs.esm.js","files":2,"inputBytes":1032,"outputBytes":583} 2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/browser/esm/nobundle","format":"esm","platform":"browser","input":"tfjs/tf-browser.ts","output":"dist/tfjs.esm.js","files":2,"inputBytes":1032,"outputBytes":583}
2022-07-05 05:01:39 STATE: Compile: {"name":"human/browser/esm/nobundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm-nobundle.js","files":73,"inputBytes":643865,"outputBytes":300113} 2022-07-07 12:07:40 STATE: Compile: {"name":"human/browser/esm/nobundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm-nobundle.js","files":73,"inputBytes":643886,"outputBytes":300114}
2022-07-05 05:01:39 STATE: Compile: {"name":"tfjs/browser/esm/custom","format":"esm","platform":"browser","input":"tfjs/tf-custom.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":1353528} 2022-07-07 12:07:41 STATE: Compile: {"name":"tfjs/browser/esm/custom","format":"esm","platform":"browser","input":"tfjs/tf-custom.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":1353528}
2022-07-05 05:01:39 STATE: Compile: {"name":"human/browser/iife/bundle","format":"iife","platform":"browser","input":"src/human.ts","output":"dist/human.js","files":73,"inputBytes":1996810,"outputBytes":1652733} 2022-07-07 12:07:41 STATE: Compile: {"name":"human/browser/iife/bundle","format":"iife","platform":"browser","input":"src/human.ts","output":"dist/human.js","files":73,"inputBytes":1996831,"outputBytes":1652728}
2022-07-05 05:01:39 STATE: Compile: {"name":"human/browser/esm/bundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm.js","files":73,"inputBytes":1996810,"outputBytes":2140087} 2022-07-07 12:07:42 STATE: Compile: {"name":"human/browser/esm/bundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm.js","files":73,"inputBytes":1996831,"outputBytes":2140100}
2022-07-05 05:01:53 STATE: Typings: {"input":"src/human.ts","output":"types/lib","files":116} 2022-07-07 12:07:57 STATE: Typings: {"input":"src/human.ts","output":"types/lib","files":116}
2022-07-05 05:01:57 STATE: TypeDoc: {"input":"src/human.ts","output":"typedoc","objects":73,"generated":true} 2022-07-07 12:08:04 STATE: TypeDoc: {"input":"src/human.ts","output":"typedoc","objects":73,"generated":true}
2022-07-05 05:01:57 STATE: Compile: {"name":"demo/typescript","format":"esm","platform":"browser","input":"demo/typescript/index.ts","output":"demo/typescript/index.js","files":1,"inputBytes":6324,"outputBytes":3057} 2022-07-07 12:08:04 STATE: Compile: {"name":"demo/typescript","format":"esm","platform":"browser","input":"demo/typescript/index.ts","output":"demo/typescript/index.js","files":1,"inputBytes":6324,"outputBytes":3057}
2022-07-05 05:01:57 STATE: Compile: {"name":"demo/faceid","format":"esm","platform":"browser","input":"demo/faceid/index.ts","output":"demo/faceid/index.js","files":2,"inputBytes":15174,"outputBytes":7820} 2022-07-07 12:08:04 STATE: Compile: {"name":"demo/faceid","format":"esm","platform":"browser","input":"demo/faceid/index.ts","output":"demo/faceid/index.js","files":2,"inputBytes":15174,"outputBytes":7820}
2022-07-05 05:02:16 STATE: Lint: {"locations":["*.json","src/**/*.ts","test/**/*.js","demo/**/*.js"],"files":106,"errors":0,"warnings":0} 2022-07-07 12:08:29 STATE: Lint: {"locations":["*.json","src/**/*.ts","test/**/*.js","demo/**/*.js"],"files":106,"errors":0,"warnings":0}
2022-07-05 05:02:16 STATE: ChangeLog: {"repository":"https://github.com/vladmandic/human","branch":"main","output":"CHANGELOG.md"} 2022-07-07 12:08:30 STATE: ChangeLog: {"repository":"https://github.com/vladmandic/human","branch":"main","output":"CHANGELOG.md"}
2022-07-05 05:02:16 INFO:  Done... 2022-07-07 12:08:30 INFO:  Done...

1368
test/test.log
View File

File diff suppressed because it is too large Load Diff