ambiser
/
human
mirror of https://github.com/vladmandic/human
9
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

add webview support

pull/356/head
Vladimir Mandic 2022-07-13 08:53:37 -04:00
parent b302c096ec
commit 79bc49b2ef
6 changed files with 743 additions and 735 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
CHANGELOG.md
View File

@ -9,8 +9,13 @@
## Changelog ## Changelog
### **HEAD -> main** 2022/07/05 mandic00@live.com ### **HEAD -> main** 2022/07/13 mandic00@live.com
### **origin/main** 2022/07/09 animeniac@live.com
- npm default install should be prod only
- fix npm v7 compatibility
- add getmodelstats method - add getmodelstats method
- rebuild - rebuild
@ -438,7 +443,7 @@
- remove efficientpose - remove efficientpose
- major version rebuild - major version rebuild
### **1.6.1** 2021/04/23 mandic00@live.com ### **1.6.1** 2021/04/22 mandic00@live.com
- add npmrc - add npmrc
- added filter.flip feature - added filter.flip feature
@ -577,7 +582,7 @@
- add typedocs and types - add typedocs and types
- strong typings - strong typings
### **1.1.2** 2021/03/13 mandic00@live.com ### **1.1.2** 2021/03/12 mandic00@live.com
- distance based on minkowski space and limited euclidean space - distance based on minkowski space and limited euclidean space
- guard against invalid input images - guard against invalid input images
@ -586,7 +591,7 @@
- switched face embedding to mobileface - switched face embedding to mobileface
### **1.0.4** 2021/03/12 mandic00@live.com ### **1.0.4** 2021/03/11 mandic00@live.com
- add face return tensor - add face return tensor
- add test for face descriptors - add test for face descriptors
@ -601,7 +606,7 @@
- enhanced age, gender, emotion detection - enhanced age, gender, emotion detection
- full rebuild - full rebuild
### **1.0.2** 2021/03/10 mandic00@live.com ### **1.0.2** 2021/03/09 mandic00@live.com
- remove blazeface-front, blazepose-upper, faceboxes - remove blazeface-front, blazepose-upper, faceboxes
- remove blazeface-front and faceboxes - remove blazeface-front and faceboxes
@ -711,7 +716,7 @@
- variable name changes, setting .rawcoords only if necessary - variable name changes, setting .rawcoords only if necessary
- option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream - option to return raw data (mesh, box) for facemesh / "preserve aspect ratio" fix from facemesh upstream
### **0.9.18** 2020/12/17 mandic00@live.com ### **0.9.18** 2020/12/16 mandic00@live.com
- add z axis scaling - add z axis scaling
- major work on body module - major work on body module
@ -745,7 +750,7 @@
- minor compatibility fixes - minor compatibility fixes
### **0.9.11** 2020/11/24 mandic00@live.com ### **0.9.11** 2020/11/23 mandic00@live.com
- implement multi-person gestures - implement multi-person gestures
- modularize pipeline models - modularize pipeline models
@ -782,7 +787,7 @@
- swtich to tfjs source import - swtich to tfjs source import
### **0.9.3** 2020/11/17 mandic00@live.com ### **0.9.3** 2020/11/16 mandic00@live.com
- switched to minified build - switched to minified build
- web worker fixes - web worker fixes
@ -812,7 +817,7 @@
- fix bug in async ops and change imports - fix bug in async ops and change imports
- fix wiki links - fix wiki links
### **0.8.6** 2020/11/10 mandic00@live.com ### **0.8.6** 2020/11/09 mandic00@live.com
- add wasm bundle - add wasm bundle
@ -852,7 +857,7 @@
- cache invalidation improvements - cache invalidation improvements
- full async operations - full async operations
### **0.7.5** 2020/11/06 mandic00@live.com ### **0.7.5** 2020/11/05 mandic00@live.com
- implemented dev-server - implemented dev-server
@ -897,12 +902,12 @@
- complete model refactoring - complete model refactoring
- fixed typo - fixed typo
### **0.6.2** 2020/11/03 mandic00@live.com ### **0.6.2** 2020/11/02 mandic00@live.com
- optimized demo - optimized demo
- package update - package update
### **0.6.1** 2020/11/03 mandic00@live.com ### **0.6.1** 2020/11/02 mandic00@live.com
- major performance improvements for all models - major performance improvements for all models
- revert "optimized canvas handling" - revert "optimized canvas handling"
@ -956,7 +961,7 @@
- implelented input resizing - implelented input resizing
### **0.4.3** 2020/10/23 mandic00@live.com ### **0.4.3** 2020/10/22 mandic00@live.com
### **0.4.2** 2020/10/20 mandic00@live.com ### **0.4.2** 2020/10/20 mandic00@live.com
@ -974,7 +979,7 @@
- pure tensor pipeline without image converts - pure tensor pipeline without image converts
- autodetect skipframes - autodetect skipframes
### **0.3.8** 2020/10/18 mandic00@live.com ### **0.3.8** 2020/10/17 mandic00@live.com
- new menu layout - new menu layout
@ -1001,28 +1006,28 @@
- optimized blazeface anchors - optimized blazeface anchors
- added error handling - added error handling
### **0.3.3** 2020/10/16 mandic00@live.com ### **0.3.3** 2020/10/15 mandic00@live.com
- added blazeface back and front models - added blazeface back and front models
### **0.3.2** 2020/10/16 mandic00@live.com ### **0.3.2** 2020/10/15 mandic00@live.com
- reduced web worker latency - reduced web worker latency
- added debugging and versioning - added debugging and versioning
- optimized demos and added scoped runs - optimized demos and added scoped runs
- added multi backend support - added multi backend support
### **0.3.1** 2020/10/15 mandic00@live.com ### **0.3.1** 2020/10/14 mandic00@live.com
### **0.2.10** 2020/10/15 mandic00@live.com ### **0.2.10** 2020/10/14 mandic00@live.com
- added emotion backend - added emotion backend
- module parametrization and performance monitoring - module parametrization and performance monitoring
- implemented multi-hand support - implemented multi-hand support
- fixed documentation typos - fixed documentation typos
### **0.2.9** 2020/10/14 mandic00@live.com ### **0.2.9** 2020/10/13 mandic00@live.com
- added node build and demo - added node build and demo
@ -1037,7 +1042,7 @@
- enable all models by default - enable all models by default
### **0.2.5** 2020/10/13 mandic00@live.com ### **0.2.5** 2020/10/12 mandic00@live.com
- fixed memory leak - fixed memory leak

1
README.md
View File

@ -19,6 +19,7 @@ JavaScript module using TensorFlow/JS Machine Learning library
Compatible with both desktop and mobile platforms Compatible with both desktop and mobile platforms
Compatible with *CPU*, *WebGL*, *WASM* backends Compatible with *CPU*, *WebGL*, *WASM* backends
Compatible with *WebWorker* execution Compatible with *WebWorker* execution
Compatible with *WebView*
- **NodeJS**: - **NodeJS**:
Compatible with both software *tfjs-node* and Compatible with both software *tfjs-node* and
GPU accelerated backends *tfjs-node-gpu* using CUDA libraries GPU accelerated backends *tfjs-node-gpu* using CUDA libraries

2
src/config.ts
View File

@ -316,7 +316,7 @@ const config: Config = {
cacheModels: true, cacheModels: true,
wasmPath: '', wasmPath: '',
wasmPlatformFetch: false, wasmPlatformFetch: false,
debug: true, debug: false,
async: true, async: true,
warmup: 'full', warmup: 'full',
cacheSensitivity: 0.70, cacheSensitivity: 0.70,

12
src/tfjs/load.ts
View File

@ -4,7 +4,8 @@ import type { GraphModel } from './types';
import type { Config } from '../config'; import type { Config } from '../config';
const options = { const options = {
cacheModels: false, cacheModels: true,
cacheSupported: true,
verbose: true, verbose: true,
debug: false, debug: false,
modelBasePath: '', modelBasePath: '',
@ -42,8 +43,9 @@ export async function loadModel(modelPath: string | undefined): Promise<GraphMod
weights: 0, weights: 0,
cached: false, cached: false,
}; };
const cachedModels = await tf.io.listModels(); // list all models already in cache options.cacheSupported = (typeof window !== 'undefined') && (typeof window.localStorage !== 'undefined') && (typeof window.indexedDB !== 'undefined'); // check if running in browser and if indexedb is available
modelStats[shortModelName].cached = options.cacheModels && Object.keys(cachedModels).includes(cachedModelName); // is model found in cache const cachedModels = (options.cacheSupported && options.cacheModels) ? await tf.io.listModels() : {}; // list all models already in cache
modelStats[shortModelName].cached = (options.cacheSupported && options.cacheModels) && Object.keys(cachedModels).includes(cachedModelName); // is model found in cache
const tfLoadOptions = typeof fetch === 'undefined' ? {} : { fetchFunc: (url, init?) => httpHandler(url, init) }; const tfLoadOptions = typeof fetch === 'undefined' ? {} : { fetchFunc: (url, init?) => httpHandler(url, init) };
const model: GraphModel = new tf.GraphModel(modelStats[shortModelName].cached ? cachedModelName : modelUrl, tfLoadOptions) as unknown as GraphModel; // create model prototype and decide if load from cache or from original modelurl const model: GraphModel = new tf.GraphModel(modelStats[shortModelName].cached ? cachedModelName : modelUrl, tfLoadOptions) as unknown as GraphModel; // create model prototype and decide if load from cache or from original modelurl
let loaded = false; let loaded = false;
@ -57,12 +59,12 @@ export async function loadModel(modelPath: string | undefined): Promise<GraphMod
model.loadSync(artifacts); // load weights model.loadSync(artifacts); // load weights
// @ts-ignore private property // @ts-ignore private property
modelStats[shortModelName].weights = model?.artifacts?.weightData?.byteLength || 0; modelStats[shortModelName].weights = model?.artifacts?.weightData?.byteLength || 0;
if (options.verbose) log('load model:', model['modelUrl'], { bytes: modelStats[shortModelName].weights }); if (options.verbose) log('load model:', model['modelUrl'], { bytes: modelStats[shortModelName].weights }, options);
loaded = true; loaded = true;
} catch (err) { } catch (err) {
log('error loading model:', modelUrl, err); log('error loading model:', modelUrl, err);
} }
if (loaded && options.cacheModels && !modelStats[shortModelName].cached) { // save model to cache if (loaded && options.cacheModels && options.cacheSupported && !modelStats[shortModelName].cached) { // save model to cache
try { try {
const saveResult = await model.save(cachedModelName); const saveResult = await model.save(cachedModelName);
log('model saved:', cachedModelName, saveResult); log('model saved:', cachedModelName, saveResult);

48
test/build.log
View File

@ -1,24 +1,24 @@
2022-07-07 12:07:40 INFO:  Application: {"name":"@vladmandic/human","version":"2.8.1"} 2022-07-13 08:48:51 INFO:  Application: {"name":"@vladmandic/human","version":"2.8.1"}
2022-07-07 12:07:40 INFO:  Environment: {"profile":"production","config":".build.json","package":"package.json","tsconfig":true,"eslintrc":true,"git":true} 2022-07-13 08:48:51 INFO:  Environment: {"profile":"production","config":".build.json","package":"package.json","tsconfig":true,"eslintrc":true,"git":true}
2022-07-07 12:07:40 INFO:  Toolchain: {"build":"0.7.3","esbuild":"0.14.48","typescript":"4.7.4","typedoc":"0.22.18","eslint":"8.19.0"} 2022-07-13 08:48:51 INFO:  Toolchain: {"build":"0.7.3","esbuild":"0.14.49","typescript":"4.7.4","typedoc":"0.22.18","eslint":"8.19.0"}
2022-07-07 12:07:40 INFO:  Build: {"profile":"production","steps":["clean","compile","typings","typedoc","lint","changelog"]} 2022-07-13 08:48:51 INFO:  Build: {"profile":"production","steps":["clean","compile","typings","typedoc","lint","changelog"]}
2022-07-07 12:07:40 STATE: Clean: {"locations":["dist/*","types/lib/*","typedoc/*"]} 2022-07-13 08:48:51 STATE: Clean: {"locations":["dist/*","types/lib/*","typedoc/*"]}
2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/nodejs/cpu","format":"cjs","platform":"node","input":"tfjs/tf-node.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":102,"outputBytes":608} 2022-07-13 08:48:51 STATE: Compile: {"name":"tfjs/nodejs/cpu","format":"cjs","platform":"node","input":"tfjs/tf-node.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":102,"outputBytes":608}
2022-07-07 12:07:40 STATE: Compile: {"name":"human/nodejs/cpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node.js","files":73,"inputBytes":643911,"outputBytes":301226} 2022-07-13 08:48:51 STATE: Compile: {"name":"human/nodejs/cpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node.js","files":73,"inputBytes":644542,"outputBytes":301481}
2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/nodejs/gpu","format":"cjs","platform":"node","input":"tfjs/tf-node-gpu.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":612} 2022-07-13 08:48:51 STATE: Compile: {"name":"tfjs/nodejs/gpu","format":"cjs","platform":"node","input":"tfjs/tf-node-gpu.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":612}
2022-07-07 12:07:40 STATE: Compile: {"name":"human/nodejs/gpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-gpu.js","files":73,"inputBytes":643915,"outputBytes":301230} 2022-07-13 08:48:51 STATE: Compile: {"name":"human/nodejs/gpu","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-gpu.js","files":73,"inputBytes":644546,"outputBytes":301485}
2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/nodejs/wasm","format":"cjs","platform":"node","input":"tfjs/tf-node-wasm.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":149,"outputBytes":664} 2022-07-13 08:48:51 STATE: Compile: {"name":"tfjs/nodejs/wasm","format":"cjs","platform":"node","input":"tfjs/tf-node-wasm.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":149,"outputBytes":664}
2022-07-07 12:07:40 STATE: Compile: {"name":"human/nodejs/wasm","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-wasm.js","files":73,"inputBytes":643967,"outputBytes":301280} 2022-07-13 08:48:51 STATE: Compile: {"name":"human/nodejs/wasm","format":"cjs","platform":"node","input":"src/human.ts","output":"dist/human.node-wasm.js","files":73,"inputBytes":644598,"outputBytes":301535}
2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/browser/version","format":"esm","platform":"browser","input":"tfjs/tf-version.ts","output":"dist/tfjs.version.js","files":1,"inputBytes":1069,"outputBytes":358} 2022-07-13 08:48:51 STATE: Compile: {"name":"tfjs/browser/version","format":"esm","platform":"browser","input":"tfjs/tf-version.ts","output":"dist/tfjs.version.js","files":1,"inputBytes":1069,"outputBytes":358}
2022-07-07 12:07:40 STATE: Compile: {"name":"tfjs/browser/esm/nobundle","format":"esm","platform":"browser","input":"tfjs/tf-browser.ts","output":"dist/tfjs.esm.js","files":2,"inputBytes":1032,"outputBytes":583} 2022-07-13 08:48:51 STATE: Compile: {"name":"tfjs/browser/esm/nobundle","format":"esm","platform":"browser","input":"tfjs/tf-browser.ts","output":"dist/tfjs.esm.js","files":2,"inputBytes":1032,"outputBytes":583}
2022-07-07 12:07:40 STATE: Compile: {"name":"human/browser/esm/nobundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm-nobundle.js","files":73,"inputBytes":643886,"outputBytes":300114} 2022-07-13 08:48:51 STATE: Compile: {"name":"human/browser/esm/nobundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm-nobundle.js","files":73,"inputBytes":644517,"outputBytes":300369}
2022-07-07 12:07:41 STATE: Compile: {"name":"tfjs/browser/esm/custom","format":"esm","platform":"browser","input":"tfjs/tf-custom.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":1353528} 2022-07-13 08:48:52 STATE: Compile: {"name":"tfjs/browser/esm/custom","format":"esm","platform":"browser","input":"tfjs/tf-custom.ts","output":"dist/tfjs.esm.js","files":1,"inputBytes":110,"outputBytes":1353528}
2022-07-07 12:07:41 STATE: Compile: {"name":"human/browser/iife/bundle","format":"iife","platform":"browser","input":"src/human.ts","output":"dist/human.js","files":73,"inputBytes":1996831,"outputBytes":1652728} 2022-07-13 08:48:52 STATE: Compile: {"name":"human/browser/iife/bundle","format":"iife","platform":"browser","input":"src/human.ts","output":"dist/human.js","files":73,"inputBytes":1997462,"outputBytes":1652983}
2022-07-07 12:07:42 STATE: Compile: {"name":"human/browser/esm/bundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm.js","files":73,"inputBytes":1996831,"outputBytes":2140100} 2022-07-13 08:48:52 STATE: Compile: {"name":"human/browser/esm/bundle","format":"esm","platform":"browser","input":"src/human.ts","output":"dist/human.esm.js","files":73,"inputBytes":1997462,"outputBytes":2140437}
2022-07-07 12:07:57 STATE: Typings: {"input":"src/human.ts","output":"types/lib","files":116} 2022-07-13 08:48:58 STATE: Typings: {"input":"src/human.ts","output":"types/lib","files":116}
2022-07-07 12:08:04 STATE: TypeDoc: {"input":"src/human.ts","output":"typedoc","objects":73,"generated":true} 2022-07-13 08:49:01 STATE: TypeDoc: {"input":"src/human.ts","output":"typedoc","objects":74,"generated":true}
2022-07-07 12:08:04 STATE: Compile: {"name":"demo/typescript","format":"esm","platform":"browser","input":"demo/typescript/index.ts","output":"demo/typescript/index.js","files":1,"inputBytes":6324,"outputBytes":3057} 2022-07-13 08:49:01 STATE: Compile: {"name":"demo/typescript","format":"esm","platform":"browser","input":"demo/typescript/index.ts","output":"demo/typescript/index.js","files":1,"inputBytes":6324,"outputBytes":3057}
2022-07-07 12:08:04 STATE: Compile: {"name":"demo/faceid","format":"esm","platform":"browser","input":"demo/faceid/index.ts","output":"demo/faceid/index.js","files":2,"inputBytes":15174,"outputBytes":7820} 2022-07-13 08:49:01 STATE: Compile: {"name":"demo/faceid","format":"esm","platform":"browser","input":"demo/faceid/index.ts","output":"demo/faceid/index.js","files":2,"inputBytes":15174,"outputBytes":7820}
2022-07-07 12:08:29 STATE: Lint: {"locations":["*.json","src/**/*.ts","test/**/*.js","demo/**/*.js"],"files":106,"errors":0,"warnings":0} 2022-07-13 08:49:10 STATE: Lint: {"locations":["*.json","src/**/*.ts","test/**/*.js","demo/**/*.js"],"files":106,"errors":0,"warnings":0}
2022-07-07 12:08:30 STATE: ChangeLog: {"repository":"https://github.com/vladmandic/human","branch":"main","output":"CHANGELOG.md"} 2022-07-13 08:49:10 STATE: ChangeLog: {"repository":"https://github.com/vladmandic/human","branch":"main","output":"CHANGELOG.md"}
2022-07-07 12:08:30 INFO:  Done... 2022-07-13 08:49:10 INFO:  Done...

1368
test/test.log
View File

File diff suppressed because it is too large Load Diff