Compare commits

..

1049 Commits
main ... v2.7.1

Author SHA1 Message Date
Vladimir Mandic 2ea6847ef9 major release 2022-05-09 08:16:00 -04:00
Vladimir Mandic f4b2f202e5 2.7.1 2022-05-09 08:14:00 -04:00
Vladimir Mandic f7a4d6840c update wiki 2022-04-23 13:02:00 -04:00
Vladimir Mandic 147f766d5e update todo 2022-04-21 09:58:13 -04:00
Vladimir Mandic 8cadcb6715 support 4k input 2022-04-21 09:39:40 -04:00
Vladimir Mandic 768c8c062c update tfjs 2022-04-21 09:38:36 -04:00
Vladimir Mandic 8aa4d3d647 add attention draw methods 2022-04-18 12:26:05 -04:00
Vladimir Mandic f0fc73b98d fix coloring function 2022-04-18 11:29:45 -04:00
Vladimir Mandic b18e2ace64 enable precompile as part of warmup 2022-04-15 07:54:27 -04:00
Vladimir Mandic 803eb0f571 prepare release beta 2022-04-14 11:55:49 -04:00
Vladimir Mandic 1eb4429da4 change default face crop 2022-04-14 11:47:08 -04:00
Vladimir Mandic 190f376d18 update wiki 2022-04-11 11:55:30 -04:00
Vladimir Mandic 3ba05e4d08 face attention model is available in human-models 2022-04-11 11:51:04 -04:00
Vladimir Mandic 97a67c8b00 beta release 2.7 2022-04-11 11:46:35 -04:00
Vladimir Mandic 048d8bac01 refactor draw methods 2022-04-11 11:46:00 -04:00
Vladimir Mandic 2d7cf52e72 implement face attention model 2022-04-11 11:45:24 -04:00
Vladimir Mandic 1bc2831f2b add electronjs demo 2022-04-10 11:00:41 -04:00
Vladimir Mandic d9d6cbaf8f rebuild 2022-04-10 10:13:13 -04:00
Vladimir Mandic 08c0d7af85 rebuild 2022-04-05 12:25:41 -04:00
Vladimir Mandic 0c99cfdbe8 update tfjs 2022-04-01 12:38:05 -04:00
Vladimir Mandic 5214eaaf6a update 2022-04-01 09:13:32 -04:00
Vladimir Mandic b5dbb83b78 2.6.5 2022-04-01 09:12:13 -04:00
Vladimir Mandic ca89a31f96 bundle offscreencanvas types 2022-04-01 09:12:04 -04:00
Vladimir Mandic 42ba9b0c4d prototype precompile pass 2022-03-19 11:02:30 -04:00
Vladimir Mandic 7c0e711e34 fix changelog generation 2022-03-16 11:38:57 -04:00
Vladimir Mandic 1ee5ebd553 fix indexdb config check 2022-03-16 11:19:56 -04:00
Vladimir Mandic 8b0ceb23be update typescript and tensorflow 2022-03-07 13:24:06 -05:00
Vladimir Mandic d47964f4a3 2.6.4 2022-02-27 07:25:45 -05:00
Vladimir Mandic cf08ef892c fix types typo 2022-02-17 08:15:57 -05:00
Vladimir Mandic f7ecc00d90 refresh 2022-02-14 07:53:28 -05:00
Vladimir Mandic 089f2641e5 add config option wasmPlatformFetch 2022-02-10 15:35:32 -05:00
Vladimir Mandic ea7d0fd2e0 2.6.3 2022-02-10 15:32:53 -05:00
Vladimir Mandic cf53f2d350 rebuild 2022-02-10 12:27:21 -05:00
Vladimir Mandic f7b7fdca1f update toolkit 2022-02-07 10:12:59 -05:00
Vladimir Mandic 306ee94cfe 2.6.2 2022-02-07 09:47:17 -05:00
Vladimir Mandic 0839aaaf0f update todo 2022-01-20 08:24:23 -05:00
Vladimir Mandic 455c7f519d release rebuild 2022-01-20 08:17:06 -05:00
Vladimir Mandic 9a94631146 2.6.1 2022-01-20 07:54:56 -05:00
Vladimir Mandic 733ce5bb66 implement model caching using indexdb 2022-01-17 11:03:21 -05:00
Vladimir Mandic afd3312109 prototype global fetch handler 2022-01-16 09:49:55 -05:00
Vladimir Mandic 6fb16a1bba update samples 2022-01-15 09:18:14 -05:00
Vladimir Mandic 5ef19460c1 update samples 2022-01-15 09:11:04 -05:00
Vladimir Mandic c8c7bb488d update samples with images under cc licence only 2022-01-14 16:10:32 -05:00
Vladimir Mandic db4e49909c fix face box and hand tracking when in front of face 2022-01-14 09:46:16 -05:00
Vladimir Mandic 3c49e5c1ef 2.5.8 2022-01-14 09:42:57 -05:00
Vladimir Mandic 60ab255d26 update 2022-01-08 12:43:44 -05:00
Vladimir Mandic b20ac01743 update wiki 2022-01-05 11:49:10 -05:00
Vladimir Mandic 93b46e00b8 update wiki 2022-01-05 09:55:07 -05:00
Vladimir Mandic 2b53c78b96 update 2022-01-05 08:34:31 -05:00
Vladimir Mandic 2b4fe6c78e update demos 2022-01-01 08:13:04 -05:00
Vladimir Mandic 266501a460 update blazepose 2021-12-31 13:58:03 -05:00
Vladimir Mandic 1e128b3dbc update dependencies 2021-12-30 12:39:29 -05:00
Vladimir Mandic c7209e5654 update hand annotations 2021-12-30 12:14:09 -05:00
Vladimir Mandic 939ce728a9 update blazepose 2021-12-29 12:37:46 -05:00
Vladimir Mandic 04a5e76816 update 2021-12-28 11:39:54 -05:00
Vladimir Mandic 36b657b901 update demos 2021-12-28 09:40:32 -05:00
Vladimir Mandic 13db065375 fix samples 2021-12-28 07:03:05 -05:00
libowen.eric d8114a2871 fix(src): typo 2021-12-28 06:59:16 -05:00
Vladimir Mandic e772efa633 change on how face box is calculated 2021-12-27 10:59:56 -05:00
Vladimir Mandic 806a2e5382 2.5.7 2021-12-27 09:29:15 -05:00
Vladimir Mandic 13ff1ff480 update 2021-12-22 10:04:41 -05:00
Vladimir Mandic 605168e7fa fix posenet 2021-12-18 12:24:01 -05:00
Vladimir Mandic 75816c7132 release refresh 2021-12-15 09:30:26 -05:00
Vladimir Mandic 4ad62d1056 2.5.6 2021-12-15 09:26:40 -05:00
Vladimir Mandic 8360cdb2ce strong type for string enums 2021-12-15 09:26:32 -05:00
Vladimir Mandic b5862fb6a2 update 2021-12-14 15:45:43 -05:00
Vladimir Mandic d28671f0a7 rebuild 2021-12-13 21:38:55 -05:00
Vladimir Mandic 8d6c319e5e update tfjs 2021-12-09 14:44:26 -05:00
Vladimir Mandic e5ee7453ea fix node detection in electron environment 2021-12-07 17:02:33 -05:00
Vladimir Mandic 5486b0d631 update 2021-12-01 08:27:05 -05:00
Vladimir Mandic c03fd41ca7 2.5.5 2021-12-01 08:21:55 -05:00
Vladimir Mandic 816c50d29d update readme 2021-11-26 12:14:40 -05:00
Vladimir Mandic 35c824fb75 added human-motion 2021-11-26 12:12:46 -05:00
Vladimir Mandic b82cde957b add offscreencanvas typedefs 2021-11-26 11:55:52 -05:00
Vladimir Mandic a60721c5d8 update blazepose and extend hand annotations 2021-11-24 16:17:03 -05:00
Vladimir Mandic d797c871c7 release preview 2021-11-23 10:40:40 -05:00
Vladimir Mandic f3720857ae fix face box scaling on detection 2021-11-23 08:36:32 -05:00
Vladimir Mandic e342e8ec47 cleanup 2021-11-22 14:44:25 -05:00
Vladimir Mandic ba05d3fe42 2.5.4 2021-11-22 14:33:46 -05:00
Vladimir Mandic 9304d59413 prototype blazepose detector 2021-11-22 14:33:40 -05:00
Vladimir Mandic 581dbf8471 minor fixes 2021-11-21 16:55:17 -05:00
Vladimir Mandic 20cb91739d add body 3d interpolation 2021-11-19 18:30:57 -05:00
Vladimir Mandic c8f85ead30 edit blazepose keypoints 2021-11-19 16:11:03 -05:00
Vladimir Mandic ff8f5968fb new build process 2021-11-18 10:10:06 -05:00
Vladimir Mandic 473389f313 2.5.3 2021-11-18 10:06:07 -05:00
Vladimir Mandic 5cccc4118a update typescript 2021-11-17 16:50:21 -05:00
Vladimir Mandic 8bb509926b create typedef rollup 2021-11-17 15:45:49 -05:00
Vladimir Mandic 12fc9079dc optimize centernet 2021-11-16 20:16:49 -05:00
Vladimir Mandic 98a1817d13 cache frequent tf constants 2021-11-16 18:31:07 -05:00
Vladimir Mandic ce2e669dad add extra face rotation prior to mesh 2021-11-16 13:07:44 -05:00
Vladimir Mandic 53fcecdf11 release 2.5.2 2021-11-15 09:26:38 -05:00
Vladimir Mandic 5811ee3fb6 improve error handling 2021-11-14 11:22:52 -05:00
Vladimir Mandic 51ac90163e 2.5.2 2021-11-14 10:43:00 -05:00
Vladimir Mandic e63b817c33 fix mobilefacenet module 2021-11-13 17:26:19 -05:00
Vladimir Mandic 50a678c33a fix gear and ssrnet modules 2021-11-13 12:23:32 -05:00
Vladimir Mandic ee2deb88dc fix for face crop when mesh is disabled 2021-11-12 15:17:08 -05:00
Vladimir Mandic eaa691f252 implement optional face masking 2021-11-12 15:07:23 -05:00
Vladimir Mandic 1d7317b3ef update todo 2021-11-11 17:02:32 -05:00
Vladimir Mandic 471896bf5b add similarity score range normalization 2021-11-11 17:01:10 -05:00
Vladimir Mandic 200bccbb43 add faceid demo 2021-11-11 11:30:55 -05:00
Vladimir Mandic db7790d7d8 documentation overhaul 2021-11-10 12:21:45 -05:00
Vladimir Mandic 2f6524722e auto tensor shape and channels handling 2021-11-09 19:39:18 -05:00
Vladimir Mandic 7a1707cf18 disable use of path2d in node 2021-11-09 18:10:54 -05:00
Vladimir Mandic 4ed8c9c190 update wiki 2021-11-09 14:45:45 -05:00
Vladimir Mandic 32539a10f5 add liveness module and facerecognition demo 2021-11-09 14:37:50 -05:00
Vladimir Mandic dfd1ee8418 initial version of facerecognition demo 2021-11-09 10:39:23 -05:00
Vladimir Mandic 4477bbffc6 rebuild 2021-11-08 16:41:30 -05:00
Vladimir Mandic c5fdd9c336 add type defs when working with relative path imports 2021-11-08 16:36:20 -05:00
Vladimir Mandic eb7e1558d0 disable humangl backend if webgl 1.0 is detected 2021-11-08 11:35:35 -05:00
Vladimir Mandic e87bd17ff4 add additional hand gestures 2021-11-08 07:36:26 -05:00
Vladimir Mandic 585ca1d1b8 2.5.1 2021-11-08 06:25:07 -05:00
Vladimir Mandic a1b7c0d88d update automated tests 2021-11-07 10:10:23 -05:00
Vladimir Mandic 85cfeb4026 new human.compare api 2021-11-07 10:03:33 -05:00
Vladimir Mandic 6cae3f00b6 added links to release notes 2021-11-07 08:14:14 -05:00
Vladimir Mandic 6e5f5829b2 update readme 2021-11-06 10:26:04 -04:00
Vladimir Mandic cb461be486 new frame change detection algorithm 2021-11-06 10:21:51 -04:00
Vladimir Mandic 51f4b1fa20 add histogram equalization 2021-11-05 15:35:53 -04:00
Vladimir Mandic f6e733b5f0 add histogram equalization 2021-11-05 15:09:54 -04:00
Vladimir Mandic 1e197a74bf implement wasm missing ops 2021-11-05 13:36:53 -04:00
Vladimir Mandic c113370c21 performance and memory optimizations 2021-11-05 11:28:06 -04:00
Vladimir Mandic cacebc7dbe fix react compatibility issues 2021-11-04 06:34:13 -04:00
Vladimir Mandic 1ddbdcdd2e improve box rescaling for all modules 2021-11-03 16:32:07 -04:00
Vladimir Mandic e45509d163 improve precision using wasm backend 2021-11-02 11:42:15 -04:00
Vladimir Mandic 4ff1a7d063 refactor predict with execute 2021-11-02 11:07:11 -04:00
Vladimir Mandic 28d306e157 update tests 2021-10-31 09:58:48 -04:00
Vladimir Mandic be767fb02d update hand landmarks model 2021-10-31 09:06:33 -04:00
Vladimir Mandic f773ef3dbe patch tfjs type defs 2021-10-31 08:03:42 -04:00
Vladimir Mandic 20ee0f5f3f start 2.5 major version 2021-10-30 12:21:54 -04:00
Vladimir Mandic 560b3ff671 build and docs cleanup 2021-10-29 15:55:20 -04:00
Vladimir Mandic 2f874b30bb fix firefox bug 2021-10-28 17:25:50 -04:00
Vladimir Mandic c7484046a7 update tfjs 2021-10-28 14:40:31 -04:00
Vladimir Mandic 884684f501 2.4.3 2021-10-28 13:59:57 -04:00
Vladimir Mandic 405fd8a6c2 additional human.performance counters 2021-10-27 09:45:38 -04:00
Vladimir Mandic 856ca5a9a0 2.4.2 2021-10-27 09:44:17 -04:00
Vladimir Mandic 0ccf0820f0 add ts demo 2021-10-27 08:16:06 -04:00
Vladimir Mandic 8b29d262d8 switch from es2018 to es2020 for main build 2021-10-26 19:38:23 -04:00
Vladimir Mandic 404a275590 switch to custom tfjs for demos 2021-10-26 15:08:05 -04:00
Vladimir Mandic 464fcf5622 update todo 2021-10-25 13:45:04 -04:00
Vladimir Mandic 32562dfb83 release 2.4 2021-10-25 13:29:29 -04:00
Vladimir Mandic 173aac2eac 2.4.1 2021-10-25 13:09:41 -04:00
Vladimir Mandic 5af91547c6 refactoring plus jsdoc comments 2021-10-25 13:09:00 -04:00
Vladimir Mandic 90503d595d increase face similarity match resolution 2021-10-25 09:44:13 -04:00
Vladimir Mandic 33e3a5913b update todo 2021-10-23 09:42:41 -04:00
Vladimir Mandic dbd64ae6c2 time based caching 2021-10-23 09:38:52 -04:00
Vladimir Mandic 6ca5df0082 turn on minification 2021-10-22 20:14:13 -04:00
Vladimir Mandic e433a08076 update todo 2021-10-22 16:11:02 -04:00
Vladimir Mandic 5afebaa588 initial work on skipTime 2021-10-22 16:09:52 -04:00
Vladimir Mandic 4cec6f5735 added generic types 2021-10-22 14:46:19 -04:00
Vladimir Mandic f1de75c5be enhanced typing exports 2021-10-22 13:49:40 -04:00
Vladimir Mandic fcd82fb07d update tfjs to 3.10.0 2021-10-22 09:48:27 -04:00
Vladimir Mandic 1680032088 add optional autodetected custom wasm path 2021-10-21 12:42:08 -04:00
Vladimir Mandic 39fd4f5147 2.3.6 2021-10-21 11:31:46 -04:00
Vladimir Mandic 06f20e86c2 fix for human.draw labels and typedefs 2021-10-21 10:54:51 -04:00
Vladimir Mandic b28327d063 refactor human.env to a class type 2021-10-21 10:26:44 -04:00
Vladimir Mandic 309bac7fbe add human.custom.esm using custom tfjs build 2021-10-20 17:49:00 -04:00
Vladimir Mandic 4f46a81eda update handtrack boxes and refactor handpose 2021-10-20 09:10:57 -04:00
Vladimir Mandic 5f6aac9928 update demos 2021-10-19 11:28:59 -04:00
Vladimir Mandic 486d5374a2 2.3.5 2021-10-19 11:25:05 -04:00
Jimmy Nyström 9025e60e3a Removed direct usage of performance.now
Switched to using the utility function that works in both nodejs and browser environments
2021-10-19 09:58:14 -04:00
Vladimir Mandic e3640ec5bd update 2021-10-19 08:09:46 -04:00
Vladimir Mandic ecd295334c 2.3.4 2021-10-19 08:05:19 -04:00
Vladimir Mandic 4ed754e539 update dependencies and refresh release 2021-10-19 07:58:51 -04:00
Vladimir Mandic 2ca389560f minor blazepose optimizations 2021-10-15 09:34:40 -04:00
Vladimir Mandic e6d794205c compress samples 2021-10-15 07:25:51 -04:00
Vladimir Mandic 96a8a3c837 remove posenet from default package 2021-10-15 06:49:41 -04:00
Vladimir Mandic cd35d733d9 enhanced movenet postprocessing 2021-10-14 12:26:59 -04:00
Vladimir Mandic 86a4cedf81 update handtrack skip algorithm 2021-10-13 14:49:41 -04:00
Vladimir Mandic 80be4436bb use transferrable buffer for worker messages 2021-10-13 11:53:54 -04:00
Vladimir Mandic 7e8973a7be update todo 2021-10-13 11:02:44 -04:00
Vladimir Mandic fcc64e845c add optional anti-spoofing module 2021-10-13 10:56:56 -04:00
Vladimir Mandic 86db81d29c update todo 2021-10-13 08:36:20 -04:00
Vladimir Mandic 4b0bb6aa50 add node-match advanced example using worker thread pool 2021-10-13 08:06:11 -04:00
Vladimir Mandic 569b1afcfd package updates 2021-10-12 14:17:33 -04:00
Vladimir Mandic e7dce1311c optimize image preprocessing 2021-10-12 11:39:18 -04:00
Vladimir Mandic de97051399 update imagefx 2021-10-12 09:48:00 -04:00
Vladimir Mandic bb7f1bd521 set webgpu optimized flags 2021-10-11 09:22:39 -04:00
Vladimir Mandic 3925d6d426 major precision improvements to movenet and handtrack 2021-10-10 22:29:20 -04:00
Vladimir Mandic 6efedef077 image processing fixes 2021-10-10 17:52:43 -04:00
Vladimir Mandic 54371fbba5 redesign body and hand caching and interpolation 2021-10-08 18:39:04 -04:00
Vladimir Mandic f20d8b9801 demo default config cleanup 2021-10-08 07:48:48 -04:00
Vladimir Mandic 8293bc26b9 improve gaze and face angle visualizations in draw 2021-10-07 10:33:10 -04:00
Vladimir Mandic d991adc940 release 2.3.1 2021-10-06 11:33:58 -04:00
Vladimir Mandic 008569c274 2.3.1 2021-10-06 11:30:44 -04:00
Vladimir Mandic 4da8745033 workaround for chrome offscreencanvas bug 2021-10-06 11:30:34 -04:00
Vladimir Mandic b8c9b98011 fix backend conflict in webworker 2021-10-04 17:03:36 -04:00
Vladimir Mandic 051ab8c9f5 add blazepose v2 and add annotations to body results 2021-10-04 16:29:15 -04:00
Vladimir Mandic 0ee5d5eceb fix backend order initialization 2021-10-03 08:12:26 -04:00
Vladimir Mandic e4ef0c39f9 added docker notes 2021-10-02 11:41:51 -04:00
Vladimir Mandic a1b13e0627 update dependencies 2021-10-02 07:46:07 -04:00
Vladimir Mandic f4056e717e updated hint rules 2021-10-01 12:07:14 -04:00
Vladimir Mandic 445325dd10 updated facematch demo 2021-10-01 11:40:57 -04:00
Vladimir Mandic 09596f8d21 update wiki 2021-09-30 14:29:14 -04:00
Vladimir Mandic b02c6fa413 breaking change: new similarity and match methods 2021-09-30 14:28:16 -04:00
Vladimir Mandic ceaff322a8 update facematch demo 2021-09-29 08:02:23 -04:00
Vladimir Mandic 443cb24241 update movenet-multipose and samples 2021-09-28 17:07:34 -04:00
Vladimir Mandic 4b807b5f11 tweaked default values 2021-09-28 13:48:29 -04:00
Vladimir Mandic bd5cc2b36b update todo 2021-09-28 12:02:47 -04:00
Vladimir Mandic 7d7ad66e16 enable handtrack as default model 2021-09-28 12:02:17 -04:00
Vladimir Mandic 0c951b322a redesign face processing 2021-09-28 12:01:48 -04:00
Vladimir Mandic 725256edc0 update types and dependencies 2021-09-27 14:39:54 -04:00
Vladimir Mandic e3fd7a5b61 refactoring 2021-09-27 13:58:13 -04:00
Vladimir Mandic 72aad6e812 define app specific types 2021-09-27 09:19:43 -04:00
Vladimir Mandic a7ddea2585 implement box caching for movenet 2021-09-27 08:53:41 -04:00
Vladimir Mandic bdc21b32ff update todo 2021-09-26 10:09:30 -04:00
Vladimir Mandic 79c6240d7e update todo 2021-09-26 10:03:39 -04:00
Vladimir Mandic e7f38c7f33 update wiki 2021-09-26 06:53:06 -04:00
Vladimir Mandic cd35d39352 autodetect number of bodies and hands 2021-09-25 19:14:03 -04:00
Vladimir Mandic 7da344b8d5 upload new samples 2021-09-25 16:31:44 -04:00
Vladimir Mandic 619f6bcc87 new samples gallery and major code folder restructure 2021-09-25 11:51:15 -04:00
Vladimir Mandic c2a1d57eeb update todo 2021-09-24 09:57:03 -04:00
Vladimir Mandic cc71f647bf new release 2021-09-24 09:55:27 -04:00
Vladimir Mandic 77b3010fb1 2.2.3 2021-09-24 09:46:35 -04:00
Vladimir Mandic 91df711db0 optimize model loading 2021-09-23 14:09:41 -04:00
Vladimir Mandic 88a9701d4a support segmentation for nodejs 2021-09-22 19:27:12 -04:00
Vladimir Mandic a259b1f0c1 update todo and docs 2021-09-22 16:00:43 -04:00
Vladimir Mandic 291af25ef8 redo segmentation and handtracking 2021-09-22 15:16:14 -04:00
Vladimir Mandic 17b251748e prototype handtracking 2021-09-21 16:48:16 -04:00
Vladimir Mandic 2ee248b05d automated browser tests 2021-09-20 22:06:49 -04:00
Vladimir Mandic 1d3d25e0bc support for dynamic backend switching 2021-09-20 21:59:49 -04:00
Vladimir Mandic 1cd528f8e5 initial automated browser tests 2021-09-20 17:17:13 -04:00
Vladimir Mandic 97a3abc654 enhanced automated test coverage 2021-09-20 09:42:34 -04:00
Vladimir Mandic a6d01c2ed3 more automated tests 2021-09-19 14:20:22 -04:00
Vladimir Mandic 7d28cd06f2 added configuration validation 2021-09-19 14:07:53 -04:00
Vladimir Mandic accd2d1e65 updated build platform and typedoc theme 2021-09-18 19:09:02 -04:00
Vladimir Mandic 22827eafe2 prevent validation failed on some model combinations 2021-09-17 14:30:57 -04:00
Vladimir Mandic d5d2afee0f webgl exception handling 2021-09-17 14:07:44 -04:00
Vladimir Mandic 22364c9583 2.2.2 2021-09-17 14:07:32 -04:00
Vladimir Mandic 5587f64f82 experimental webgl status monitoring 2021-09-17 11:23:00 -04:00
Vladimir Mandic 29428ca093 major release 2021-09-16 10:49:42 -04:00
Vladimir Mandic 47b4f40a19 2.2.1 2021-09-16 10:46:24 -04:00
Vladimir Mandic 1a81b000c9 add vr model demo 2021-09-16 10:15:20 -04:00
Vladimir Mandic b56b6ef9ef update readme 2021-09-15 19:12:05 -04:00
Vladimir Mandic c3da1c0b78 all tests passing 2021-09-15 19:02:51 -04:00
Vladimir Mandic 6a7338a89f redefine draw helpers interface 2021-09-15 18:58:54 -04:00
Vladimir Mandic bfe889c03d add simple webcam and webrtc demo 2021-09-15 13:59:18 -04:00
Vladimir Mandic d29ac601d4 added visual results browser to demo 2021-09-15 11:15:38 -04:00
Vladimir Mandic f492002a3c reorganize tfjs bundle 2021-09-14 22:07:13 -04:00
Vladimir Mandic 86506b8b8c experimental custom tfjs bundle - disabled 2021-09-14 20:07:08 -04:00
Vladimir Mandic 5916396d48 add platform and backend capabilities detection 2021-09-13 23:24:04 -04:00
Vladimir Mandic d2f92820b6 update changelog and todo 2021-09-13 13:54:42 -04:00
Vladimir Mandic d3f38e9029 update dependencies 2021-09-13 13:34:41 -04:00
Vladimir Mandic b338abca11 enhanced automated tests 2021-09-13 13:30:46 -04:00
Vladimir Mandic 1dfe9e00e5 enable canvas patching for nodejs 2021-09-13 13:30:08 -04:00
Vladimir Mandic 36960735d1 full ts strict typechecks 2021-09-13 13:29:14 -04:00
Vladimir Mandic 45af8e225c fix multiple memory leaks 2021-09-13 13:28:35 -04:00
Vladimir Mandic 97ce559e60 modularize human class and add model validation 2021-09-12 18:37:06 -04:00
Vladimir Mandic 71be3eb818 update todo 2021-09-12 13:18:33 -04:00
Vladimir Mandic 0ebf93154f add dynamic kernel op detection 2021-09-12 13:17:33 -04:00
Vladimir Mandic 5382624645 added human.env diagnostic class 2021-09-12 12:42:17 -04:00
Vladimir Mandic bd6e291987 minor typos 2021-09-12 08:49:56 -04:00
Vladimir Mandic d3ecc64468 release candidate 2021-09-12 00:30:11 -04:00
Vladimir Mandic 27d0b4dfa4 parametrize face config 2021-09-12 00:05:06 -04:00
Vladimir Mandic 6970c43ec5 mark all config items as optional 2021-09-11 23:59:41 -04:00
Vladimir Mandic 7c0751b15e redefine config and result interfaces 2021-09-11 23:54:35 -04:00
Vladimir Mandic b6f3f6a494 fix usge of string enums 2021-09-11 23:08:18 -04:00
Vladimir Mandic 69021653be start using partial definitions 2021-09-11 16:11:00 -04:00
Vladimir Mandic 1d27bc03ed implement event emitters 2021-09-11 16:00:16 -04:00
Vladimir Mandic 56aad00975 fix iife loader 2021-09-11 11:42:48 -04:00
Vladimir Mandic 0692c9c71f update sourcemaps 2021-09-11 11:17:13 -04:00
Vladimir Mandic 65b114ae28 simplify dependencies 2021-09-11 10:29:31 -04:00
Vladimir Mandic 48aa359bb5 change build process 2021-09-10 21:21:29 -04:00
Vladimir Mandic e55df688d8 updated wiki 2021-09-06 08:17:48 -04:00
Vladimir Mandic 558c61de05 update lint exceptions 2021-09-05 17:05:46 -04:00
Vladimir Mandic 4aac3a02b9 update wiki 2021-09-05 16:48:57 -04:00
Vladimir Mandic ce703f48fd add benchmark info 2021-09-05 16:42:11 -04:00
Vladimir Mandic b0bd103db2 update hand detector processing algorithm 2021-09-02 08:50:16 -04:00
Vladimir Mandic ff6cadead0 update 2021-08-31 18:24:30 -04:00
Vladimir Mandic 50c66535c5 simplify canvas handling in nodejs 2021-08-31 18:22:16 -04:00
Vladimir Mandic 6965b67b08 full rebuild 2021-08-31 14:50:16 -04:00
Vladimir Mandic 30a2e6a074 2.1.5 2021-08-31 14:49:07 -04:00
Vladimir Mandic 7af32d08c3 added demo node-canvas 2021-08-31 14:48:55 -04:00
Vladimir Mandic 6066ad8e31 update node-fetch 2021-08-31 13:29:29 -04:00
Vladimir Mandic cfeac24fc2 dynamically generate default wasm path 2021-08-31 13:00:06 -04:00
Vladimir Mandic 42100caad7 updated wiki 2021-08-23 08:41:50 -04:00
Vladimir Mandic 40de479c1e implement finger poses in hand detection and gestures 2021-08-20 20:43:03 -04:00
Vladimir Mandic 070bb3a2c1 implemented movenet-multipose model 2021-08-20 09:05:07 -04:00
Vladimir Mandic aabe01f9b0 update todo 2021-08-19 17:28:07 -04:00
Vladimir Mandic 25a98c1cce 2.1.4 2021-08-19 16:17:03 -04:00
Vladimir Mandic a009d58377 add static type definitions to main class 2021-08-19 16:16:56 -04:00
Vladimir Mandic 2eb8b91549 fix interpolation overflow 2021-08-18 14:28:31 -04:00
Vladimir Mandic 9ae7f21c81 rebuild full 2021-08-17 18:49:49 -04:00
Vladimir Mandic 6854256668 update angle calculations 2021-08-17 18:46:50 -04:00
Vladimir Mandic 83fb9cac03 improve face box caching 2021-08-17 09:15:47 -04:00
Vladimir Mandic f4a1fb2d48 strict type checks 2021-08-17 08:51:17 -04:00
Vladimir Mandic 7c6d9dd3fe add webgu checks 2021-08-15 08:09:40 -04:00
Vladimir Mandic cdb7cd5267 update todo 2021-08-14 18:02:39 -04:00
Vladimir Mandic d047ddcd73 experimental webgpu support 2021-08-14 18:00:26 -04:00
Vladimir Mandic 6f5d4d2fe2 add experimental webgu demo 2021-08-14 13:39:26 -04:00
Vladimir Mandic df58810073 add backend initialization checks 2021-08-14 11:17:51 -04:00
Vladimir Mandic a77a39dbfa complete async work 2021-08-14 11:16:26 -04:00
Vladimir Mandic f77aa04dfe update node-webcam 2021-08-13 18:47:37 -04:00
Vladimir Mandic 2f9d7dedae list detect cameras 2021-08-13 10:34:09 -04:00
Vladimir Mandic fc01b2a4a6 switch to async data reads 2021-08-12 09:31:16 -04:00
Vladimir Mandic 24e389019b 2.1.3 2021-08-12 09:29:48 -04:00
Vladimir Mandic 8b6e9327a4 fix centernet & update blazeface 2021-08-11 18:59:02 -04:00
Vladimir Mandic d467fbbafd update todo 2021-08-09 10:46:03 -04:00
Vladimir Mandic 88977c0fe4 update model list 2021-08-06 08:50:50 -04:00
Vladimir Mandic 8572d589ca minor update 2021-08-06 08:29:41 -04:00
Vladimir Mandic f68e76c945 minor update 2021-08-05 10:38:04 -04:00
Vladimir Mandic eeed676823 update build process to remove warnings 2021-07-31 20:42:28 -04:00
Vladimir Mandic 8d6e8dd1f6 update todo 2021-07-31 07:43:50 -04:00
Vladimir Mandic b48a809c5c update typedoc links 2021-07-31 07:29:37 -04:00
Vladimir Mandic e741ba7f02 replace movenet with lightning-v4 2021-07-30 07:18:54 -04:00
Vladimir Mandic 767e7b93ad update eslint rules 2021-07-30 06:49:41 -04:00
Vladimir Mandic a0248a9f7c enable webgl uniform support for faster warmup 2021-07-29 16:35:16 -04:00
Vladimir Mandic 4a5b7c40bb 2.1.2 2021-07-29 16:34:03 -04:00
Vladimir Mandic d08b657ecd fix unregistered ops in tfjs 2021-07-29 16:06:03 -04:00
Vladimir Mandic 92e9b87829 update build 2021-07-29 12:50:06 -04:00
Vladimir Mandic 7c0256b502 fix typo 2021-07-29 11:26:19 -04:00
Vladimir Mandic 21f457d7a0 updated wiki 2021-07-29 11:06:34 -04:00
Vladimir Mandic b3a04b0315 rebuild new release 2021-07-29 11:03:21 -04:00
Vladimir Mandic 036757265e 2.1.1 2021-07-29 11:02:02 -04:00
Vladimir Mandic dbeaabbcbe updated gesture types 2021-07-29 11:01:50 -04:00
Vladimir Mandic 26c49a59d0 update tfjs and typescript 2021-07-29 09:53:13 -04:00
Vladimir Mandic 35def2fd62 updated minimum version of nodejs to v14 2021-07-29 09:41:17 -04:00
Vladimir Mandic fab6eb0b06 add note on manually disping tensor 2021-06-18 13:39:20 -04:00
Vladimir Mandic a9fffcb434 update todo 2021-06-18 09:19:34 -04:00
Vladimir Mandic 3b75e5e82c modularize model loading 2021-06-18 09:16:21 -04:00
Vladimir Mandic 53b4939a62 update typedoc 2021-06-18 07:25:33 -04:00
Vladimir Mandic 473d1450bc 2.0.3 2021-06-18 07:20:33 -04:00
Vladimir Mandic 5ce07ef6c8 update 2021-06-16 15:47:01 -04:00
Vladimir Mandic 24181693a1 update 2021-06-16 15:46:05 -04:00
Vladimir Mandic f9f85ef4fe fix demo paths 2021-06-16 15:40:35 -04:00
Vladimir Mandic 72d02e10f1 added multithreaded demo 2021-06-14 10:23:06 -04:00
Vladimir Mandic df33ca0a31 2.0.2 2021-06-14 10:20:49 -04:00
Vladimir Mandic 2dba5ceb5b reorganize demos 2021-06-14 08:16:10 -04:00
Vladimir Mandic c07d8c2317 fix centernet box width & height 2021-06-11 16:12:24 -04:00
Vladimir Mandic eb6d344560 update todo 2021-06-09 07:27:19 -04:00
Vladimir Mandic 0bb866547a update 2021-06-09 07:19:03 -04:00
Vladimir Mandic e2f585c92c update demo menu documentation 2021-06-09 07:17:54 -04:00
Vladimir Mandic 017724ff61 update 2021-06-08 07:37:15 -04:00
Vladimir Mandic 8c9c87ffc7 add body segmentation sample 2021-06-08 07:29:08 -04:00
Vladimir Mandic 011011d29c add release notes 2021-06-08 07:09:37 -04:00
Vladimir Mandic 05ee137610 release 2.0 2021-06-08 07:06:16 -04:00
Vladimir Mandic bf1e64f4fc 2.0.1 2021-06-08 07:02:11 -04:00
Vladimir Mandic 337cb9b381 add video drag&drop capability 2021-06-07 08:38:16 -04:00
Vladimir Mandic 76db076d32 update readme 2021-06-06 20:49:48 -04:00
Vladimir Mandic e80a23c92b update packages 2021-06-06 20:47:59 -04:00
Vladimir Mandic 8b0f7c1ccb modularize build platform 2021-06-06 20:34:29 -04:00
Vladimir Mandic a4befc8106 custom build tfjs from sources 2021-06-06 19:00:34 -04:00
Vladimir Mandic 266679d7fb update wasm to tfjs 3.7.0 2021-06-06 12:58:06 -04:00
Vladimir Mandic c4a32361de update defaults 2021-06-05 20:06:36 -04:00
Vladimir Mandic 7f5e5aa00a modularize build platform 2021-06-05 17:51:46 -04:00
Vladimir Mandic bf42a1c64e enable body segmentation and background replacement in demo 2021-06-05 16:13:41 -04:00
Vladimir Mandic 9f63f39e4c minor git corruption 2021-06-05 15:23:17 -04:00
Vladimir Mandic 819b4a3f88 update 2021-06-05 15:10:28 -04:00
Vladimir Mandic 791087bb78 update 2021-06-05 13:02:01 -04:00
Vladimir Mandic fd5973b0c2 unified build 2021-06-05 12:59:11 -04:00
Vladimir Mandic a092f681f9 enable body segmentation and background replacement 2021-06-05 11:54:49 -04:00
Vladimir Mandic f0f7e00969 work on body segmentation 2021-06-04 20:22:05 -04:00
Vladimir Mandic 3c43aa57db added experimental body segmentation module 2021-06-04 13:52:40 -04:00
Vladimir Mandic c5f0ebe03f add meet and selfie models 2021-06-04 13:51:01 -04:00
Vladimir Mandic 78431fca00 update for tfjs 3.7.0 2021-06-04 09:20:59 -04:00
Vladimir Mandic 09af0460c2 update 2021-06-04 07:03:34 -04:00
Vladimir Mandic 3be72e428d update gaze strength calculations 2021-06-03 09:53:11 -04:00
Vladimir Mandic b713dd5271 update build with automatic linter 2021-06-03 09:41:53 -04:00
Vladimir Mandic f552172f64 add live hints to demo 2021-06-02 17:29:50 -04:00
Vladimir Mandic 66318c3355 switch worker from module to iife importscripts 2021-06-02 16:46:07 -04:00
Vladimir Mandic d8f80b2d81 release candidate 2021-06-02 13:39:02 -04:00
Vladimir Mandic d77d8e4942 update wiki 2021-06-02 13:35:59 -04:00
Vladimir Mandic 9d7a3148ec update tests and demos 2021-06-02 13:35:33 -04:00
Vladimir Mandic f2c937d2cd added samples to git 2021-06-02 12:44:12 -04:00
Vladimir Mandic 71cde6b67d implemented drag & drop for image processing 2021-06-02 12:43:43 -04:00
Vladimir Mandic b79e380114 release candidate 2021-06-01 08:59:09 -04:00
Vladimir Mandic e19c65526c breaking changes to results.face output properties 2021-06-01 07:37:17 -04:00
Vladimir Mandic a7f779378d breaking changes to results.object output properties 2021-06-01 07:07:01 -04:00
Vladimir Mandic 8e727302e4 breaking changes to results.hand output properties 2021-06-01 07:01:59 -04:00
Vladimir Mandic efc7e530df breaking changes to results.body output properties 2021-06-01 06:55:40 -04:00
Vladimir Mandic d71bf8bb2f update wiki 2021-05-31 10:40:24 -04:00
Vladimir Mandic 5788a80a7a implemented human.next global interpolation method 2021-05-31 10:40:07 -04:00
Vladimir Mandic 8d228ecff3 update wiki 2021-05-30 23:22:21 -04:00
Vladimir Mandic b22d92b0bc finished draw buffering and smoothing and enabled by default 2021-05-30 23:21:48 -04:00
Vladimir Mandic 3dc60c73ae update wiki 2021-05-30 18:46:23 -04:00
Vladimir Mandic 5761eb282a update typedoc definitions 2021-05-30 18:45:39 -04:00
Vladimir Mandic c9446e93cb update pwa scope 2021-05-30 18:00:51 -04:00
Vladimir Mandic 764c2fae0c implemented service worker 2021-05-30 17:56:40 -04:00
Vladimir Mandic 3e761f4801 update todo 2021-05-30 12:05:27 -04:00
Vladimir Mandic 4194d0e752 quantized centernet 2021-05-30 12:03:52 -04:00
Vladimir Mandic 22d70df905 release candidate 2021-05-30 12:03:34 -04:00
Vladimir Mandic a2793091f2 added usage restrictions 2021-05-30 09:51:23 -04:00
Vladimir Mandic 8b9ccd9670 update security policy 2021-05-30 09:41:24 -04:00
Vladimir Mandic 363e1a3370 quantize handdetect model 2021-05-29 18:29:57 -04:00
Vladimir Mandic 5916b7ff18 update todo list 2021-05-29 09:24:09 -04:00
Vladimir Mandic c980736c30 added experimental movenet-lightning and removed blazepose from default dist 2021-05-29 09:20:01 -04:00
Vladimir Mandic a8082b0815 update 2021-05-28 15:54:29 -04:00
Vladimir Mandic 1678d3192f added experimental face.rotation.gaze 2021-05-28 15:53:51 -04:00
Vladimir Mandic 54ddb2e2be fix and optimize for mobile platform 2021-05-28 10:43:48 -04:00
Vladimir Mandic bb5fe30692 lock typescript to 4.2 due to typedoc incompatibility with 4.3 2021-05-27 16:07:02 -04:00
Vladimir Mandic ade5d65d06 1.9.4 2021-05-27 16:05:20 -04:00
Vladimir Mandic 4596666c0d fix demo facecompare 2021-05-26 08:52:31 -04:00
Vladimir Mandic 7ba3f9b24e webhint and lighthouse optimizations 2021-05-26 08:47:31 -04:00
Vladimir Mandic 1af638c5c5 update 2021-05-26 07:59:52 -04:00
Vladimir Mandic da7721d580 add camera startup diag messages 2021-05-26 07:57:51 -04:00
Vladimir Mandic 4ab5bbb6e6 update all box calculations 2021-05-25 08:58:20 -04:00
Vladimir Mandic 70af13ce8c implemented unified result.persons that combines face, body and hands for each person 2021-05-24 11:10:13 -04:00
Vladimir Mandic 5e11171344 update iris distance docs 2021-05-24 07:18:03 -04:00
Vladimir Mandic edf2d896cf update iris distance calculations 2021-05-24 07:16:38 -04:00
Vladimir Mandic 4d404c7592 added experimental results interpolation for smooth draw operations 2021-05-23 13:55:33 -04:00
Vladimir Mandic fefc3123b9 1.9.3 2021-05-23 13:54:44 -04:00
Vladimir Mandic bdceeea1d8 use green weighted for input diff calculation 2021-05-23 13:54:22 -04:00
Vladimir Mandic 6e894c3baf implement experimental drawOptions.bufferedOutput and bufferedFactor 2021-05-23 13:52:49 -04:00
Vladimir Mandic 5aeb449a67 use explicit tensor interface 2021-05-22 21:54:18 -04:00
Vladimir Mandic 0fb63cea21 add tfjs types and remove all instances of any 2021-05-22 21:47:59 -04:00
Vladimir Mandic 0b907e56d9 enhance strong typing 2021-05-22 14:53:51 -04:00
Vladimir Mandic b11ae9bfae rebuild all for release 2021-05-22 13:17:07 -04:00
Vladimir Mandic 597de41751 1.9.2 2021-05-22 13:15:11 -04:00
Vladimir Mandic 4b221c17ea add id and boxraw on missing objects 2021-05-22 12:41:29 -04:00
Vladimir Mandic 714d95f6ed restructure results strong typing 2021-05-22 12:33:19 -04:00
Vladimir Mandic 618ef6f7fa update dependencies 2021-05-21 06:54:02 -04:00
Vladimir Mandic 670918ea37 1.9.1 2021-05-21 06:51:31 -04:00
Vladimir Mandic 6ced134157 caching improvements 2021-05-20 19:14:07 -04:00
Vladimir Mandic 0c741d2055 add experimental mb3-centernet object detection 2021-05-19 08:27:28 -04:00
Vladimir Mandic 5b0dd5783d individual model skipFrames values still max high threshold for caching 2021-05-18 11:38:22 -04:00
Vladimir Mandic 5741bce450 config.videoOptimized has been removed and config.cacheSensitivity has been added instead 2021-05-18 11:36:57 -04:00
Vladimir Mandic 956e4743c7 caching determination is now dynamic based on detection of input change and not based on input types 2021-05-18 11:36:24 -04:00
Vladimir Mandic 8741695dbd human 1.9.0 beta with breaking changes regarding caching 2021-05-18 11:26:16 -04:00
Vladimir Mandic 53c070e3c5 update dependencies 2021-05-18 08:22:33 -04:00
Vladimir Mandic 2cf87b8151 1.8.5 2021-05-18 08:14:56 -04:00
Vladimir Mandic 8ad1d98970 update demos 2021-05-17 09:35:41 -04:00
Vladimir Mandic 0131b10767 update 2021-05-17 08:56:57 -04:00
Vladimir Mandic 91d37adde0 add node-video sample 2021-05-16 23:55:08 -04:00
Vladimir Mandic eba2f66151 update 2021-05-11 15:08:27 -04:00
Vladimir Mandic 2a30491cb9 add node-webcam demo 2021-05-11 10:11:55 -04:00
Vladimir Mandic 4f9d5c7b09 fix node build and update model signatures 2021-05-11 07:53:06 -04:00
Vladimir Mandic a66516fa34 1.8.4 2021-05-11 07:09:44 -04:00
Vladimir Mandic 48351b1539 update & fix posenet 2021-05-05 10:07:44 -04:00
Vladimir Mandic 5546580eda 1.8.3 2021-05-05 09:55:39 -04:00
Vladimir Mandic 0120290047 switch posenet weights 2021-05-04 20:46:33 -04:00
Vladimir Mandic 31a794a8db update tfjs version 2021-05-04 11:19:46 -04:00
Vladimir Mandic 12b9cca3ae 1.8.2 2021-05-04 11:17:50 -04:00
Vladimir Mandic 16540a2f5a release 1.8 with major changes and tfjs 3.6.0 2021-04-30 11:55:04 -04:00
Vladimir Mandic be315a7a73 1.8.1 2021-04-30 11:53:56 -04:00
Vladimir Mandic ab3d8e647e update 2021-04-28 08:58:21 -04:00
Vladimir Mandic a9b9c2f254 blazeface optimizations 2021-04-28 08:55:26 -04:00
Vladimir Mandic 2546e7b7e9 add hand labels in draw 2021-04-26 07:37:29 -04:00
Vladimir Mandic b41d4783ef cleanup demo workflow 2021-04-26 07:19:30 -04:00
Vladimir Mandic 0a42d1a2d6 update browser demo defaults 2021-04-25 16:58:18 -04:00
Vladimir Mandic 88b646b283 convert blazeface to module 2021-04-25 16:56:10 -04:00
Vladimir Mandic 8678b4d57f version 1.8 release candidate 2021-04-25 14:32:55 -04:00
Vladimir Mandic 7145c04248 build NodeJS deliverables in non-minified form 2021-04-25 14:32:19 -04:00
Vladimir Mandic 8ecaea5d3b stop building sourcemaps for NodeJS deliverables 2021-04-25 14:32:07 -04:00
Vladimir Mandic 08baeb8768 remove deallocate, profile, scoped 2021-04-25 14:31:51 -04:00
Vladimir Mandic 211c3f9a48 replaced maxFaces, maxDetections, maxHands, maxResults with maxDetected 2021-04-25 14:31:39 -04:00
Vladimir Mandic 53d5ecbb75 replaced nmsRadius with built-in default 2021-04-25 14:31:26 -04:00
Vladimir Mandic db39ac03c0 unified minConfidence and scoreThresdold as minConfidence 2021-04-25 14:31:13 -04:00
Vladimir Mandic 266f547a88 add exception handlers to all demos 2021-04-25 14:30:40 -04:00
Vladimir Mandic 5e089f1f12 remove blazeface-front and add unhandledrejection handler 2021-04-25 14:15:38 -04:00
Vladimir Mandic 3c11ef4189 major update for 1.8 release candidate 2021-04-25 13:16:04 -04:00
Vladimir Mandic 439a7af2e4 enable webworker detection 2021-04-25 07:51:01 -04:00
Vladimir Mandic 25d8396082 1.7.1 2021-04-25 07:50:12 -04:00
Vladimir Mandic 5bf05e379f remove obsolete binary models 2021-04-24 18:46:22 -04:00
Vladimir Mandic 437f2f914e enable cross origin isolation 2021-04-24 18:43:59 -04:00
Vladimir Mandic 4dba6b29cb rewrite posenet decoder 2021-04-24 16:04:49 -04:00
Vladimir Mandic 970660c3bf update demo node decoder 2021-04-24 12:12:10 -04:00
Vladimir Mandic e060f1947a update posenet model 2021-04-24 11:49:26 -04:00
Vladimir Mandic 69ba00f535 remove efficientpose 2021-04-24 09:31:46 -04:00
Vladimir Mandic b40d5deca7 major version rebuild 2021-04-22 19:47:04 -04:00
Vladimir Mandic 1411c7153d 1.6.1 2021-04-22 19:46:29 -04:00
Vladimir Mandic 403493fff6 update for tfjs 3.5.0 2021-04-22 19:46:18 -04:00
Vladimir Mandic 1300dc03e1 update todo 2021-04-22 18:39:29 -04:00
Vladimir Mandic 5cb05205a7 add npmrc 2021-04-20 08:02:21 -04:00
Vladimir Mandic a363a08633 update 2021-04-19 16:19:03 -04:00
Vladimir Mandic c4a353787a added filter.flip feature 2021-04-19 16:02:47 -04:00
Vladimir Mandic 22cf040972 update todo 2021-04-19 11:36:44 -04:00
Vladimir Mandic 589861ddef update 2021-04-19 11:24:17 -04:00
Vladimir Mandic 9e1041d9d7 update 2021-04-19 11:20:24 -04:00
Vladimir Mandic af54748ccf update node demo 2021-04-19 11:17:55 -04:00
Vladimir Mandic 0fd7995542 added demo load image from http 2021-04-19 11:15:29 -04:00
Vladimir Mandic d270a45492 update gestures 2021-04-19 09:30:04 -04:00
Vladimir Mandic e0666cebe2 mobile demo optimization and iris gestures 2021-04-18 19:33:40 -04:00
Vladimir Mandic e2aff03f71 full rebuild 2021-04-16 18:03:15 -04:00
Vladimir Mandic 093e50b305 new look 2021-04-16 18:00:24 -04:00
Vladimir Mandic 3eab916003 added benchmarks 2021-04-16 09:31:58 -04:00
Vladimir Mandic 6745c5c3d9 added node-multiprocess demo 2021-04-16 08:34:16 -04:00
Vladimir Mandic da92d53667 update 2021-04-15 17:13:27 -04:00
Vladimir Mandic 97fe47df86 fix image orientation 2021-04-15 15:26:31 -04:00
Vladimir Mandic c1a087ae92 flat app style 2021-04-15 15:01:27 -04:00
Vladimir Mandic 82730dda1d add full nodejs test coverage 2021-04-15 09:43:55 -04:00
Vladimir Mandic f625e1e7ef 1.5.2 2021-04-14 12:53:12 -04:00
Vladimir Mandic e42465d004 update tfjs 3.4.0 2021-04-14 12:53:00 -04:00
Vladimir Mandic e231e68d2d experimental node-wasm support 2021-04-13 21:45:45 -04:00
Vladimir Mandic 3618e831cc 1.5.1 2021-04-13 11:32:26 -04:00
Vladimir Mandic 43f564a978 fix for safari imagebitmap 2021-04-13 11:32:22 -04:00
Vladimir Mandic 0af73ab567 refactored human.config and human.draw 2021-04-13 11:05:52 -04:00
Vladimir Mandic 8ab49c7440 1.4.3 2021-04-12 17:49:14 -04:00
Vladimir Mandic de2b82e45d implement webrtc 2021-04-12 17:48:59 -04:00
Vladimir Mandic 9aa295e714 1.4.2 2021-04-12 08:29:58 -04:00
Vladimir Mandic 6e0fe556e8 added support for multiple instances of human 2021-04-12 08:29:52 -04:00
Vladimir Mandic f4f73e52fd update 2021-04-10 23:37:44 -04:00
Vladimir Mandic 485e122472 update cdn links 2021-04-10 23:16:06 -04:00
Vladimir Mandic dbab940f55 update 2021-04-09 21:58:45 -04:00
Vladimir Mandic 03d6be4e4c fix typedoc 2021-04-09 21:53:48 -04:00
Vladimir Mandic e8dffde291 update readme 2021-04-09 21:31:53 -04:00
Vladimir Mandic 0e20bfe665 exception handling 2021-04-09 10:02:40 -04:00
Vladimir Mandic e1f285f314 1.4.1 2021-04-09 08:08:05 -04:00
Vladimir Mandic b572858b97 add modelBasePath option 2021-04-09 08:07:58 -04:00
Vladimir Mandic 3cc1a8201f update badges 2021-04-08 19:16:17 -04:00
Vladimir Mandic 7ea4eec64e update cdn links 2021-04-08 18:37:58 -04:00
Vladimir Mandic 0788b049b3 update 2021-04-08 12:10:15 -04:00
Vladimir Mandic d886f86021 1.3.5 2021-04-06 11:38:07 -04:00
Vladimir Mandic 7022c265a7 update tslib 2021-04-06 11:38:01 -04:00
Vladimir Mandic 320638ba41 update wiki 2021-04-06 07:45:44 -04:00
Vladimir Mandic 9b5ced8f62 add dynamic viewport and fix web worker 2021-04-05 11:48:24 -04:00
Vladimir Mandic 3cf07dc501 add cdn links 2021-04-05 09:35:56 -04:00
Vladimir Mandic 2d263bd2d4 1.3.4 2021-04-04 09:26:35 -04:00
Vladimir Mandic 6958539708 update 2021-04-04 09:26:32 -04:00
Vladimir Mandic e9bd519bc3 implement webhint 2021-04-04 09:25:18 -04:00
Vladimir Mandic 6723c2a322 update 2021-04-03 11:36:53 -04:00
Vladimir Mandic 6d07e937a8 update keywords 2021-04-03 11:10:07 -04:00
Vladimir Mandic d615ef1e6e 1.3.3 2021-04-03 10:49:30 -04:00
Vladimir Mandic b3e8747126 fix linting and tests 2021-04-03 10:49:14 -04:00
Vladimir Mandic cb3014edc1 1.3.2 2021-04-02 08:39:50 -04:00
Vladimir Mandic 34f01f2053 input type validation 2021-04-02 08:37:35 -04:00
Vladimir Mandic e78ecaaf18 update 2021-04-02 08:15:39 -04:00
Vladimir Mandic 1aa7ea2db6 update 2021-04-02 08:09:06 -04:00
Vladimir Mandic abf0ba65c9 update 2021-04-02 08:06:28 -04:00
Vladimir Mandic 0e27370b8c update demos 2021-04-02 08:05:19 -04:00
Vladimir Mandic 5931a6f541 normalize all scores 2021-04-01 09:24:56 -04:00
Vladimir Mandic 942fa18b52 1.3.1 2021-03-30 09:04:51 -04:00
Vladimir Mandic 8ffec55b4a update 2021-03-30 09:04:49 -04:00
Vladimir Mandic 49a9717b90 added face3d demo 2021-03-30 09:03:18 -04:00
Vladimir Mandic b4c4364953 initial work on face3d three.js demo 2021-03-29 15:59:16 -04:00
Vladimir Mandic fd3b67851b enable buffering 2021-03-29 15:05:14 -04:00
Vladimir Mandic db6aa75ddb new icons 2021-03-29 15:01:16 -04:00
Vladimir Mandic 1d715dfead new serve module and demo structure 2021-03-29 14:40:34 -04:00
Vladimir Mandic 6d978c0b9b move gl flags to correct location 2021-03-28 13:22:22 -04:00
Vladimir Mandic 957872e5a1 minor rotation calculation fix 2021-03-28 08:49:56 -04:00
Vladimir Mandic 9cdae38de1 remove debug output 2021-03-28 08:44:53 -04:00
Vladimir Mandic 491ebe018f new face rotation calculations 2021-03-28 08:40:39 -04:00
ButzYung e8d082f6ad cleanup 2021-03-28 07:32:31 -04:00
ButzYung 395b72680b rotationMatrixToEulerAngle, and fixes 2021-03-28 07:32:31 -04:00
ButzYung 749fc56646 face rotation matrix 2021-03-28 07:32:31 -04:00
Vladimir Mandic 5653b4577d update 2021-03-27 15:45:37 -04:00
Vladimir Mandic 8b9374c09b experimental: add efficientpose 2021-03-27 15:43:48 -04:00
Vladimir Mandic 9bb0769637 implement nanodet 2021-03-27 10:25:31 -04:00
Vladimir Mandic dc7abbb459 start working on efficientpose 2021-03-26 18:50:19 -04:00
Vladimir Mandic f1ae581c7b update contributing guide 2021-03-26 15:37:43 -04:00
Vladimir Mandic a63e33042e update contributing guidelines 2021-03-26 12:37:08 -04:00
Vladimir Mandic 93d318aff1 update 2021-03-25 08:50:41 -04:00
Vladimir Mandic 32e634306a 1.2.5 2021-03-25 08:44:04 -04:00
Vladimir Mandic 58781b8626 fix broken exports 2021-03-25 08:43:51 -04:00
Vladimir Mandic 7ca8f5c448 update faces database 2021-03-24 11:43:28 -04:00
Vladimir Mandic cece9f71fc updated face description 2021-03-24 11:08:49 -04:00
Vladimir Mandic f296ed1427 added face matching example to docs 2021-03-23 15:35:54 -04:00
Vladimir Mandic f50ff300c8 update 2021-03-23 15:25:43 -04:00
Vladimir Mandic 13ea73e650 improve fact matching 2021-03-23 15:24:58 -04:00
Vladimir Mandic 125381a67f 1.2.4 2021-03-23 14:46:50 -04:00
Vladimir Mandic 272985bb25 update nanodet and face rotation check 2021-03-23 14:46:44 -04:00
Vladimir Mandic 0d27b59f63 1.2.3 2021-03-21 17:47:05 -04:00
Vladimir Mandic ca511a5385 update demos 2021-03-21 17:47:00 -04:00
Vladimir Mandic bf89c7ee98 1.2.2 2021-03-21 16:16:17 -04:00
Vladimir Mandic 05ccdc3eb0 precise face rotation 2021-03-21 16:16:13 -04:00
Vladimir Mandic f78957af02 update 2021-03-21 14:24:10 -04:00
Vladimir Mandic 8301f49db7 1.2.1 2021-03-21 14:21:47 -04:00
Vladimir Mandic 3193811a63 update wiki 2021-03-21 14:19:04 -04:00
Vladimir Mandic 64adb7ebd8 new module: face description 2021-03-21 14:18:51 -04:00
Vladimir Mandic 79a434b384 1.1.11 2021-03-21 07:49:58 -04:00
Vladimir Mandic 0e15a5e846 refactor face classes 2021-03-21 07:49:55 -04:00
Vladimir Mandic 2829608a0e 1.1.10 2021-03-18 16:55:04 -04:00
Vladimir Mandic 4e7aef79e4 cleanup 2021-03-18 16:55:00 -04:00
Vladimir Mandic c31bc21d80 update github templates 2021-03-18 11:58:46 -04:00
Vladimir Mandic 61db8f0754 update 2021-03-18 07:40:08 -04:00
Vladimir Mandic 18e1712864 update keywords 2021-03-18 07:14:24 -04:00
Vladimir Mandic 13a2472f75 update 2021-03-17 20:26:43 -04:00
Vladimir Mandic c1c6b66315 redefine tensor 2021-03-17 20:23:12 -04:00
Vladimir Mandic de0fffbfdc enforce types 2021-03-17 20:16:40 -04:00
Vladimir Mandic d7571891e7 regen type declarations 2021-03-17 18:57:00 -04:00
Vladimir Mandic 6f8ee4db53 switch to single jumbo dts 2021-03-17 18:48:02 -04:00
Vladimir Mandic 72329a42a9 update node demo 2021-03-17 18:36:12 -04:00
Vladimir Mandic 2464b8c0ef update 2021-03-17 18:33:12 -04:00
Vladimir Mandic b0ed511405 type definitions 2021-03-17 18:23:19 -04:00
Vladimir Mandic f796d39e60 update package docs 2021-03-17 14:45:15 -04:00
Vladimir Mandic 57343739b6 1.1.9 2021-03-17 14:35:23 -04:00
Vladimir Mandic 8523da07b5 fix box clamping and raw output 2021-03-17 14:35:11 -04:00
Vladimir Mandic 4819930e43 update readme 2021-03-17 12:03:36 -04:00
Vladimir Mandic aa79cd3b20 hierarchical readme notes 2021-03-17 12:01:54 -04:00
Vladimir Mandic 6c3daeeff1 update readme 2021-03-17 11:48:34 -04:00
Vladimir Mandic 1a92741bac update 2021-03-17 11:41:57 -04:00
Vladimir Mandic f95d097ef0 1.1.8 2021-03-17 11:40:35 -04:00
Vladimir Mandic 7738c2ad3d update 2021-03-17 11:40:31 -04:00
Vladimir Mandic c356b02625 add experimental nanodet object detection 2021-03-17 11:32:37 -04:00
Vladimir Mandic a0cf463f03 full models signature 2021-03-17 09:01:59 -04:00
Vladimir Mandic b92474954a 1.1.7 2021-03-16 07:16:28 -04:00
Vladimir Mandic 8313cc9092 fix for seedrandom 2021-03-16 07:16:25 -04:00
Vladimir Mandic 733c124afe update todo 2021-03-15 12:31:23 -04:00
Vladimir Mandic 0d8e0f9316 custom typedoc 2021-03-15 12:29:51 -04:00
Vladimir Mandic 295148fc0f 1.1.6 2021-03-15 12:14:52 -04:00
Vladimir Mandic 5e082a5294 implement human.match and embedding demo 2021-03-15 12:14:48 -04:00
Vladimir Mandic ea02afb813 1.1.5 2021-03-15 08:56:45 -04:00
Vladimir Mandic 2d844828c0 full rebuild 2021-03-15 08:56:36 -04:00
Vladimir Mandic 7f31cee9d6 update 2021-03-15 08:52:28 -04:00
Vladimir Mandic 87c3171712 1.1.4 2021-03-14 13:49:05 -04:00
Vladimir Mandic 6294882ea8 fix broken build 2021-03-14 13:49:01 -04:00
Vladimir Mandic 6c6f2bebac 1.1.3 2021-03-14 13:39:52 -04:00
Vladimir Mandic 1da76b4adc update 2021-03-14 13:39:47 -04:00
Vladimir Mandic 71368d3889 added api specs 2021-03-13 22:38:35 -05:00
Vladimir Mandic 43d283125b add typedocs and types 2021-03-13 22:31:09 -05:00
Vladimir Mandic fc8e5bf258 strong typings 2021-03-13 13:47:45 -05:00
Vladimir Mandic da656f62a0 update 2021-03-13 12:30:03 -05:00
Vladimir Mandic c2fd74d097 update docs 2021-03-13 12:13:45 -05:00
Vladimir Mandic 588fa49d4c update 2021-03-13 11:37:16 -05:00
Vladimir Mandic 896cb0aac0 update embedding and strong typings 2021-03-13 11:26:53 -05:00
Vladimir Mandic 092873adaa 1.1.2 2021-03-12 18:24:38 -05:00
Vladimir Mandic 75f3e3c269 distance based on minkowski space and limited euclidean space 2021-03-12 18:24:34 -05:00
Vladimir Mandic 8de79a92b2 guard against invalid input images 2021-03-12 16:43:36 -05:00
Vladimir Mandic f8703e3dc0 1.1.1 2021-03-12 12:54:34 -05:00
Vladimir Mandic ca400cd45d update wiki 2021-03-12 12:54:30 -05:00
Vladimir Mandic 12b0058a1b switched face embedding to mobileface 2021-03-12 12:54:08 -05:00
Vladimir Mandic 162ace9fc3 updated docs 2021-03-11 22:11:49 -05:00
Vladimir Mandic dc4fae1169 1.0.4 2021-03-11 22:04:54 -05:00
Vladimir Mandic e4574d9fdf add face return tensor 2021-03-11 22:04:44 -05:00
Vladimir Mandic c3ecdf5486 add test for face descriptors 2021-03-11 18:26:04 -05:00
Vladimir Mandic 622de45e50 wip on embedding 2021-03-11 13:31:36 -05:00
Vladimir Mandic 1c921294ba simplify face box coordinate calculations 2021-03-11 11:44:22 -05:00
Vladimir Mandic fac2b6b544 annotated models and removed gender-ssrnet 2021-03-11 10:30:20 -05:00
Vladimir Mandic d5b620dbe8 autodetect inputSizes 2021-03-11 10:26:14 -05:00
Vladimir Mandic 13285582d6 update todo 2021-03-10 10:54:39 -05:00
Vladimir Mandic 8a7b0e5f90 update 2021-03-10 10:28:20 -05:00
Vladimir Mandic 3708cf8541 update todo 2021-03-10 10:12:39 -05:00
Vladimir Mandic d511aa0a8b 1.0.3 2021-03-10 10:02:55 -05:00
Vladimir Mandic 3b984fdd35 strong typing for public classes and hide private classes 2021-03-10 10:02:52 -05:00
Vladimir Mandic 74ad5a2837 enhanced age, gender, emotion detection 2021-03-10 09:44:45 -05:00
Vladimir Mandic 02696a65b3 full rebuild 2021-03-09 18:36:26 -05:00
Vladimir Mandic a5ef2081de 1.0.2 2021-03-09 18:34:43 -05:00
Vladimir Mandic 7d512d8327 update tfjs and esbuild 2021-03-09 18:34:04 -05:00
Vladimir Mandic 1abbd2c909 remove blazeface-front, blazepose-upper, faceboxes 2021-03-09 18:33:50 -05:00
Vladimir Mandic f8b13a8bba remove blazeface-front and faceboxes 2021-03-09 18:32:35 -05:00
Vladimir Mandic 47ff4f8d40 update 2021-03-09 13:37:49 -05:00
Vladimir Mandic 8f0403073c update 2021-03-09 13:19:52 -05:00
Vladimir Mandic cc466f5554 update 2021-03-09 13:18:08 -05:00
Vladimir Mandic 16ecba7f75 1.0.1 2021-03-09 13:15:59 -05:00
Vladimir Mandic 9fd20393be fix for face detector when mesh is disabled 2021-03-09 13:15:40 -05:00
Vladimir Mandic 8e9f78c6d7 update badges 2021-03-08 15:06:56 -05:00
Vladimir Mandic 05b37e0266 optimize for npm 2021-03-08 14:12:12 -05:00
Vladimir Mandic 16bd0d227f 0.40.9 2021-03-08 10:06:40 -05:00
Vladimir Mandic f4bc35c3b3 fix performance issue when running with low confidence 2021-03-08 10:06:34 -05:00
Vladimir Mandic b02a2962f3 0.40.8 2021-03-08 07:32:30 -05:00
Vladimir Mandic 36451ea918 update docs and demo 2021-03-08 07:32:24 -05:00
Vladimir Mandic 14a430edd4 0.40.7 2021-03-06 17:23:36 -05:00
Vladimir Mandic 0813fc0e00 update 2021-03-06 17:23:24 -05:00
Vladimir Mandic cd949deec9 implemented 3d face angle calculations 2021-03-06 17:22:47 -05:00
Vladimir Mandic 0d7cb8d8ea 0.40.6 2021-03-06 10:38:22 -05:00
Vladimir Mandic 18a4789639 add curve draw output 2021-03-06 10:38:04 -05:00
Vladimir Mandic 0316725a01 update 2021-03-05 14:42:32 -05:00
Vladimir Mandic 157ceb5bec update readme 2021-03-05 14:40:44 -05:00
Vladimir Mandic a7ad592693 update 2021-03-05 14:31:06 -05:00
Vladimir Mandic 86f68b4e7d 0.40.5 2021-03-05 14:30:46 -05:00
Vladimir Mandic de2ff38ff8 fix human.draw 2021-03-05 14:30:09 -05:00
Vladimir Mandic 48f072308e 0.40.4 2021-03-05 11:44:00 -05:00
Vladimir Mandic c2e74d2ba1 update human.draw helper methods 2021-03-05 11:43:50 -05:00
Vladimir Mandic 388c968e9c fix demo 2021-03-05 07:45:30 -05:00
Vladimir Mandic ee20490791 0.40.3 2021-03-05 07:45:20 -05:00
Vladimir Mandic 05084df8ad 0.40.2 2021-03-05 07:39:47 -05:00
Vladimir Mandic 95519c45b7 added blazepose-upper 2021-03-05 07:39:37 -05:00
Vladimir Mandic e55a8de6b0 0.40.1 2021-03-04 10:33:18 -05:00
Vladimir Mandic 75161c9127 implement blazepose and update demos 2021-03-04 10:33:08 -05:00
Vladimir Mandic 8ccc8f7f47 update 2021-03-03 12:10:44 -05:00
Vladimir Mandic 83a74f4149 add todo list 2021-03-03 12:04:59 -05:00
Vladimir Mandic c6837fc608 update 2021-03-03 10:00:14 -05:00
Vladimir Mandic 074a078396 0.30.6 2021-03-03 09:59:31 -05:00
Vladimir Mandic b77b98e8d4 fine tuning age and face models 2021-03-03 09:59:04 -05:00
Vladimir Mandic 4eb6fa709c 0.30.5 2021-03-02 11:27:47 -05:00
Vladimir Mandic 7969623849 add debug logging flag 2021-03-02 11:27:42 -05:00
Vladimir Mandic 28d00d997f 0.30.4 2021-03-01 17:20:06 -05:00
Vladimir Mandic 278cc818af added skipInitial flag 2021-03-01 17:20:02 -05:00
Vladimir Mandic 0cf856d73b update 2021-02-28 07:39:48 -05:00
Vladimir Mandic cc489e860e 0.30.3 2021-02-28 07:39:29 -05:00
Vladimir Mandic f44f0bb0e5 update 2021-02-28 07:38:13 -05:00
meeki007 6dc288bff6 typo 2021-02-28 07:37:37 -05:00
Vladimir Mandic b448f71260 update 2021-02-26 10:13:31 -05:00
Vladimir Mandic fc20b3f48f 0.30.2 2021-02-26 10:13:10 -05:00
Vladimir Mandic 7654f52c28 update 2021-02-26 10:05:56 -05:00
Vladimir Mandic b7e0674abe rebuild 2021-02-26 09:04:15 -05:00
meeki007 2afb1887d4 fix typo 2021-02-26 08:59:12 -05:00
Vladimir Mandic 8ac0176d90 update 2021-02-25 07:51:26 -05:00
Vladimir Mandic fea37e8b15 0.30.1 2021-02-25 07:51:00 -05:00
Vladimir Mandic d1ef671f89 update to tfjs 3.2.0 2021-02-25 07:50:13 -05:00
Vladimir Mandic e48d7526c7 0.20.11 2021-02-24 09:57:39 -05:00
Vladimir Mandic 1fcbd5d950 update default gender model 2021-02-24 09:57:33 -05:00
Vladimir Mandic 97feba5e28 update 2021-02-22 09:14:10 -05:00
Vladimir Mandic c41fbd706a 0.20.10 2021-02-22 09:13:16 -05:00
Vladimir Mandic 87e0559706 updated model defaults 2021-02-22 09:13:11 -05:00
Vladimir Mandic 7bfe845d03 update 2021-02-21 21:49:59 -05:00
Vladimir Mandic cc836bd21c 0.20.9 2021-02-21 21:49:36 -05:00
Vladimir Mandic 8a8d34dfd3 update 2021-02-21 14:50:03 -05:00
Vladimir Mandic 191298f408 0.20.8 2021-02-21 14:46:56 -05:00
Vladimir Mandic 4eacad24ae update 2021-02-21 14:07:04 -05:00
Vladimir Mandic d1322124fe 0.20.7 2021-02-21 14:06:35 -05:00
Vladimir Mandic 8aa92fad2c build fix 2021-02-21 14:06:13 -05:00
Vladimir Mandic 3b29de4de8 0.20.6 2021-02-21 13:34:30 -05:00
Vladimir Mandic a6f8c8951e embedding fix 2021-02-21 13:34:26 -05:00
Vladimir Mandic 9a7b143269 update wiki 2021-02-21 08:19:55 -05:00
Vladimir Mandic bf030ddc9b 0.20.5 2021-02-21 07:21:03 -05:00
Vladimir Mandic 1a5d5cb37e fix imagefx and add dev builds 2021-02-21 07:20:58 -05:00
Vladimir Mandic 56feb455a1 update 2021-02-19 08:36:47 -05:00
Vladimir Mandic cbe5573ca5 0.20.4 2021-02-19 08:35:48 -05:00
Vladimir Mandic e24882f472 update imagefx 2021-02-19 08:35:41 -05:00
Vladimir Mandic be8cb4431d update 2021-02-17 10:23:21 -05:00
Vladimir Mandic b07eec1829 0.20.3 2021-02-17 10:22:49 -05:00
Vladimir Mandic 3ca1773e12 update tfjs to 3.1.0 2021-02-17 10:22:38 -05:00
Vladimir Mandic eae3fb89b2 rebuild 2021-02-13 09:22:40 -05:00
Vladimir Mandic deba687c79 0.20.2 2021-02-13 09:22:10 -05:00
Vladimir Mandic a8ccee95bf update lint rules 2021-02-13 09:21:48 -05:00
Vladimir Mandic 17c9aa6583 updated lint rules 2021-02-13 09:16:41 -05:00
Vladimir Mandic 36058ac875 update dev server 2021-02-13 08:42:10 -05:00
Vladimir Mandic e8281bc48d Merge branch 'main' of https://github.com/vladmandic/human into main 2021-02-08 13:30:52 -05:00
Vladimir Mandic 79e4ab4a86 update readme 2021-02-08 13:30:49 -05:00
Vladimir Mandic 4a660c6204 Create codeql-analysis.yml 2021-02-08 13:29:58 -05:00
Vladimir Mandic 21bba7a486 Create SECURITY.md 2021-02-08 13:28:32 -05:00
Vladimir Mandic 238f527244 update template 2021-02-08 13:25:28 -05:00
Vladimir Mandic bffc34edc2 add templates 2021-02-08 13:24:23 -05:00
Vladimir Mandic ee00d0fbd8 update default github docs 2021-02-08 13:20:37 -05:00
Vladimir Mandic 905c1ac31a update 2021-02-08 13:10:10 -05:00
Vladimir Mandic c546458172 0.20.1 2021-02-08 13:09:36 -05:00
Vladimir Mandic 47a17adcdf menu fixes 2021-02-08 13:07:49 -05:00
Vladimir Mandic 402ef8ae32 updated typings 2021-02-08 12:47:38 -05:00
Vladimir Mandic 3824deba29 convert to typescript 2021-02-08 11:39:09 -05:00
Vladimir Mandic d373ee088c update 2021-02-06 17:42:47 -05:00
Vladimir Mandic 5aea22fc4b 0.11.5 2021-02-06 17:42:21 -05:00
Vladimir Mandic 60b817ba4c added faceboxes alternative model 2021-02-06 17:41:53 -05:00
Vladimir Mandic 70f9ee05ad 0.11.4 2021-02-06 10:19:45 -05:00
Vladimir Mandic dc33353bd3 update 2021-02-06 10:19:45 -05:00
Vladimir Mandic fc118c7e06 0.11.3 2021-02-02 20:35:15 -05:00
Vladimir Mandic 3efb713855 update 2021-02-02 20:35:14 -05:00
Vladimir Mandic 7dfcef0023 update wiki 2021-01-30 13:24:06 -05:00
Vladimir Mandic 1a7da68c4e 0.11.2 2021-01-30 13:23:26 -05:00
Vladimir Mandic fbd1eb94c2 added warmup for nodejs 2021-01-30 13:23:07 -05:00
Vladimir Mandic 49137f72d4 update for tfjs 3.0.0 2021-01-29 10:26:58 -05:00
Vladimir Mandic 0faa021bb1 0.11.1 2021-01-29 10:25:26 -05:00
Vladimir Mandic 5d6a123f0d 0.10.2 2021-01-22 08:15:34 -05:00
Vladimir Mandic 7e1a1392ae update 2021-01-22 08:15:33 -05:00
Vladimir Mandic c17311e3d1 update to tfjs 2.8.5 2021-01-20 08:05:30 -05:00
Vladimir Mandic 205855d583 0.10.1 2021-01-20 08:04:59 -05:00
Vladimir Mandic fdead00263 update 2021-01-18 08:22:51 -05:00
Vladimir Mandic 510f9a39f6 0.9.26 2021-01-18 08:22:35 -05:00
Vladimir Mandic 01e3eec30e fix face detection when mesh is disabled 2021-01-18 08:22:25 -05:00
Vladimir Mandic 0231e12d2b version bump 2021-01-13 11:14:23 -05:00
Vladimir Mandic dcab4a6c9c 0.9.25 2021-01-13 11:13:43 -05:00
Vladimir Mandic 46cf1694bf update 2021-01-13 09:40:04 -05:00
Vladimir Mandic 2a041305e9 added humangl custom backend 2021-01-13 09:35:31 -05:00
Vladimir Mandic 27159a59dc rebuild 2021-01-12 10:18:58 -05:00
Vladimir Mandic 61c4d7ae6d code cleanup and enable minification 2021-01-12 09:55:08 -05:00
Vladimir Mandic 44ae1b517a fix safari incopatibility 2021-01-12 08:24:00 -05:00
Vladimir Mandic caa414835f 0.9.24 2021-01-12 08:23:19 -05:00
Vladimir Mandic 872fb5fdb2 work on blazepose 2021-01-11 14:35:57 -05:00
Vladimir Mandic 520b6b4af1 update changelog 2021-01-11 09:02:55 -05:00
Vladimir Mandic 499ed87323 full rebuild 2021-01-11 09:02:43 -05:00
Vladimir Mandic 35d1de7d41 0.9.23 2021-01-11 09:02:15 -05:00
Vladimir Mandic 98bd96426a added iris gesture 2021-01-11 09:02:02 -05:00
Vladimir Mandic c0fd223bcb update 2021-01-06 06:52:01 -05:00
Vladimir Mandic 049facf72b fix emotion labels 2021-01-06 06:51:20 -05:00
Vladimir Mandic f586b8d3b1 full rebuild 2021-01-05 16:50:25 -05:00
Vladimir Mandic a79fe751e8 0.9.22 2021-01-05 16:49:55 -05:00
Vladimir Mandic 113d8c1ca7 update wiki 2021-01-05 16:48:19 -05:00
Vladimir Mandic 6ba9cda7bb remove iris coords if iris is disabled 2021-01-05 16:41:54 -05:00
Vladimir Mandic f7a51349bf update iris objects 2021-01-05 16:33:43 -05:00
Vladimir Mandic e4a85fcb85 web worker fix 2021-01-03 10:41:56 -05:00
Vladimir Mandic 4fb51ad583 0.9.21 2021-01-03 10:41:27 -05:00
Vladimir Mandic bf1dd37e35 0.9.20 2021-01-03 10:23:49 -05:00
Vladimir Mandic 59e465c158 update to tfjs 2.8.2 2021-01-03 10:23:45 -05:00
Vladimir Mandic 79ac90f684 stricter linting, fix face annotations 2020-12-27 08:12:22 -05:00
Vladimir Mandic 2f3b952c28 update nodejs platform support 2020-12-23 13:55:22 -05:00
Vladimir Mandic 1556b8af85 0.9.19 2020-12-23 13:54:47 -05:00
Vladimir Mandic a85cfcf37d added rawBox and rawMesh 2020-12-22 12:34:47 -05:00
ButzYung 94fb9408ad Variable name changes, setting .rawCoords only if necessary 2020-12-22 12:20:36 -05:00
ButzYung 2eb0578501 Option to return raw data (mesh, box) for Facemesh / "preserve aspect ratio" fix from Facemesh upstream 2020-12-22 12:20:36 -05:00
Vladimir Mandic 8451446d2b updated readme 2020-12-21 08:12:18 -05:00
Vladimir Mandic a472540b36 update 2020-12-17 21:09:25 -05:00
Vladimir Mandic 7a968fcdc6 0.9.18 2020-12-16 19:17:04 -05:00
Vladimir Mandic 253030830f add z axis scaling 2020-12-16 19:16:54 -05:00
Vladimir Mandic 641d56547d major work on body module 2020-12-16 18:36:24 -05:00
Vladimir Mandic ab87288e7d republish due to tfjs 2.8.0 issues 2020-12-16 14:49:14 -05:00
Vladimir Mandic 31f728e502 0.9.17 2020-12-15 08:44:45 -05:00
Vladimir Mandic 3902a5e25d updated tfjs 2020-12-15 08:44:42 -05:00
Vladimir Mandic ff058a4148 updated 2020-12-12 18:35:16 -05:00
Vladimir Mandic 5921d14bfe added custom webgl backend 2020-12-12 18:34:30 -05:00
Vladimir Mandic 7f42a89346 0.9.16 2020-12-12 11:25:13 -05:00
Vladimir Mandic 161ae5fa6a update dependencies 2020-12-12 11:25:12 -05:00
Vladimir Mandic c1a92d07f3 change default ports 2020-12-12 10:15:51 -05:00
Vladimir Mandic a2ec69ec65 0.9.15 2020-12-11 10:11:54 -05:00
Vladimir Mandic 4d43356d73 improved caching and warmup 2020-12-11 10:11:49 -05:00
Vladimir Mandic c85c8fa286 rebuild 2020-12-10 15:47:43 -05:00
Vladimir Mandic f54fef7d3f 0.9.14 2020-12-10 15:47:06 -05:00
Vladimir Mandic a1aaa44e8d conditional hand rotation 2020-12-10 15:46:45 -05:00
Vladimir Mandic 8bc1318ca8 updated wiki 2020-12-10 14:48:12 -05:00
Vladimir Mandic 15004506d4 staggered skipframes 2020-12-10 14:47:53 -05:00
Vladimir Mandic 58e0822582 0.9.13 2020-12-08 10:54:22 -05:00
Vladimir Mandic 5c6e610a37 implemented face and hand boundary checks 2020-12-08 10:50:26 -05:00
Vladimir Mandic 7a0e5a555d embedded sample for warmup 2020-12-08 09:58:30 -05:00
Vladimir Mandic f65effc69d switch to central logger 2020-12-08 09:00:44 -05:00
Vladimir Mandic 3c54914a5c 0.9.12 2020-11-26 10:37:08 -05:00
Vladimir Mandic 40e0f1c4c4 minor compatibility fixes 2020-11-26 10:37:04 -05:00
Vladimir Mandic e1b3fff07c update for node v15 2020-11-25 09:13:19 -05:00
Vladimir Mandic 6777aa54dd 0.9.11 2020-11-23 23:42:42 -05:00
Vladimir Mandic 4679b859b8 implement multi-person gestures 2020-11-23 23:36:04 -05:00
Vladimir Mandic 2978b319c3 modularize pipeline models 2020-11-23 22:55:01 -05:00
Vladimir Mandic 70503ef152 2020-11-23 08:44:34 -05:00
Vladimir Mandic 578f151eee updated embedding function 2020-11-23 08:40:17 -05:00
Vladimir Mandic 604e5632a8 updated wiki 2020-11-23 08:18:08 -05:00
Vladimir Mandic 999c45c622 updated firefox css styling 2020-11-23 07:44:10 -05:00
Vladimir Mandic 7803d18a8c 0.9.10 2020-11-21 12:22:00 -05:00
Vladimir Mandic 853745512e changed build for optimized node & browser 2020-11-21 12:21:47 -05:00
Vladimir Mandic 5373809370 updated font 2020-11-21 07:33:41 -05:00
Vladimir Mandic 8528aa130f 0.9.9 2020-11-21 07:20:17 -05:00
Vladimir Mandic 80c545b598 new screenshots 2020-11-21 07:19:20 -05:00
Vladimir Mandic 097458610e update dependencies 2020-11-20 08:53:40 -05:00
Vladimir Mandic a84c26c781 camera exception handling 2020-11-20 07:52:50 -05:00
Vladimir Mandic 05042392be 0.9.8 2020-11-19 16:16:58 -05:00
Vladimir Mandic 41f7386583 force f16 textures 2020-11-19 16:16:51 -05:00
Vladimir Mandic 899d8be570 bugfix embedding check 2020-11-19 15:22:08 -05:00
Vladimir Mandic 43eebb73ad 0.9.7 2020-11-19 14:46:08 -05:00
Vladimir Mandic ef13c1e560 ui redesign 2020-11-19 14:45:59 -05:00
Vladimir Mandic 4f0ecf388b 0.9.6 2020-11-18 09:15:22 -05:00
Vladimir Mandic 7f5007851e optimize camera resize on mobile 2020-11-18 09:15:03 -05:00
Vladimir Mandic adcb55c83a completed tfjs wrapper 2020-11-18 08:26:28 -05:00
Vladimir Mandic 9a64dc6de2 update wiki 2020-11-17 17:48:05 -05:00
Vladimir Mandic 8004b3c37d 0.9.5 2020-11-17 17:47:16 -05:00
Vladimir Mandic d81a8bdc1e fix serious performance bug around skipframes 2020-11-17 17:42:44 -05:00
Vladimir Mandic 01ad1809f2 swtich to custom tfjs bundle 2020-11-17 12:38:48 -05:00
Vladimir Mandic 0fe1033903 0.9.4 2020-11-17 10:19:15 -05:00
Vladimir Mandic a773b9e2b9 swtich to tfjs source import 2020-11-17 10:18:15 -05:00
Vladimir Mandic 594c4e271f 0.9.3 2020-11-16 23:58:23 -05:00
Vladimir Mandic 104ea6fe8e updated build process 2020-11-16 23:58:06 -05:00
Vladimir Mandic 2eeb577387 updated build scripts 2020-11-16 16:40:25 -05:00
Vladimir Mandic c7671ef89d switched to minified build 2020-11-16 15:51:46 -05:00
Vladimir Mandic a8d52c45d8 web worker fixes 2020-11-15 09:28:57 -05:00
Vladimir Mandic d038b1e266 update buffered output 2020-11-14 17:22:59 -05:00
Vladimir Mandic c6d579dbf1 full rebuild 2020-11-14 07:05:20 -05:00
Vladimir Mandic ea57054a66 0.9.2 2020-11-14 07:02:43 -05:00
Vladimir Mandic 86321b2523 fix camera restart on resize 2020-11-14 07:02:05 -05:00
Vladimir Mandic 93d8c9fe64 update package description 2020-11-13 16:43:48 -05:00
Vladimir Mandic 591c01abc2 0.9.1 2020-11-13 16:42:56 -05:00
Vladimir Mandic ff0ec7cbbb version bump 2020-11-13 16:42:50 -05:00
Vladimir Mandic 85178e74a3 full rebuild 2020-11-13 16:42:00 -05:00
Vladimir Mandic 9d80512e36 implemented face embedding 2020-11-13 16:13:35 -05:00
Vladimir Mandic a7f2bf2303 added internal benchmark tool 2020-11-12 17:00:06 -05:00
Vladimir Mandic ea88b72c52 updated face uv coordinates 2020-11-12 14:52:32 -05:00
Vladimir Mandic f9dc5993f7 0.8.8 2020-11-12 12:59:00 -05:00
Vladimir Mandic bfbb3042af updated packages 2020-11-12 12:58:55 -05:00
Vladimir Mandic 1b0a037d3f reduced bundle size 2020-11-12 12:17:57 -05:00
Vladimir Mandic a3fa102a77 implemented buffered processing 2020-11-12 09:21:26 -05:00
Vladimir Mandic 17a8e20a53 fix for conditional model loading 2020-11-11 22:40:05 -05:00
Vladimir Mandic 4839b2caff 0.8.7 2020-11-11 15:02:53 -05:00
Vladimir Mandic fcdb076549 added performance notes 2020-11-11 15:02:49 -05:00
Vladimir Mandic 3e973c5555 added notes on models 2020-11-10 10:23:11 -05:00
Vladimir Mandic ec19579e53 update dependencies 2020-11-10 09:54:07 -05:00
Vladimir Mandic 53b4542140 fix bug in async ops and change imports 2020-11-10 08:57:39 -05:00
Vladimir Mandic b5ed43b434 fix wiki links 2020-11-09 20:23:05 -05:00
Vladimir Mandic 535b7e97a4 0.8.6 2020-11-09 20:13:44 -05:00
Vladimir Mandic de2819a96e add wasm bundle 2020-11-09 20:13:38 -05:00
Vladimir Mandic a9c393635f 0.8.5 2020-11-09 14:26:19 -05:00
Vladimir Mandic 57a93768f9 reimplemented blazeface processing 2020-11-09 14:26:10 -05:00
Vladimir Mandic 24b6fa1f23 0.8.4 2020-11-09 09:31:17 -05:00
Vladimir Mandic 4dba0c7247 added additional gestures 2020-11-09 09:31:11 -05:00
Vladimir Mandic a24c6e134b implemented blink detection 2020-11-09 08:57:24 -05:00
Vladimir Mandic 22d847311e fix wasm module 2020-11-09 06:32:11 -05:00
Vladimir Mandic cc2606a92a updated readme 2020-11-08 12:44:08 -05:00
Vladimir Mandic 6248a1dce9 updated wiki 2020-11-08 12:40:46 -05:00
Vladimir Mandic b36d3cf974 updated wiki 2020-11-08 12:35:26 -05:00
Vladimir Mandic 5e4ea1bc01 updated wiki 2020-11-08 12:33:24 -05:00
Vladimir Mandic c669648d7f 0.8.3 2020-11-08 12:32:36 -05:00
Vladimir Mandic a400b30178 refresh 2020-11-08 12:32:31 -05:00
Vladimir Mandic d2c6b6560d optimizations 2020-11-08 12:26:45 -05:00
Vladimir Mandic d7e85fd777 update 2020-11-08 10:06:23 -05:00
Vladimir Mandic f2bb62b099 update wiki 2020-11-08 09:56:27 -05:00
Vladimir Mandic 11d3f76df0 update hand model 2020-11-08 09:56:02 -05:00
Vladimir Mandic cd6f1f7e7a update hand algorithm 2020-11-08 01:17:25 -05:00
Vladimir Mandic 0adac25629 0.8.2 2020-11-08 01:16:28 -05:00
Vladimir Mandic 2da12bf1dc fix typos 2020-11-07 20:44:15 -05:00
Vladimir Mandic f92dac385f update build script 2020-11-07 20:15:09 -05:00
Vladimir Mandic 2738a763db commit 2020-11-07 11:36:45 -05:00
Vladimir Mandic f7d8b9e3ef updated changelog 2020-11-07 11:35:43 -05:00
Vladimir Mandic 1770b5b6f0 0.8.1 2020-11-07 11:34:44 -05:00
Vladimir Mandic 90abc61e4f updated ui 2020-11-07 11:34:09 -05:00
Vladimir Mandic 50c2648711 fix hand detection performance 2020-11-07 11:25:03 -05:00
Vladimir Mandic c9649d7397 optimized model loader 2020-11-07 10:37:19 -05:00
Vladimir Mandic 481352c5e1 updated wiki links 2020-11-07 09:47:26 -05:00
Vladimir Mandic 694b88c05b Merge branch 'main' of https://github.com/vladmandic/human into main 2020-11-07 09:44:43 -05:00
Vladimir Mandic 0a430ffce4 update dependencies 2020-11-07 09:44:33 -05:00
Vladimir Mandic 952b050bff updated wiki 2020-11-07 09:42:54 -05:00
Vladimir Mandic 520b0ae747 created wiki 2020-11-07 09:39:54 -05:00
Vladimir Mandic aee2c6caf6 Update issue templates 2020-11-07 09:37:56 -05:00
Vladimir Mandic fc85f4a799 optimize font resizing 2020-11-06 22:20:42 -05:00
Vladimir Mandic 0670083e92 fix nms sync call 2020-11-06 19:25:37 -05:00
Vladimir Mandic b8468be1ed 0.7.6 2020-11-06 16:21:25 -05:00
Vladimir Mandic b508761999 fixed memory leaks and updated docs 2020-11-06 16:21:20 -05:00
Vladimir Mandic c3cf00b583 model tuning 2020-11-06 15:35:58 -05:00
Vladimir Mandic eb3dbea2ca cache invalidation improvements 2020-11-06 13:50:16 -05:00
Vladimir Mandic 6f22d2a3d3 full async operations 2020-11-06 11:39:39 -05:00
Vladimir Mandic 544a3b0ef2 0.7.5 2020-11-05 23:46:43 -05:00
Vladimir Mandic 21f7926d1d implemented dev-server 2020-11-05 23:46:37 -05:00
Vladimir Mandic 7b8b295306 0.7.4 2020-11-05 16:00:46 -05:00
Vladimir Mandic e15e80302e fix canvas size on different orientation 2020-11-05 15:59:28 -05:00
Vladimir Mandic 02ba731a60 updated emotion models 2020-11-05 15:38:09 -05:00
Vladimir Mandic fe413b547d updated changelog 2020-11-05 09:12:43 -05:00
Vladimir Mandic 6569ee7446 switched from es2020 to es2018 build target 2020-11-05 09:12:31 -05:00
Vladimir Mandic cb42330b9e 0.7.3 2020-11-05 09:06:17 -05:00
Vladimir Mandic bc35bf7584 optimized camera and mobile layout 2020-11-05 09:06:09 -05:00
Vladimir Mandic 91ddc0c57d fixed worker and filter compatibility 2020-11-05 08:21:23 -05:00
Vladimir Mandic 14b89145c9 0.7.2 2020-11-04 14:59:33 -05:00
Vladimir Mandic 9fa7e3d467 major work on handpose model 2020-11-04 14:59:30 -05:00
Vladimir Mandic 479fc2547c updated mobile build 2020-11-04 12:10:26 -05:00
Vladimir Mandic 125055e74a update demo build process 2020-11-04 11:57:44 -05:00
Vladimir Mandic d7ddc3ae2c updated docs 2020-11-04 11:45:24 -05:00
Vladimir Mandic 74d08df07f updated changelog 2020-11-04 11:44:07 -05:00
Vladimir Mandic cce9535be5 0.7.1 2020-11-04 11:44:00 -05:00
Vladimir Mandic 37321af4ae changed demo build process 2020-11-04 11:43:51 -05:00
Vladimir Mandic 91f5294925 0.6.7 2020-11-04 10:18:30 -05:00
Vladimir Mandic d94aa0362c implemented simple gesture recognition 2020-11-04 10:18:22 -05:00
Vladimir Mandic 3cec6710d4 0.6.6 2020-11-04 01:13:44 -05:00
Vladimir Mandic 6653cff104 remove debug code 2020-11-04 01:13:40 -05:00
Vladimir Mandic 8dda59f5d9 0.6.5 2020-11-04 01:11:30 -05:00
Vladimir Mandic 430a950112 redo hand detection 2020-11-04 01:11:24 -05:00
Vladimir Mandic bfdcb301f4 0.6.4 2020-11-03 15:24:11 -05:00
Vladimir Mandic 7860760017 added manifest 2020-11-03 15:24:02 -05:00
Vladimir Mandic 948b4d2cce 0.6.3 2020-11-03 11:11:57 -05:00
Vladimir Mandic 99db8d3724 update changelog 2020-11-03 11:11:53 -05:00
Vladimir Mandic 9b0b8cf390 enhanced processing resolution 2020-11-03 10:55:33 -05:00
Vladimir Mandic d3bd65ba50 fix pause restart 2020-11-03 09:40:04 -05:00
Vladimir Mandic 2b5901ff55 complete model refactoring 2020-11-03 09:34:36 -05:00
Vladimir Mandic 0f5ccda33c fixed typo 2020-11-02 22:17:56 -05:00
Vladimir Mandic 30265459f9 0.6.2 2020-11-02 22:15:40 -05:00
Vladimir Mandic 161f168bba optimized demo 2020-11-02 22:15:37 -05:00
Vladimir Mandic 97447e9e49 updated docs 2020-11-02 18:56:12 -05:00
Vladimir Mandic cbf12723bd package update 2020-11-02 18:55:17 -05:00
Vladimir Mandic d9e842b41a 0.6.1 2020-11-02 18:54:13 -05:00
Vladimir Mandic eb3bd3dae9 major performance improvements for all models 2020-11-02 18:54:03 -05:00
Vladimir Mandic 0d99113f77 Revert "optimized canvas handling"
This reverts commit 80d7133a8e.
2020-11-02 17:31:37 -05:00
Vladimir Mandic 80d7133a8e optimized canvas handling 2020-11-02 17:25:35 -05:00
Vladimir Mandic 5e3193bf54 minor optimization to imagefx 2020-11-02 12:21:30 -05:00
Vladimir Mandic 45aff915f5 fix demo image sample 2020-11-01 14:16:47 -05:00
Vladimir Mandic e6d195187b added tfjs-vis to distribution 2020-11-01 13:59:25 -05:00
Vladimir Mandic 448ac2096a 0.5.5 2020-11-01 13:10:40 -05:00
Vladimir Mandic 68719ded32 updated changelog 2020-11-01 13:10:36 -05:00
Vladimir Mandic 34646100d0 changed defaults 2020-11-01 13:10:22 -05:00
Vladimir Mandic 78643806d1 0.5.4 2020-11-01 13:07:58 -05:00
Vladimir Mandic 813d052ea8 implemented memory profiler 2020-11-01 13:07:53 -05:00
Vladimir Mandic 8d859e2c92 0.5.3 2020-10-30 15:45:00 -04:00
Vladimir Mandic cc3d5f939e improved debug logging 2020-10-30 11:57:23 -04:00
Vladimir Mandic e8d36eea69 0.5.2 2020-10-30 10:30:12 -04:00
Vladimir Mandic d0e036526e updated changelog 2020-10-30 10:30:07 -04:00
Vladimir Mandic e3a00d48d0 updated menu ui 2020-10-30 10:29:50 -04:00
Vladimir Mandic e39831224d added wasm and webgpu backends 2020-10-30 10:23:49 -04:00
Vladimir Mandic c5ccdda811 0.5.1 2020-10-30 07:35:30 -04:00
Vladimir Mandic 763d9d0b5e updated changelog 2020-10-30 07:35:19 -04:00
Vladimir Mandic 602d940c1d updated dependencies 2020-10-30 07:34:55 -04:00
Vladimir Mandic f9d78b27e6 improve demo line continous draws 2020-10-30 07:32:35 -04:00
Vladimir Mandic bf1135828f 0.4.10 2020-10-30 07:14:47 -04:00
Vladimir Mandic e8fc546d1e fix for seedrandom 2020-10-30 07:14:42 -04:00
Vladimir Mandic 81a460c412 0.4.9 2020-10-29 00:09:33 -04:00
Vladimir Mandic a6c5bc3d70 updated dependencies 2020-10-29 00:09:29 -04:00
Vladimir Mandic ec7fafce52 updated changelog 2020-10-28 15:03:23 -04:00
Vladimir Mandic 5d89ecf20b 0.4.8 2020-10-28 15:03:08 -04:00
Vladimir Mandic 8d445e7ebb updated build targets and tfjs to 2.7.0 2020-10-28 15:02:59 -04:00
Vladimir Mandic a7c13e7c25 updated menu library 2020-10-27 14:20:21 -04:00
Vladimir Mandic 57d30f2710 Revert "updated menu handler"
This reverts commit c2781a3965.
2020-10-27 14:19:24 -04:00
Vladimir Mandic c2781a3965 updated menu handler 2020-10-27 14:07:39 -04:00
Vladimir Mandic 8295608ca2 0.4.7 2020-10-27 12:49:24 -04:00
Vladimir Mandic bea70535eb 0.4.6 2020-10-27 11:56:50 -04:00
Vladimir Mandic b6ed98dde8 fix firefox compatibility bug 2020-10-27 11:56:41 -04:00
Vladimir Mandic e4ebce3642 0.4.5 2020-10-27 10:32:21 -04:00
Vladimir Mandic 85fcb44e2b updated docs 2020-10-27 10:32:12 -04:00
Vladimir Mandic 9be15515b5 updated samples 2020-10-27 10:30:28 -04:00
Vladimir Mandic 113a26ab2c 0.4.4 2020-10-27 10:06:14 -04:00
Vladimir Mandic 4664228d4b implelented input resizing 2020-10-27 10:06:01 -04:00
Vladimir Mandic d579102a4f updated docs 2020-10-25 21:33:06 -04:00
Vladimir Mandic 80013d6822 updated 2020-10-22 18:50:28 -04:00
Vladimir Mandic c6b8523ca5 0.4.3 2020-10-22 18:50:21 -04:00
Vladimir Mandic eb221d42fd update build process 2020-10-22 18:50:09 -04:00
Vladimir Mandic 21031af836 0.4.2 2020-10-20 10:08:23 -04:00
Vladimir Mandic a299806dc0 updated docs 2020-10-20 10:08:06 -04:00
Vladimir Mandic 2566a9633f log initialization 2020-10-20 07:58:20 -04:00
Vladimir Mandic ee8e7080de updated changelog 2020-10-19 11:04:34 -04:00
Vladimir Mandic 279de1eeec 0.4.1 2020-10-19 11:04:02 -04:00
Vladimir Mandic 36675c5152 breaking change: convert to object class 2020-10-19 11:03:48 -04:00
Vladimir Mandic dfc67f0188 compatibility notes 2020-10-18 14:14:05 -04:00
Vladimir Mandic 0d523c3744 0.3.9 2020-10-18 12:22:39 -04:00
Vladimir Mandic e7229689c5 updated docs 2020-10-18 12:21:17 -04:00
Vladimir Mandic 362ee4441e implemented image filters 2020-10-18 12:12:09 -04:00
Vladimir Mandic f5f3a2c1d5 pure tensor pipeline without image converts 2020-10-18 09:21:53 -04:00
Vladimir Mandic 7a6741e3b4 autodetect skipFrames 2020-10-18 08:07:45 -04:00
Vladimir Mandic 859ed1789a 0.3.8 2020-10-17 21:00:43 -04:00
Vladimir Mandic 2f60e4c72c new menu layout 2020-10-17 20:59:43 -04:00
Vladimir Mandic 12c5350174 0.3.7 2020-10-17 11:43:44 -04:00
Vladimir Mandic 35226fa0a1 updated changelog 2020-10-17 11:43:40 -04:00
Vladimir Mandic 08101e4c45 added diagnostics output 2020-10-17 11:43:04 -04:00
Vladimir Mandic 0411ed1371 parallelized agegender operations 2020-10-17 11:38:24 -04:00
Vladimir Mandic 5a4a056e32 updated readme 2020-10-17 10:32:02 -04:00
Vladimir Mandic b47fc362cf 0.3.6 2020-10-17 10:25:37 -04:00
Vladimir Mandic fdb08a91df fixed webcam initialization 2020-10-17 10:25:27 -04:00
Vladimir Mandic 8643a2191a fixed memory leaks and added scoped runs 2020-10-17 10:06:02 -04:00
Vladimir Mandic e36ac717cb modularized draw 2020-10-17 07:34:45 -04:00
Vladimir Mandic f3bf35533e added state handling 2020-10-17 07:15:23 -04:00
Vladimir Mandic 9f86c24fe3 refactored package file layout 2020-10-17 06:30:00 -04:00
Vladimir Mandic 4c3bb5fed9 updated changelog 2020-10-16 15:22:20 -04:00
Vladimir Mandic 4eb6bb3d8c 0.3.5 2020-10-16 15:22:01 -04:00
Vladimir Mandic 2bb1f151e4 added auto-generated changelog 2020-10-16 15:21:56 -04:00
Vladimir Mandic 9634cbc608 updated readme 2020-10-16 15:13:29 -04:00
Vladimir Mandic 8511c99e81 0.3.4 2020-10-16 15:05:01 -04:00
Vladimir Mandic 45730a07a6 updated examples plus bugfixes 2020-10-16 15:04:51 -04:00
Vladimir Mandic 6a9a7b2467 added camera selection 2020-10-16 11:23:59 -04:00
Vladimir Mandic 93b5aa16ed optimized blazeface anchors 2020-10-16 10:48:10 -04:00
Vladimir Mandic fc0940a481 added error handling 2020-10-16 10:12:12 -04:00
Vladimir Mandic 2110bafb63 updated default values 2020-10-15 20:29:51 -04:00
Vladimir Mandic a39eaa6399 0.3.3 2020-10-15 20:25:15 -04:00
Vladimir Mandic 03db7fd8da added blazeface back and front models 2020-10-15 20:20:37 -04:00
Vladimir Mandic 699ab9f309 0.3.2 2020-10-15 18:16:19 -04:00
Vladimir Mandic 6de5ed0663 reduced web worker latency 2020-10-15 18:16:05 -04:00
Vladimir Mandic ff386b6e64 added debugging and versioning 2020-10-15 15:25:58 -04:00
Vladimir Mandic 567966a162 optimized demos and added scoped runs 2020-10-15 09:43:16 -04:00
Vladimir Mandic 3547da7130 added multi backend support 2020-10-15 08:16:34 -04:00
Vladimir Mandic e93487315e 0.3.1 2020-10-14 18:30:26 -04:00
Vladimir Mandic 19cffbf9f8 0.2.10 2020-10-14 18:22:56 -04:00
Vladimir Mandic 38d02819d3 added emotion backend 2020-10-14 18:22:38 -04:00
Vladimir Mandic 420607c490 module parametrization and performance monitoring 2020-10-14 13:23:02 -04:00
Vladimir Mandic 9e1776906f implemented multi-hand support 2020-10-14 11:43:33 -04:00
Vladimir Mandic f484493b6f updated dependencies 2020-10-13 21:08:06 -04:00
Vladimir Mandic 493b2c61b4 fixed documentation typos 2020-10-13 20:59:09 -04:00
Vladimir Mandic cf8cd83aa8 0.2.9 2020-10-13 20:52:34 -04:00
Vladimir Mandic 251c75d553 added node build and demo 2020-10-13 20:52:30 -04:00
Vladimir Mandic e7a71b7367 0.2.8 2020-10-13 10:08:50 -04:00
Vladimir Mandic 43e0f6cb3b 0.2.7 2020-10-13 10:07:03 -04:00
Vladimir Mandic 247004e3eb new examples 2020-10-13 10:06:49 -04:00
Vladimir Mandic abc4950eff 0.2.6 2020-10-13 09:59:27 -04:00
Vladimir Mandic a1adaf3599 updated demo 2020-10-13 09:59:21 -04:00
Vladimir Mandic ca6d7a662d enable all models by default 2020-10-12 22:03:55 -04:00
Vladimir Mandic d8bb9d3db9 0.2.5 2020-10-12 22:01:44 -04:00
Vladimir Mandic f81e183950 fixed memory leak 2020-10-12 22:01:35 -04:00
Vladimir Mandic 8fdb0c892c 0.2.4 2020-10-12 15:05:30 -04:00
Vladimir Mandic e5287aae0b 0.2.3 2020-10-12 15:03:59 -04:00
Vladimir Mandic 8332a0c077 updated keywords 2020-10-12 15:03:51 -04:00
Vladimir Mandic 4ff0623c66 updated readme 2020-10-12 14:46:08 -04:00
Vladimir Mandic da0faab688 0.2.2 2020-10-12 14:41:59 -04:00
Vladimir Mandic e47bf74c6a updated docs 2020-10-12 14:41:50 -04:00
Vladimir Mandic fa89a67b31 updated readme 2020-10-12 14:33:07 -04:00
Vladimir Mandic b3fcc7feac updated linting 2020-10-12 11:19:52 -04:00
Vladimir Mandic 38e960e38e 0.2.1 2020-10-12 11:05:46 -04:00
Vladimir Mandic d5b4c531fb updated readme 2020-10-12 11:05:29 -04:00
Vladimir Mandic 1fdd4d3753 added sample image 2020-10-12 10:59:55 -04:00
Vladimir Mandic 151655e464 updated docs 2020-10-12 10:31:36 -04:00
Vladimir Mandic 11cbd0ccd6 updated docs 2020-10-12 10:28:42 -04:00
Vladimir Mandic 33bca90e14 updated docs 2020-10-12 10:27:22 -04:00
Vladimir Mandic fa702d47f6 updated model path 2020-10-12 10:20:51 -04:00
Vladimir Mandic 75994f4b57 updated docs 2020-10-12 10:16:31 -04:00
Vladimir Mandic 16d3099f74 updated model path 2020-10-12 10:15:00 -04:00
Vladimir Mandic ce33614987 updated model path 2020-10-12 10:14:26 -04:00
Vladimir Mandic df80506b43 updated iife and esm demos 2020-10-12 10:08:00 -04:00
Vladimir Mandic 5516daf2d6 updated demo 2020-10-11 21:21:41 -04:00
Vladimir Mandic c018daee34 initial public commit 2020-10-11 19:22:43 -04:00
451 changed files with 64419 additions and 73558 deletions

View File

@ -8,14 +8,13 @@
"profiles": { "profiles": {
"production": ["clean", "compile", "typings", "typedoc", "lint", "changelog"], "production": ["clean", "compile", "typings", "typedoc", "lint", "changelog"],
"development": ["serve", "watch", "compile"], "development": ["serve", "watch", "compile"],
"serve": ["serve"], "serve": ["serve"]
"clean": ["clean"]
}, },
"clean": { "clean": {
"locations": ["dist/*", "types/*", "typedoc/*"] "locations": ["dist/*", "types/lib/*", "typedoc/*"]
}, },
"lint": { "lint": {
"locations": [ "**/*.json", "src/**/*.ts", "test/**/*.js", "demo/**/*.js", "**/*.md" ], "locations": [ "*.json", "src/**/*.ts", "test/**/*.js", "demo/**/*.js" ],
"rules": { } "rules": { }
}, },
"changelog": { "changelog": {
@ -24,8 +23,8 @@
"serve": { "serve": {
"sslKey": "node_modules/@vladmandic/build/cert/https.key", "sslKey": "node_modules/@vladmandic/build/cert/https.key",
"sslCrt": "node_modules/@vladmandic/build/cert/https.crt", "sslCrt": "node_modules/@vladmandic/build/cert/https.crt",
"httpPort": 8000, "httpPort": 10030,
"httpsPort": 8001, "httpsPort": 10031,
"documentRoot": ".", "documentRoot": ".",
"defaultFolder": "demo", "defaultFolder": "demo",
"defaultFile": "index.html" "defaultFile": "index.html"
@ -39,13 +38,6 @@
"banner": { "js": "/*\n Human\n homepage: <https://github.com/vladmandic/human>\n author: <https://github.com/vladmandic>'\n*/\n" } "banner": { "js": "/*\n Human\n homepage: <https://github.com/vladmandic/human>\n author: <https://github.com/vladmandic>'\n*/\n" }
}, },
"targets": [ "targets": [
{
"name": "tfjs/browser/version",
"platform": "browser",
"format": "esm",
"input": "tfjs/tf-version.ts",
"output": "dist/tfjs.version.js"
},
{ {
"name": "tfjs/nodejs/cpu", "name": "tfjs/nodejs/cpu",
"platform": "node", "platform": "node",
@ -84,7 +76,6 @@
"format": "cjs", "format": "cjs",
"input": "tfjs/tf-node-wasm.ts", "input": "tfjs/tf-node-wasm.ts",
"output": "dist/tfjs.esm.js", "output": "dist/tfjs.esm.js",
"minify": false,
"external": ["@tensorflow"] "external": ["@tensorflow"]
}, },
{ {
@ -95,6 +86,13 @@
"output": "dist/human.node-wasm.js", "output": "dist/human.node-wasm.js",
"external": ["@tensorflow"] "external": ["@tensorflow"]
}, },
{
"name": "tfjs/browser/version",
"platform": "browser",
"format": "esm",
"input": "tfjs/tf-version.ts",
"output": "dist/tfjs.version.js"
},
{ {
"name": "tfjs/browser/esm/nobundle", "name": "tfjs/browser/esm/nobundle",
"platform": "browser", "platform": "browser",
@ -109,17 +107,16 @@
"format": "esm", "format": "esm",
"input": "src/human.ts", "input": "src/human.ts",
"output": "dist/human.esm-nobundle.js", "output": "dist/human.esm-nobundle.js",
"sourcemap": false, "sourcemap": true,
"external": ["@tensorflow"] "external": ["@tensorflow"]
}, },
{ {
"name": "tfjs/browser/esm/bundle", "name": "tfjs/browser/esm/custom",
"platform": "browser", "platform": "browser",
"format": "esm", "format": "esm",
"input": "tfjs/tf-browser.ts", "input": "tfjs/tf-custom.ts",
"output": "dist/tfjs.esm.js", "output": "dist/tfjs.esm.js",
"sourcemap": false, "sourcemap": false
"minify": true
}, },
{ {
"name": "human/browser/iife/bundle", "name": "human/browser/iife/bundle",
@ -160,15 +157,6 @@
"output": "demo/faceid/index.js", "output": "demo/faceid/index.js",
"sourcemap": true, "sourcemap": true,
"external": ["*/human.esm.js"] "external": ["*/human.esm.js"]
},
{
"name": "demo/tracker",
"platform": "browser",
"format": "esm",
"input": "demo/tracker/index.ts",
"output": "demo/tracker/index.js",
"sourcemap": true,
"external": ["*/human.esm.js"]
} }
] ]
}, },

View File

@ -1,204 +1,93 @@
{ {
"globals": { "globals": {},
},
"rules": {
"@typescript-eslint/no-require-imports":"off"
},
"overrides": [
{
"files": ["**/*.ts"],
"parser": "@typescript-eslint/parser",
"parserOptions": { "ecmaVersion": "latest", "project": ["./tsconfig.json"] },
"plugins": ["@typescript-eslint"],
"env": {
"browser": true,
"commonjs": false,
"node": false,
"es2021": true
},
"extends": [
"airbnb-base",
"eslint:recommended",
"plugin:@typescript-eslint/strict",
"plugin:@typescript-eslint/recommended",
"plugin:@typescript-eslint/recommended-requiring-type-checking",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:import/recommended",
"plugin:promise/recommended"
],
"rules": {
"@typescript-eslint/ban-ts-comment":"off",
"@typescript-eslint/dot-notation":"off",
"@typescript-eslint/no-empty-interface":"off",
"@typescript-eslint/no-empty-object-type":"off",
"@typescript-eslint/no-inferrable-types":"off",
"@typescript-eslint/no-misused-promises":"off",
"@typescript-eslint/no-require-imports":"off",
"@typescript-eslint/no-unnecessary-condition":"off",
"@typescript-eslint/no-unnecessary-type-assertion":"off",
"@typescript-eslint/no-unsafe-argument":"off",
"@typescript-eslint/no-unsafe-assignment":"off",
"@typescript-eslint/no-unsafe-call":"off",
"@typescript-eslint/no-unsafe-member-access":"off",
"@typescript-eslint/no-unsafe-return":"off",
"@typescript-eslint/non-nullable-type-assertion-style":"off",
"@typescript-eslint/prefer-for-of":"off",
"@typescript-eslint/prefer-nullish-coalescing":"off",
"@typescript-eslint/prefer-ts-expect-error":"off",
"@typescript-eslint/restrict-plus-operands":"off",
"@typescript-eslint/restrict-template-expressions":"off",
"dot-notation":"off",
"guard-for-in":"off",
"import/extensions": ["off", "always"],
"import/no-unresolved":"off",
"import/prefer-default-export":"off",
"lines-between-class-members":"off",
"max-len": [1, 275, 3],
"no-async-promise-executor":"off",
"no-await-in-loop":"off",
"no-bitwise":"off",
"no-continue":"off",
"no-lonely-if":"off",
"no-mixed-operators":"off",
"no-param-reassign":"off",
"no-plusplus":"off",
"no-regex-spaces":"off",
"no-restricted-syntax":"off",
"no-return-assign":"off",
"no-void":"off",
"object-curly-newline":"off",
"prefer-destructuring":"off",
"prefer-template":"off",
"radix":"off"
}
},
{
"files": ["**/*.d.ts"],
"parser": "@typescript-eslint/parser",
"parserOptions": { "ecmaVersion": "latest", "project": ["./tsconfig.json"] },
"plugins": ["@typescript-eslint"],
"env": {
"browser": true,
"commonjs": false,
"node": false,
"es2021": true
},
"extends": [
"airbnb-base",
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended",
"plugin:@typescript-eslint/recommended-requiring-type-checking",
"plugin:@typescript-eslint/strict",
"plugin:import/recommended",
"plugin:promise/recommended"
],
"rules": {
"@typescript-eslint/array-type":"off",
"@typescript-eslint/ban-types":"off",
"@typescript-eslint/consistent-indexed-object-style":"off",
"@typescript-eslint/consistent-type-definitions":"off",
"@typescript-eslint/no-empty-interface":"off",
"@typescript-eslint/no-explicit-any":"off",
"@typescript-eslint/no-invalid-void-type":"off",
"@typescript-eslint/no-unnecessary-type-arguments":"off",
"@typescript-eslint/no-unnecessary-type-constraint":"off",
"comma-dangle":"off",
"indent":"off",
"lines-between-class-members":"off",
"max-classes-per-file":"off",
"max-len":"off",
"no-multiple-empty-lines":"off",
"no-shadow":"off",
"no-use-before-define":"off",
"quotes":"off",
"semi":"off"
}
},
{
"files": ["**/*.js"],
"parserOptions": { "sourceType": "module", "ecmaVersion": "latest" },
"plugins": [],
"env": { "env": {
"browser": true, "browser": true,
"commonjs": true, "commonjs": true,
"node": true, "node": true,
"es2021": true "es2021": true
}, },
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 2021
},
"plugins": [
"@typescript-eslint",
"html"
],
"extends": [ "extends": [
"airbnb-base", "airbnb-base",
"eslint:recommended", "eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended",
"plugin:import/errors",
"plugin:import/warnings",
"plugin:json/recommended-with-comments",
"plugin:node/recommended", "plugin:node/recommended",
"plugin:promise/recommended" "plugin:promise/recommended"
], ],
"rules": {
"dot-notation":"off",
"import/extensions": ["error", "always"],
"import/no-extraneous-dependencies":"off",
"max-len": [1, 275, 3],
"no-await-in-loop":"off",
"no-bitwise":"off",
"no-continue":"off",
"no-mixed-operators":"off",
"no-param-reassign":"off",
"no-plusplus":"off",
"no-regex-spaces":"off",
"no-restricted-syntax":"off",
"no-return-assign":"off",
"node/no-unsupported-features/es-syntax":"off",
"object-curly-newline":"off",
"prefer-destructuring":"off",
"prefer-template":"off",
"radix":"off"
}
},
{
"files": ["**/*.json"],
"parserOptions": { "ecmaVersion": "latest" },
"plugins": ["json"],
"env": {
"browser": false,
"commonjs": false,
"node": false,
"es2021": false
},
"extends": []
},
{
"files": ["**/*.md"],
"plugins": ["markdown"],
"processor": "markdown/markdown",
"rules": {
"no-undef":"off"
}
},
{
"files": ["**/*.md/*.js"],
"rules": {
"@typescript-eslint/no-unused-vars":"off",
"@typescript-eslint/triple-slash-reference":"off",
"import/newline-after-import":"off",
"import/no-unresolved":"off",
"no-console":"off",
"no-global-assign":"off",
"no-multi-spaces":"off",
"no-restricted-globals":"off",
"no-undef":"off",
"no-unused-vars":"off",
"node/no-missing-import":"off",
"node/no-missing-require":"off",
"promise/catch-or-return":"off"
}
}
],
"ignorePatterns": [ "ignorePatterns": [
"node_modules",
"assets", "assets",
"dist",
"demo/helpers/*.js", "demo/helpers/*.js",
"demo/typescript/*.js", "demo/typescript/*.js",
"demo/faceid/*.js", "demo/faceid/*.js",
"demo/tracker/*.js", "dist",
"typedoc" "media",
] "models",
"node_modules",
"types/human.d.ts"
],
"rules": {
"@typescript-eslint/ban-ts-comment": "off",
"@typescript-eslint/explicit-module-boundary-types": "off",
"@typescript-eslint/no-shadow": "error",
"@typescript-eslint/no-var-requires": "off",
"@typescript-eslint/prefer-as-const": "off",
"@typescript-eslint/triple-slash-reference": "off",
"@typescript-eslint/no-inferrable-types": "off",
"@typescript-eslint/no-empty-interface": ["error", { "allowSingleExtends": true }],
"camelcase": "off",
"class-methods-use-this": "off",
"dot-notation": "off",
"func-names": "off",
"guard-for-in": "off",
"import/extensions": "off",
"import/named": "off",
"import/no-extraneous-dependencies": "off",
"import/no-named-as-default": "off",
"import/no-unresolved": "off",
"import/prefer-default-export": "off",
"lines-between-class-members": "off",
"max-len": [1, 275, 3],
"newline-per-chained-call": "off",
"no-async-promise-executor": "off",
"no-await-in-loop": "off",
"no-bitwise": "off",
"no-case-declarations":"off",
"no-continue": "off",
"no-else-return": "off",
"no-lonely-if": "off",
"no-loop-func": "off",
"no-mixed-operators": "off",
"no-param-reassign":"off",
"no-plusplus": "off",
"no-process-exit": "off",
"no-regex-spaces": "off",
"no-restricted-globals": "off",
"no-restricted-syntax": "off",
"no-return-assign": "off",
"no-shadow": "off",
"no-underscore-dangle": "off",
"node/no-missing-import": ["error", { "tryExtensions": [".js", ".json", ".ts"] }],
"node/no-unpublished-import": "off",
"node/no-unpublished-require": "off",
"node/no-unsupported-features/es-syntax": "off",
"node/shebang": "off",
"object-curly-newline": "off",
"prefer-destructuring": "off",
"prefer-template":"off",
"promise/always-return": "off",
"promise/catch-or-return": "off",
"promise/no-nesting": "off",
"radix": "off"
}
} }

11
.github/FUNDING.yml vendored
View File

@ -1,11 +0,0 @@
github: [vladmandic]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

10
.gitignore vendored
View File

@ -1,9 +1,5 @@
.vscode
pnpm-lock.yaml
*.swp
node_modules/ node_modules/
types/lib types/lib
pnpm-lock.yaml
package-lock.json
*.swp
samples/**/*.mp4
samples/**/*.webm
temp
tmp

View File

@ -1,7 +1,6 @@
{ {
"MD012": false, "MD012": false,
"MD013": false, "MD013": false,
"MD029": false,
"MD033": false, "MD033": false,
"MD036": false, "MD036": false,
"MD041": false "MD041": false

6
.npmrc
View File

@ -1,5 +1 @@
force=true force = true
omit=dev
legacy-peer-deps=true
strict-peer-dependencies=false
node-options='--no-deprecation'

10
.vscode/settings.json vendored
View File

@ -1,10 +0,0 @@
{
"search.exclude": {
"dist/*": true,
"node_modules/*": true,
"types": true,
"typedoc": true,
},
"search.useGlobalIgnoreFiles": true,
"search.useParentIgnoreFiles": true
}

View File

@ -1,6 +1,6 @@
# @vladmandic/human # @vladmandic/human
Version: **3.3.6** Version: **2.7.1**
Description: **Human: AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition, Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction, Gesture Recognition** Description: **Human: AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition, Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction, Gesture Recognition**
Author: **Vladimir Mandic <mandic00@live.com>** Author: **Vladimir Mandic <mandic00@live.com>**
@ -9,226 +9,17 @@
## Changelog ## Changelog
### **3.3.6** 2025/08/26 mandic00@live.com
### **origin/main** 2025/02/05 mandic00@live.com
- full rebuild
### **3.3.5** 2025/02/05 mandic00@live.com
- rebuild
- add human.draw.tensor method
### **3.3.4** 2024/10/24 mandic00@live.com
### **3.3.3** 2024/10/14 mandic00@live.com
- add loaded property to model stats and mark models not loaded correctly.
- release build
### **3.3.2** 2024/09/11 mandic00@live.com
- full rebuild
### **3.3.1** 2024/09/11 mandic00@live.com
- add config.face.detector.square option
- human 3.3 alpha test run
- human 3.3 alpha with new build environment
- release rebuild
- fix flazeface tensor scale and update build platform
### **3.2.2** 2024/04/17 mandic00@live.com
### **release: 3.2.1** 2024/02/15 mandic00@live.com
### **3.2.1** 2024/02/15 mandic00@live.com
### **3.2.0** 2023/12/06 mandic00@live.com
- set browser false when navigator object is empty
- https://github.com/vladmandic/human/issues/402
### **release: 3.1.2** 2023/09/18 mandic00@live.com
- full rebuild
### **3.1.2** 2023/09/18 mandic00@live.com
- major toolkit upgrade
- full rebuild
- major toolkit upgrade
### **3.1.1** 2023/08/05 mandic00@live.com
- fixes plus tfjs upgrade for new release
### **3.0.7** 2023/06/12 mandic00@live.com
- full rebuild
- fix memory leak in histogramequalization
- initial work on tracker
### **3.0.6** 2023/03/21 mandic00@live.com
- add optional crop to multiple models
- fix movenet-multipose
- add electron detection
- fix gender-ssrnet-imdb
- add movenet-multipose workaround
- rebuild and publish
- add face.detector.minsize configurable setting
- add affectnet
### **3.0.5** 2023/02/02 mandic00@live.com
- add gear-e models
- detect react-native
- redo blazeface annotations
### **3.0.4** 2023/01/29 mandic00@live.com
- make naviator calls safe
- fix facedetector-only configs
### **3.0.3** 2023/01/07 mandic00@live.com
- full rebuild
### **3.0.2** 2023/01/06 mandic00@live.com
- default face.rotation disabled
### **release: 3.0.1** 2022/11/22 mandic00@live.com
### **3.0.1** 2022/11/22 mandic00@live.com
- support dynamic loads
- polish demos
- add facedetect demo and fix model async load
- enforce markdown linting
- cleanup git history
- default empty result
- refactor draw and models namespaces
- refactor distance
- add basic anthropometry
- added webcam id specification
- include external typedefs
- prepare external typedefs
- rebuild all
- include project files for types
- architectural improvements
- refresh dependencies
- add named exports
- add draw label templates
- reduce dev dependencies
- tensor rank strong typechecks
- rebuild dependencies
### **2.11.1** 2022/10/09 mandic00@live.com
- add rvm segmentation model
- add human.webcam methods
- create funding.yml
- fix rotation interpolation
### **2.10.3** 2022/09/21 mandic00@live.com
- add human.video method
### **2.10.2** 2022/09/11 mandic00@live.com
- add node.js esm compatibility (#292)
- release
### **2.10.1** 2022/09/07 mandic00@live.com
- release candidate
- add config flags
- test update
- release preview
- optimize startup sequence
- reorder backend init code
- test embedding
- embedding test
- add browser iife tests
- minor bug fixes and increased test coverage
- extend release tests
- add model load exception handling
- add softwarekernels config option
- expand type safety
- full eslint rule rewrite
### **2.9.4** 2022/08/20 mandic00@live.com
- add browser test
- add tensorflow library detection
- fix wasm detection
- enumerate additional models
- release refresh
### **2.9.3** 2022/08/10 mandic00@live.com
- rehault testing framework
- release refresh
- add insightface
### **2.9.2** 2022/08/08 mandic00@live.com
- release rebuild
### **2.9.1** 2022/07/25 mandic00@live.com
- full rebuild
- release cleanup
- tflite experiments
- add load monitor test
- beta for upcoming major release
- swtich to release version of tfjs
- placeholder for face contours
- improve face compare in main demo
- add webview support
- fix(gear): ensure gear.modelpath is used for loadmodel()
- npm default install should be prod only
- fix npm v7 compatibility
- add getmodelstats method
- rebuild
- release build
### **2.8.1** 2022/06/08 mandic00@live.com
- webgpu and wasm optimizations
- add faceboxes prototype
- full rebuild
### **2.7.4** 2022/05/24 mandic00@live.com
### **2.7.3** 2022/05/24 mandic00@live.com
- add face.mesh.keepinvalid config flag
- initial work for new facemesh model
### **2.7.2** 2022/05/12 mandic00@live.com
- fix demo when used with video files
- major release
### **2.7.1** 2022/05/09 mandic00@live.com ### **2.7.1** 2022/05/09 mandic00@live.com
- support 4k input - support 4k input
- add attention draw methods
### **origin/main** 2022/04/18 mandic00@live.com
- fix coloring function - fix coloring function
- enable precompile as part of warmup - enable precompile as part of warmup
- prepare release beta - prepare release beta
- change default face crop - change default face crop
- face attention model is available in human-models
- beta release 2.7 - beta release 2.7
- refactor draw methods - refactor draw methods
- implement face attention model - implement face attention model
@ -540,6 +331,7 @@
- implemented human.next global interpolation method - implemented human.next global interpolation method
- finished draw buffering and smoothing and enabled by default - finished draw buffering and smoothing and enabled by default
- implemented service worker - implemented service worker
- quantized centernet
- release candidate - release candidate
- added usage restrictions - added usage restrictions
- quantize handdetect model - quantize handdetect model

280
README.md
View File

@ -1,9 +1,9 @@
[![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/vladmandic)
![Git Version](https://img.shields.io/github/package-json/v/vladmandic/human?style=flat-square&svg=true&label=git) ![Git Version](https://img.shields.io/github/package-json/v/vladmandic/human?style=flat-square&svg=true&label=git)
![NPM Version](https://img.shields.io/npm/v/@vladmandic/human.png?style=flat-square) ![NPM Version](https://img.shields.io/npm/v/@vladmandic/human.png?style=flat-square)
![Last Commit](https://img.shields.io/github/last-commit/vladmandic/human?style=flat-square&svg=true) ![Last Commit](https://img.shields.io/github/last-commit/vladmandic/human?style=flat-square&svg=true)
![License](https://img.shields.io/github/license/vladmandic/human?style=flat-square&svg=true) ![License](https://img.shields.io/github/license/vladmandic/human?style=flat-square&svg=true)
![GitHub Status Checks](https://img.shields.io/github/checks-status/vladmandic/human/main?style=flat-square&svg=true) ![GitHub Status Checks](https://img.shields.io/github/checks-status/vladmandic/human/main?style=flat-square&svg=true)
![Vulnerabilities](https://img.shields.io/snyk/vulnerabilities/github/vladmandic/human?style=flat-square&svg=true)
# Human Library # Human Library
@ -13,45 +13,18 @@
<br> <br>
## Highlights JavaScript module using TensorFlow/JS Machine Learning library
- Compatible with most server-side and client-side environments and frameworks - **Browser**:
- Combines multiple machine learning models which can be switched on-demand depending on the use-case Compatible with both desktop and mobile platforms
- Related models are executed in an attention pipeline to provide details when needed Compatible with *CPU*, *WebGL*, *WASM* backends
- Optimized input pre-processing that can enhance image quality of any type of inputs Compatible with *WebWorker* execution
- Detection of frame changes to trigger only required models for improved performance - **NodeJS**:
- Intelligent temporal interpolation to provide smooth results regardless of processing performance Compatible with both software *tfjs-node* and
- Simple unified API GPU accelerated backends *tfjs-node-gpu* using CUDA libraries
- Built-in Image, Video and WebCam handling
[*Jump to Quick Start*](#quick-start)
<br> <br>
## Compatibility
**Browser**:
- Compatible with both desktop and mobile platforms
- Compatible with *WebGPU*, *WebGL*, *WASM*, *CPU* backends
- Compatible with *WebWorker* execution
- Compatible with *WebView*
- Primary platform: *Chromium*-based browsers
- Secondary platform: *Firefox*, *Safari*
**NodeJS**:
- Compatibile with *WASM* backend for executions on architectures where *tensorflow* binaries are not available
- Compatible with *tfjs-node* using software execution via *tensorflow* shared libraries
- Compatible with *tfjs-node* using GPU-accelerated execution via *tensorflow* shared libraries and nVidia CUDA
- Supported versions are from **14.x** to **22.x**
- NodeJS version **23.x** is not supported due to breaking changes and issues with `@tensorflow/tfjs`
<br>
## Releases
- [Release Notes](https://github.com/vladmandic/human/releases)
- [NPM Link](https://www.npmjs.com/package/@vladmandic/human)
## Demos
*Check out [**Simple Live Demo**](https://vladmandic.github.io/human/demo/typescript/index.html) fully annotated app as a good start starting point ([html](https://github.com/vladmandic/human/blob/main/demo/typescript/index.html))([code](https://github.com/vladmandic/human/blob/main/demo/typescript/index.ts))* *Check out [**Simple Live Demo**](https://vladmandic.github.io/human/demo/typescript/index.html) fully annotated app as a good start starting point ([html](https://github.com/vladmandic/human/blob/main/demo/typescript/index.html))([code](https://github.com/vladmandic/human/blob/main/demo/typescript/index.ts))*
*Check out [**Main Live Demo**](https://vladmandic.github.io/human/demo/index.html) app for advanced processing of of webcam, video stream or images static images with all possible tunable options* *Check out [**Main Live Demo**](https://vladmandic.github.io/human/demo/index.html) app for advanced processing of of webcam, video stream or images static images with all possible tunable options*
@ -59,53 +32,49 @@
- To start video detection, simply press *Play* - To start video detection, simply press *Play*
- To process images, simply drag & drop in your Browser window - To process images, simply drag & drop in your Browser window
- Note: For optimal performance, select only models you'd like to use - Note: For optimal performance, select only models you'd like to use
- Note: If you have modern GPU, *WebGL* (default) backend is preferred, otherwise select *WASM* backend - Note: If you have modern GPU, WebGL (default) backend is preferred, otherwise select WASM backend
<br> <br>
## Releases
- [Release Notes](https://github.com/vladmandic/human/releases)
- [NPM Link](https://www.npmjs.com/package/@vladmandic/human)
## Demos
- [**List of all Demo applications**](https://github.com/vladmandic/human/wiki/Demos) - [**List of all Demo applications**](https://github.com/vladmandic/human/wiki/Demos)
- [**Live Examples galery**](https://vladmandic.github.io/human/samples/index.html) - [**Live Examples galery**](https://vladmandic.github.io/human/samples/index.html)
### Browser Demos ### Browser Demos
*All browser demos are self-contained without any external dependencies*
- **Full** [[*Live*]](https://vladmandic.github.io/human/demo/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo): Main browser demo app that showcases all Human capabilities - **Full** [[*Live*]](https://vladmandic.github.io/human/demo/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo): Main browser demo app that showcases all Human capabilities
- **Simple** [[*Live*]](https://vladmandic.github.io/human/demo/typescript/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/typescript): Simple demo in WebCam processing demo in TypeScript - **Simple** [[*Live*]](https://vladmandic.github.io/human/demo/typescript/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/typescript): Simple demo in WebCam processing demo in TypeScript
- **Embedded** [[*Live*]](https://vladmandic.github.io/human/demo/video/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/video/index.html): Even simpler demo with tiny code embedded in HTML file - **Face Match** [[*Live*]](https://vladmandic.github.io/human/demo/facematch/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/facematch): Extract faces from images, calculates face descriptors and simmilarities and matches them to known database
- **Face Detect** [[*Live*]](https://vladmandic.github.io/human/demo/facedetect/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/facedetect): Extract faces from images and processes details
- **Face Match** [[*Live*]](https://vladmandic.github.io/human/demo/facematch/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/facematch): Extract faces from images, calculates face descriptors and similarities and matches them to known database
- **Face ID** [[*Live*]](https://vladmandic.github.io/human/demo/faceid/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/faceid): Runs multiple checks to validate webcam input before performing face match to faces in IndexDB - **Face ID** [[*Live*]](https://vladmandic.github.io/human/demo/faceid/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/faceid): Runs multiple checks to validate webcam input before performing face match to faces in IndexDB
- **Multi-thread** [[*Live*]](https://vladmandic.github.io/human/demo/multithread/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/multithread): Runs each Human module in a separate web worker for highest possible performance - **Multi-thread** [[*Live*]](https://vladmandic.github.io/human/demo/multithread/index.html) [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/multithread): Runs each Human module in a separate web worker for highest possible performance
- **NextJS** [[*Live*]](https://vladmandic.github.io/human-next/out/index.html) [[*Details*]](https://github.com/vladmandic/human-next): Use Human with TypeScript, NextJS and ReactJS - **NextJS** [[*Live*]](https://vladmandic.github.io/human-next/out/index.html) [[*Details*]](https://github.com/vladmandic/human-next): Use Human with TypeScript, NextJS and ReactJS
- **ElectronJS** [[*Details*]](https://github.com/vladmandic/human-electron): Use Human with TypeScript and ElectonJS to create standalone cross-platform apps - **ElectronJS** [[*Details*]](https://github.com/vladmandic/human-electron): Use Human with TypeScript and ElectonJS to create standalone cross-platform apps
- **3D Analysis with BabylonJS** [[*Live*]](https://vladmandic.github.io/human-motion/src/index.html) [[*Details*]](https://github.com/vladmandic/human-motion): 3D tracking and visualization of heead, face, eye, body and hand - **3D Analysis** [[*Live*]](https://vladmandic.github.io/human-motion/src/index.html) [[*Details*]](https://github.com/vladmandic/human-motion): 3D tracking and visualization of heead, face, eye, body and hand
- **VRM Virtual Model Tracking with Three.JS** [[*Live*]](https://vladmandic.github.io/human-three-vrm/src/human-vrm.html) [[*Details*]](https://github.com/vladmandic/human-three-vrm): VR model with head, face, eye, body and hand tracking - **Avatar Bone Mapping** [[*Live*]](https://vladmandic.github.io/human-vrm/src/human-avatar.html) [[*Details*]](https://github.com/vladmandic/human-avatar): Human skeleton with full bone mapping using look and inverse kinematics controllers
- **VRM Virtual Model Tracking with BabylonJS** [[*Live*]](https://vladmandic.github.io/human-bjs-vrm/src/index.html) [[*Details*]](https://github.com/vladmandic/human-bjs-vrm): VR model with head, face, eye, body and hand tracking - **Virtual Model Tracking** [[*Live*]](https://vladmandic.github.io/human-vrm/src/human-vrm.html) [[*Details*]](https://github.com/vladmandic/human-vrm): VR model with head, face, eye, body and hand tracking
### NodeJS Demos ### NodeJS Demos
*NodeJS demos may require extra dependencies which are used to decode inputs* - **Main** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Process images from files, folders or URLs using native methods
*See header of each demo to see its dependencies as they are not automatically installed with `Human`* - **Canvas** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Process image from file or URL and draw results to a new image file using `node-canvas`
- **Video** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Processing of video input using `ffmpeg`
- **WebCam** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Processing of webcam screenshots using `fswebcam`
- **Events** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Showcases usage of `Human` eventing to get notifications on processing
- **Similarity** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Compares two input images for similarity of detected faces
- **Face Match** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/facematch): Parallel processing of face **match** in multiple child worker threads
- **Multiple Workers** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Runs multiple parallel `human` by dispaching them to pool of pre-created worker processes
- **Main** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs/node.js): Process images from files, folders or URLs using native methods
- **Canvas** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs/node-canvas.js): Process image from file or URL and draw results to a new image file using `node-canvas`
- **Video** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs/node-video.js): Processing of video input using `ffmpeg`
- **WebCam** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs/node-webcam.js): Processing of webcam screenshots using `fswebcam`
- **Events** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs/node-event.js): Showcases usage of `Human` eventing to get notifications on processing
- **Similarity** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs/node-similarity.js): Compares two input images for similarity of detected faces
- **Face Match** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/facematch/node-match.js): Parallel processing of face **match** in multiple child worker threads
- **Multiple Workers** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/multithread/node-multiprocess.js): Runs multiple parallel `human` by dispaching them to pool of pre-created worker processes
- **Dynamic Load** [[*Details*]](https://github.com/vladmandic/human/tree/main/demo/nodejs): Loads Human dynamically with multiple different desired backends
## Project pages ## Project pages
- [**Code Repository**](https://github.com/vladmandic/human) - [**Code Repository**](https://github.com/vladmandic/human)
- [**NPM Package**](https://www.npmjs.com/package/@vladmandic/human) - [**NPM Package**](https://www.npmjs.com/package/@vladmandic/human)
- [**Issues Tracker**](https://github.com/vladmandic/human/issues) - [**Issues Tracker**](https://github.com/vladmandic/human/issues)
- [**TypeDoc API Specification - Main class**](https://vladmandic.github.io/human/typedoc/classes/Human.html) - [**TypeDoc API Specification**](https://vladmandic.github.io/human/typedoc/classes/Human.html)
- [**TypeDoc API Specification - Full**](https://vladmandic.github.io/human/typedoc/)
- [**Change Log**](https://github.com/vladmandic/human/blob/main/CHANGELOG.md) - [**Change Log**](https://github.com/vladmandic/human/blob/main/CHANGELOG.md)
- [**Current To-do List**](https://github.com/vladmandic/human/blob/main/TODO.md) - [**Current To-do List**](https://github.com/vladmandic/human/blob/main/TODO.md)
@ -116,7 +85,6 @@
- [**Usage & Functions**](https://github.com/vladmandic/human/wiki/Usage) - [**Usage & Functions**](https://github.com/vladmandic/human/wiki/Usage)
- [**Configuration Details**](https://github.com/vladmandic/human/wiki/Config) - [**Configuration Details**](https://github.com/vladmandic/human/wiki/Config)
- [**Result Details**](https://github.com/vladmandic/human/wiki/Result) - [**Result Details**](https://github.com/vladmandic/human/wiki/Result)
- [**Customizing Draw Methods**](https://github.com/vladmandic/human/wiki/Draw)
- [**Caching & Smoothing**](https://github.com/vladmandic/human/wiki/Caching) - [**Caching & Smoothing**](https://github.com/vladmandic/human/wiki/Caching)
- [**Input Processing**](https://github.com/vladmandic/human/wiki/Image) - [**Input Processing**](https://github.com/vladmandic/human/wiki/Image)
- [**Face Recognition & Face Description**](https://github.com/vladmandic/human/wiki/Embedding) - [**Face Recognition & Face Description**](https://github.com/vladmandic/human/wiki/Embedding)
@ -148,24 +116,27 @@
<hr><br> <hr><br>
## App Examples ## Examples
Visit [Examples gallery](https://vladmandic.github.io/human/samples/index.html) for more examples Visit [Examples galery](https://vladmandic.github.io/human/samples/samples.html) for more examples
[<img src="assets/samples.jpg" width="640"/>](assets/samples.jpg) <https://vladmandic.github.io/human/samples/samples.html>
![samples](assets/samples.jpg)
<br> <br>
## Options ## Options
All options as presented in the demo application... All options as presented in the demo application...
[demo/index.html](demo/index.html) > [demo/index.html](demo/index.html)
[<img src="assets/screenshot-menu.png"/>](assets/screenshot-menu.png)
![Options visible in demo](assets/screenshot-menu.png)
<br> <br>
**Results Browser:** **Results Browser:**
[ *Demo -> Display -> Show Results* ]<br> [ *Demo -> Display -> Show Results* ]<br>
[<img src="assets/screenshot-results.png"/>](assets/screenshot-results.png) ![Results](assets/screenshot-results.png)
<br> <br>
@ -177,47 +148,37 @@ sorts them by similarity to selected face
and optionally matches detected face with database of known people to guess their names and optionally matches detected face with database of known people to guess their names
> [demo/facematch](demo/facematch/index.html) > [demo/facematch](demo/facematch/index.html)
[<img src="assets/screenshot-facematch.jpg" width="640"/>](assets/screenshot-facematch.jpg) ![Face Matching](assets/screenshot-facematch.jpg)
2. **Face Detect:**
Extracts all detect faces from loaded images on-demand and highlights face details on a selected face
> [demo/facedetect](demo/facedetect/index.html)
[<img src="assets/screenshot-facedetect.jpg" width="640"/>](assets/screenshot-facedetect.jpg)
3. **Face ID:**
Performs validation check on a webcam input to detect a real face and matches it to known faces stored in database
> [demo/faceid](demo/faceid/index.html)
[<img src="assets/screenshot-faceid.jpg" width="640"/>](assets/screenshot-faceid.jpg)
<br> <br>
4. **3D Rendering:** 2. **3D Rendering:**
> [human-motion](https://github.com/vladmandic/human-motion) > [human-motion](https://github.com/vladmandic/human-motion)
[<img src="https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-face.jpg" width="640"/>](https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-face.jpg) ![Face3D](https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-face.jpg)
[<img src="https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-body.jpg" width="640"/>](https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-body.jpg) ![Body3D](https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-body.jpg)
[<img src="https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-hand.jpg" width="640"/>](https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-hand.jpg) ![Hand3D](https://github.com/vladmandic/human-motion/raw/main/assets/screenshot-hand.jpg)
<br> <br>
5. **VR Model Tracking:** 3. **Avatar Bone Mapping:**
> [human-three-vrm](https://github.com/vladmandic/human-three-vrm) > [human-avatar](https://github.com/vladmandic/human-avatar)
> [human-bjs-vrm](https://github.com/vladmandic/human-bjs-vrm)
[<img src="https://github.com/vladmandic/human-three-vrm/raw/main/assets/human-vrm-screenshot.jpg" width="640"/>](https://github.com/vladmandic/human-three-vrm/raw/main/assets/human-vrm-screenshot.jpg) ![Avatar](https://github.com/vladmandic/human-avatar/raw/main/assets/screenshot.jpg)
<br>
6. **Human as OS native application:** 4. **VR Model Tracking:**
> [human-electron](https://github.com/vladmandic/human-electron) > [human-vrmmotion](https://github.com/vladmandic/human-vrm)
![VRM](https://github.com/vladmandic/human-vrm/raw/main/assets/human-vrm-screenshot.jpg)
<br> <br>
**468-Point Face Mesh Defails:** **468-Point Face Mesh Defails:**
(view in full resolution to see keypoints) (view in full resolution to see keypoints)
[<img src="assets/facemesh.png" width="400"/>](assets/facemesh.png) ![FaceMesh](assets/facemesh.png)
<br><hr><br> <br><hr><br>
@ -227,25 +188,44 @@ Simply load `Human` (*IIFE version*) directly from a cloud CDN in your HTML file
(pick one: `jsdelirv`, `unpkg` or `cdnjs`) (pick one: `jsdelirv`, `unpkg` or `cdnjs`)
```html ```html
<!DOCTYPE HTML>
<script src="https://cdn.jsdelivr.net/npm/@vladmandic/human/dist/human.js"></script> <script src="https://cdn.jsdelivr.net/npm/@vladmandic/human/dist/human.js"></script>
<script src="https://unpkg.dev/@vladmandic/human/dist/human.js"></script> <script src="https://unpkg.dev/@vladmandic/human/dist/human.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/human/3.0.0/human.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/human/2.1.5/human.js"></script>
``` ```
For details, including how to use `Browser ESM` version or `NodeJS` version of `Human`, see [**Installation**](https://github.com/vladmandic/human/wiki/Install) For details, including how to use `Browser ESM` version or `NodeJS` version of `Human`, see [**Installation**](https://github.com/vladmandic/human/wiki/Install)
<br> <br>
## Code Examples ## Inputs
Simple app that uses Human to process video input and `Human` library can process all known input types:
- `Image`, `ImageData`, `ImageBitmap`, `Canvas`, `OffscreenCanvas`, `Tensor`,
- `HTMLImageElement`, `HTMLCanvasElement`, `HTMLVideoElement`, `HTMLMediaElement`
Additionally, `HTMLVideoElement`, `HTMLMediaElement` can be a standard `<video>` tag that links to:
- WebCam on user's system
- Any supported video type
For example: `.mp4`, `.avi`, etc.
- Additional video types supported via *HTML5 Media Source Extensions*
Live streaming examples:
- **HLS** (*HTTP Live Streaming*) using `hls.js`
- **DASH** (Dynamic Adaptive Streaming over HTTP) using `dash.js`
- **WebRTC** media track using built-in support
<br>
## Example
Example simple app that uses Human to process video input and
draw output on screen using internal draw helper functions draw output on screen using internal draw helper functions
```js ```js
// create instance of human with simple configuration using default values // create instance of human with simple configuration using default values
const config = { backend: 'webgl' }; const config = { backend: 'webgl' };
const human = new Human.Human(config); const human = new Human(config);
// select input HTMLVideoElement and output HTMLCanvasElement from page // select input HTMLVideoElement and output HTMLCanvasElement from page
const inputVideo = document.getElementById('video-id'); const inputVideo = document.getElementById('video-id');
const outputCanvas = document.getElementById('canvas-id'); const outputCanvas = document.getElementById('canvas-id');
@ -264,7 +244,6 @@ function detectVideo() {
human.draw.gesture(outputCanvas, result.gesture); human.draw.gesture(outputCanvas, result.gesture);
// and loop immediate to the next frame // and loop immediate to the next frame
requestAnimationFrame(detectVideo); requestAnimationFrame(detectVideo);
return result;
}); });
} }
@ -325,7 +304,7 @@ async function detectVideo() {
async function drawVideo() { async function drawVideo() {
if (result) { // check if result is available if (result) { // check if result is available
const interpolated = human.next(result); // get smoothened result using last-known results const interpolated = human.next(result); // calculate next interpolated frame
human.draw.all(outputCanvas, interpolated); // draw the frame human.draw.all(outputCanvas, interpolated); // draw the frame
} }
requestAnimationFrame(drawVideo); // run draw loop requestAnimationFrame(drawVideo); // run draw loop
@ -335,108 +314,26 @@ detectVideo(); // start detection loop
drawVideo(); // start draw loop drawVideo(); // start draw loop
``` ```
or same, but using built-in full video processing instead of running manual frame-by-frame loop:
```js
const human = new Human(); // create instance of Human
const inputVideo = document.getElementById('video-id');
const outputCanvas = document.getElementById('canvas-id');
async function drawResults() {
const interpolated = human.next(); // get smoothened result using last-known results
human.draw.all(outputCanvas, interpolated); // draw the frame
requestAnimationFrame(drawResults); // run draw loop
}
human.video(inputVideo); // start detection loop which continously updates results
drawResults(); // start draw loop
```
or using built-in webcam helper methods that take care of video handling completely:
```js
const human = new Human(); // create instance of Human
const outputCanvas = document.getElementById('canvas-id');
async function drawResults() {
const interpolated = human.next(); // get smoothened result using last-known results
human.draw.canvas(outputCanvas, human.webcam.element); // draw current webcam frame
human.draw.all(outputCanvas, interpolated); // draw the frame detectgion results
requestAnimationFrame(drawResults); // run draw loop
}
await human.webcam.start({ crop: true });
human.video(human.webcam.element); // start detection loop which continously updates results
drawResults(); // start draw loop
```
And for even better results, you can run detection in a separate web worker thread And for even better results, you can run detection in a separate web worker thread
<br><hr><br>
## Inputs
`Human` library can process all known input types:
- `Image`, `ImageData`, `ImageBitmap`, `Canvas`, `OffscreenCanvas`, `Tensor`,
- `HTMLImageElement`, `HTMLCanvasElement`, `HTMLVideoElement`, `HTMLMediaElement`
Additionally, `HTMLVideoElement`, `HTMLMediaElement` can be a standard `<video>` tag that links to:
- WebCam on user's system
- Any supported video type
e.g. `.mp4`, `.avi`, etc.
- Additional video types supported via *HTML5 Media Source Extensions*
e.g.: **HLS** (*HTTP Live Streaming*) using `hls.js` or **DASH** (*Dynamic Adaptive Streaming over HTTP*) using `dash.js`
- **WebRTC** media track using built-in support
<br><hr><br>
## Detailed Usage
- [**Wiki Home**](https://github.com/vladmandic/human/wiki)
- [**List of all available methods, properies and namespaces**](https://github.com/vladmandic/human/wiki/Usage)
- [**TypeDoc API Specification - Main class**](https://vladmandic.github.io/human/typedoc/classes/Human.html)
- [**TypeDoc API Specification - Full**](https://vladmandic.github.io/human/typedoc/)
![typedoc](assets/screenshot-typedoc.png)
<br><hr><br>
## TypeDefs
`Human` is written using TypeScript strong typing and ships with full **TypeDefs** for all classes defined by the library bundled in `types/human.d.ts` and enabled by default
*Note*: This does not include embedded `tfjs`
If you want to use embedded `tfjs` inside `Human` (`human.tf` namespace) and still full **typedefs**, add this code:
> import type * as tfjs from '@vladmandic/human/dist/tfjs.esm';
> const tf = human.tf as typeof tfjs;
This is not enabled by default as `Human` does not ship with full **TFJS TypeDefs** due to size considerations
Enabling `tfjs` TypeDefs as above creates additional project (dev-only as only types are required) dependencies as defined in `@vladmandic/human/dist/tfjs.esm.d.ts`:
> @tensorflow/tfjs-core, @tensorflow/tfjs-converter, @tensorflow/tfjs-backend-wasm, @tensorflow/tfjs-backend-webgl
<br><hr><br> <br><hr><br>
## Default models ## Default models
Default models in Human library are: Default models in Human library are:
- **Face Detection**: *MediaPipe BlazeFace Back variation* - **Face Detection**: MediaPipe BlazeFace Back variation
- **Face Mesh**: *MediaPipe FaceMesh* - **Face Mesh**: MediaPipe FaceMesh
- **Face Iris Analysis**: *MediaPipe Iris* - **Face Iris Analysis**: MediaPipe Iris
- **Face Description**: *HSE FaceRes* - **Face Description**: HSE FaceRes
- **Emotion Detection**: *Oarriaga Emotion* - **Emotion Detection**: Oarriaga Emotion
- **Body Analysis**: *MoveNet Lightning variation* - **Body Analysis**: MoveNet Lightning variation
- **Hand Analysis**: *HandTrack & MediaPipe HandLandmarks* - **Hand Analysis**: HandTrack & MediaPipe HandLandmarks
- **Body Segmentation**: *Google Selfie* - **Body Segmentation**: Google Selfie
- **Object Detection**: *CenterNet with MobileNet v3* - **Object Detection**: CenterNet with MobileNet v3
Note that alternative models are provided and can be enabled via configuration Note that alternative models are provided and can be enabled via configuration
For example, body pose detection by default uses *MoveNet Lightning*, but can be switched to *MultiNet Thunder* for higher precision or *Multinet MultiPose* for multi-person detection or even *PoseNet*, *BlazePose* or *EfficientPose* depending on the use case For example, `PoseNet` model can be switched for `BlazePose`, `EfficientPose` or `MoveNet` model depending on the use case
For more info, see [**Configuration Details**](https://github.com/vladmandic/human/wiki/Configuration) and [**List of Models**](https://github.com/vladmandic/human/wiki/Models) For more info, see [**Configuration Details**](https://github.com/vladmandic/human/wiki/Configuration) and [**List of Models**](https://github.com/vladmandic/human/wiki/Models)
@ -448,9 +345,9 @@ For more info, see [**Configuration Details**](https://github.com/vladmandic/hum
<br><hr><br> <br><hr><br>
`Human` library is written in [TypeScript](https://www.typescriptlang.org/docs/handbook/intro.html) **5.1** using [TensorFlow/JS](https://www.tensorflow.org/js/) **4.10** and conforming to latest `JavaScript` [ECMAScript version 2022](https://262.ecma-international.org/) standard `Human` library is written in `TypeScript` [4.6](https://www.typescriptlang.org/docs/handbook/intro.html)
Conforming to latest `JavaScript` [ECMAScript version 2021](https://262.ecma-international.org/) standard
Build target for distributables is `JavaScript` [EMCAScript version 2018](https://262.ecma-international.org/9.0/) Build target is `JavaScript` [EMCAScript version 2018](https://262.ecma-international.org/11.0/)
<br> <br>
@ -459,7 +356,6 @@ and [**API Specification**](https://vladmandic.github.io/human/typedoc/classes/H
<br> <br>
[![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/vladmandic)
![Stars](https://img.shields.io/github/stars/vladmandic/human?style=flat-square&svg=true) ![Stars](https://img.shields.io/github/stars/vladmandic/human?style=flat-square&svg=true)
![Forks](https://badgen.net/github/forks/vladmandic/human) ![Forks](https://badgen.net/github/forks/vladmandic/human)
![Code Size](https://img.shields.io/github/languages/code-size/vladmandic/human?style=flat-square&svg=true) ![Code Size](https://img.shields.io/github/languages/code-size/vladmandic/human?style=flat-square&svg=true)

41
TODO.md
View File

@ -1,38 +1,31 @@
# To-Do list for Human library # To-Do list for Human library
## Work-in-Progress ## Work in Progress
<hr><br> ### Exploring
## Known Issues & Limitations - Optical flow: <https://docs.opencv.org/3.3.1/db/d7f/tutorial_js_lucas_kanade.html>
- Advanced histogram equalization: Adaptive, Contrast Limited, CLAHE
- TFLite models: <https://js.tensorflow.org/api_tflite/0.0.1-alpha.4/>
- Body segmentation: `robust-video-matting`
### Face with Attention <br><hr><br>
`FaceMesh-Attention` is not supported when using `WASM` backend due to missing kernel op in **TFJS** ## Known Issues
No issues with default model `FaceMesh`
### Object Detection #### WebGPU
`NanoDet` model is not supported when using `WASM` backend due to missing kernel op in **TFJS** Experimental support only until support is officially added in Chromium
No issues with default model `MB3-CenterNet`
## Body Detection using MoveNet-MultiPose ### Face Detection
Model does not return valid detection scores (all other functionality is not impacted) Enhanced rotation correction for face detection is not working in NodeJS due to missing kernel op in TFJS
Feature is automatically disabled in NodeJS without user impact
### Firefox - Backend NodeJS missing kernel op `RotateWithOffset`
<https://github.com/tensorflow/tfjs/issues/5473>
Running in **web workers** requires `OffscreenCanvas` which is still disabled by default in **Firefox** <br><hr><br>
Enable via `about:config` -> `gfx.offscreencanvas.enabled`
[Details](https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas#browser_compatibility)
### Safari ## Pending Release Notes
No support for running in **web workers** as Safari still does not support `OffscreenCanvas`
[Details](https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas#browser_compatibility)
## React-Native
`Human` support for **React-Native** is best-effort, but not part of the main development focus
<hr><br>

View File

@ -1,8 +1,9 @@
{ {
"$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json", "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json",
"mainEntryPointFilePath": "types/lib/src/human.d.ts", "mainEntryPointFilePath": "types/lib/src/human.d.ts",
"bundledPackages": ["@types/offscreencanvas", "@tensorflow/tfjs-core", "@tensorflow/tfjs-converter", "@tensorflow/tfjs-data"],
"compiler": { "compiler": {
"skipLibCheck": true "skipLibCheck": false
}, },
"newlineKind": "lf", "newlineKind": "lf",
"dtsRollup": { "dtsRollup": {

Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

140
build.js
View File

@ -1,25 +1,16 @@
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const log = require('@vladmandic/pilogger');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require const Build = require('@vladmandic/build').Build;
const Build = require('@vladmandic/build').Build; // eslint-disable-line node/no-unpublished-require const APIExtractor = require('@microsoft/api-extractor');
const APIExtractor = require('@microsoft/api-extractor'); // eslint-disable-line node/no-unpublished-require
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require
const packageJSON = require('./package.json');
const logFile = 'test/build.log'; function copy(src, dst) {
const modelsOut = 'models/models.json'; if (!fs.existsSync(src)) return;
const modelsFolders = [ const buffer = fs.readFileSync(src);
'./models', fs.writeFileSync(dst, buffer);
'../human-models/models', }
'../blazepose/model/',
'../anti-spoofing/model',
'../efficientpose/models',
'../insightface/models',
'../movenet/models',
'../nanodet/models',
];
const apiExtractorIgnoreList = [ // eslint-disable-line no-unused-vars // eslint-disable-next-line @typescript-eslint/no-unused-vars
const apiExtractorIgnoreList = [
'ae-missing-release-tag', 'ae-missing-release-tag',
'tsdoc-param-tag-missing-hyphen', 'tsdoc-param-tag-missing-hyphen',
'tsdoc-escape-right-brace', 'tsdoc-escape-right-brace',
@ -31,97 +22,15 @@ const apiExtractorIgnoreList = [ // eslint-disable-line no-unused-vars
'tsdoc-unnecessary-backslash', 'tsdoc-unnecessary-backslash',
]; ];
const regEx = [
{ search: 'types="@webgpu/types/dist"', replace: 'path="../src/types/webgpu.d.ts"' },
{ search: 'types="offscreencanvas"', replace: 'path="../src/types/offscreencanvas.d.ts"' },
];
function copyFile(src, dst) {
if (!fs.existsSync(src)) {
log.warn('Copy:', { input: src, output: dst });
return;
}
log.state('Copy:', { input: src, output: dst });
const buffer = fs.readFileSync(src);
fs.writeFileSync(dst, buffer);
}
function writeFile(str, dst) {
log.state('Write:', { output: dst });
fs.writeFileSync(dst, str);
}
function regExFile(src, entries) {
if (!fs.existsSync(src)) {
log.warn('Filter:', { src });
return;
}
log.state('Filter:', { input: src });
for (const entry of entries) {
const buffer = fs.readFileSync(src, 'UTF-8');
const lines = buffer.split(/\r?\n/);
const out = [];
for (const line of lines) {
if (line.includes(entry.search)) out.push(line.replace(entry.search, entry.replace));
else out.push(line);
}
fs.writeFileSync(src, out.join('\n'));
}
}
async function analyzeModels() {
log.info('Analyze models:', { folders: modelsFolders.length, result: modelsOut });
let totalSize = 0;
const models = {};
const allModels = [];
for (const folder of modelsFolders) {
try {
if (!fs.existsSync(folder)) continue;
const stat = fs.statSync(folder);
if (!stat.isDirectory) continue;
const dir = fs.readdirSync(folder);
const found = dir.map((f) => `file://${folder}/${f}`).filter((f) => f.endsWith('json'));
log.state('Models', { folder, models: found.length });
allModels.push(...found);
} catch {
// log.warn('Cannot enumerate:', modelFolder);
}
}
for (const url of allModels) {
// if (!f.endsWith('.json')) continue;
// const url = `file://${modelsDir}/${f}`;
const model = new tf.GraphModel(url); // create model prototype and decide if load from cache or from original modelurl
model.findIOHandler();
const artifacts = await model.handler.load();
const size = artifacts?.weightData?.byteLength || 0;
totalSize += size;
const name = path.basename(url).replace('.json', '');
if (!models[name]) models[name] = size;
}
const json = JSON.stringify(models, null, 2);
fs.writeFileSync(modelsOut, json);
log.state('Models:', { count: Object.keys(models).length, totalSize });
}
async function main() { async function main() {
log.logFile(logFile);
log.data('Build', { name: packageJSON.name, version: packageJSON.version });
// run production build // run production build
const build = new Build(); const build = new Build();
await build.run('production'); await build.run('production');
// patch tfjs typedefs // patch tfjs typedefs
copyFile('node_modules/@vladmandic/tfjs/types/tfjs-core.d.ts', 'types/tfjs-core.d.ts'); log.state('Copy:', { input: 'tfjs/tfjs.esm.d.ts' });
copyFile('node_modules/@vladmandic/tfjs/types/tfjs.d.ts', 'types/tfjs.esm.d.ts'); copy('tfjs/tfjs.esm.d.ts', 'types/lib/dist/tfjs.esm.d.ts');
copyFile('src/types/tsconfig.json', 'types/tsconfig.json');
copyFile('src/types/eslint.json', 'types/.eslintrc.json');
copyFile('src/types/tfjs.esm.d.ts', 'dist/tfjs.esm.d.ts');
regExFile('types/tfjs-core.d.ts', regEx);
// run api-extractor to create typedef rollup // run api-extractor to create typedef rollup
const extractorConfig = APIExtractor.ExtractorConfig.loadFileAndPrepare('.api-extractor.json'); const extractorConfig = APIExtractor.ExtractorConfig.loadFileAndPrepare('api-extractor.json');
try {
const extractorResult = APIExtractor.Extractor.invoke(extractorConfig, { const extractorResult = APIExtractor.Extractor.invoke(extractorConfig, {
localBuild: true, localBuild: true,
showVerboseMessages: false, showVerboseMessages: false,
@ -134,20 +43,15 @@ async function main() {
}, },
}); });
log.state('API-Extractor:', { succeeeded: extractorResult.succeeded, errors: extractorResult.errorCount, warnings: extractorResult.warningCount }); log.state('API-Extractor:', { succeeeded: extractorResult.succeeded, errors: extractorResult.errorCount, warnings: extractorResult.warningCount });
} catch (err) { // distribute typedefs
log.error('API-Extractor:', err); log.state('Copy:', { input: 'types/human.d.ts' });
} copy('types/human.d.ts', 'dist/human.esm-nobundle.d.ts');
regExFile('types/human.d.ts', regEx); copy('types/human.d.ts', 'dist/human.esm.d.ts');
writeFile('export * from \'../types/human\';', 'dist/human.esm-nobundle.d.ts'); copy('types/human.d.ts', 'dist/human.d.ts');
writeFile('export * from \'../types/human\';', 'dist/human.esm.d.ts'); copy('types/human.d.ts', 'dist/human.node-gpu.d.ts');
writeFile('export * from \'../types/human\';', 'dist/human.d.ts'); copy('types/human.d.ts', 'dist/human.node.d.ts');
writeFile('export * from \'../types/human\';', 'dist/human.node-gpu.d.ts'); copy('types/human.d.ts', 'dist/human.node-wasm.d.ts');
writeFile('export * from \'../types/human\';', 'dist/human.node.d.ts'); log.info('Human Build complete...');
writeFile('export * from \'../types/human\';', 'dist/human.node-wasm.d.ts');
// generate model signature
await analyzeModels();
log.info('Human Build complete...', { logFile });
} }
main(); main();

View File

@ -8,7 +8,6 @@ For details on other demos see Wiki: [**Demos**](https://github.com/vladmandic/h
`index.html`: Full demo using `Human` ESM module running in Browsers, `index.html`: Full demo using `Human` ESM module running in Browsers,
Includes: Includes:
- Selectable inputs: - Selectable inputs:
- Sample images - Sample images
- Image via drag & drop - Image via drag & drop
@ -38,14 +37,12 @@ Includes:
in `index.js:ui` in `index.js:ui`
```js ```js
const ui = {
console: true, // log messages to browser console console: true, // log messages to browser console
useWorker: true, // use web workers for processing useWorker: true, // use web workers for processing
buffered: true, // should output be buffered between frames buffered: true, // should output be buffered between frames
interpolated: true, // should output be interpolated for smoothness between frames interpolated: true, // should output be interpolated for smoothness between frames
results: false, // show results tree results: false, // show results tree
useWebRTC: false, // use webrtc as camera source instead of local webcam useWebRTC: false, // use webrtc as camera source instead of local webcam
};
``` ```
Demo implements several ways to use `Human` library, Demo implements several ways to use `Human` library,

4
demo/benchmark/README.md Normal file
View File

@ -0,0 +1,4 @@
# Human Benchmarks
- `node.js` runs benchmark using `tensorflow` backend in **NodeJS**
- `index.html` runs benchmark using `wasm`, `webgl`, `humangl` and `webgpu` backends in **Browser**

86
demo/benchmark/index.html Normal file
View File

@ -0,0 +1,86 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Human</title>
<meta name="viewport" content="width=device-width" id="viewport">
<meta name="keywords" content="Human">
<meta name="application-name" content="Human">
<meta name="description" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="msapplication-tooltip" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="theme-color" content="#000000">
<link rel="manifest" href="../manifest.webmanifest">
<link rel="shortcut icon" href="../../favicon.ico" type="image/x-icon">
<link rel="apple-touch-icon" href="../../assets/icon.png">
<style>
@font-face { font-family: 'Lato'; font-display: swap; font-style: normal; font-weight: 100; src: local('Lato'), url('../../assets/lato-light.woff2') }
html { font-family: 'Lato', 'Segoe UI'; font-size: 16px; font-variant: small-caps; }
body { margin: 0; background: black; color: white; overflow-x: hidden; width: 100vw; height: 100vh; }
body::-webkit-scrollbar { display: none; }
.status { position: absolute; width: 100vw; bottom: 10%; text-align: center; font-size: 3rem; font-weight: 100; text-shadow: 2px 2px #303030; }
.log { position: absolute; bottom: 0; margin: 0.4rem 0.4rem 0 0.4rem; font-size: 0.9rem; }
</style>
</head>
<body>
<div id="status" class="status"></div>
<img id="image" src="../../samples/in/group-1.jpg" alt="test image" style="display: none">
<div id="log" class="log"></div>
<script type="module">
import Human from '../../dist/human.esm.js';
const loop = 20;
const backends = ['wasm', 'webgl', 'humangl', 'webgpu'];
// eslint-disable-next-line no-console
const log = (...msg) => console.log(...msg);
const myConfig = {
modelBasePath: 'https://vladmandic.github.io/human/models',
debug: true,
async: true,
cacheSensitivity: 0,
filter: { enabled: false },
face: {
enabled: true,
detector: { enabled: true, rotation: false },
mesh: { enabled: true },
iris: { enabled: true },
description: { enabled: true },
emotion: { enabled: false },
antispoof: { enabled: true },
liveness: { enabled: true },
},
hand: { enabled: true },
body: { enabled: true },
object: { enabled: true },
};
async function benchmark(backend) {
myConfig.backend = backend;
const human = new Human(myConfig);
await human.tf.ready();
log('Human:', human.version);
await human.load();
const loaded = Object.keys(human.models).filter((a) => human.models[a]);
log('Loaded:', loaded);
log('Memory state:', human.tf.engine().memory());
const element = document.getElementById('image');
const processed = await human.image(element);
const t0 = human.now();
await human.detect(processed.tensor, myConfig);
const t1 = human.now();
log('Backend:', human.tf.getBackend());
log('Warmup:', Math.round(t1 - t0));
for (let i = 0; i < loop; i++) await human.detect(processed.tensor, myConfig);
const t2 = human.now();
log('Average:', Math.round((t2 - t1) / loop));
}
async function main() {
for (const backend of backends) await benchmark(backend);
}
window.onload = main;
</script>
</body>
</html>

65
demo/benchmark/node.js Normal file
View File

@ -0,0 +1,65 @@
// eslint-disable-next-line no-unused-vars, @typescript-eslint/no-unused-vars
const tf = require('@tensorflow/tfjs-node-gpu');
const log = require('@vladmandic/pilogger');
const canvasJS = require('canvas');
const Human = require('../../dist/human.node-gpu.js').default;
const input = './samples/in/group-1.jpg';
const loop = 20;
const myConfig = {
backend: 'tensorflow',
modelBasePath: 'https://vladmandic.github.io/human/models',
wasmPath: 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@3.9.0/dist/',
debug: true,
async: true,
cacheSensitivity: 0,
filter: { enabled: false },
face: {
enabled: true,
detector: { enabled: true, rotation: false },
mesh: { enabled: true },
iris: { enabled: true },
description: { enabled: true },
emotion: { enabled: true },
antispoof: { enabled: true },
liveness: { enabled: true },
},
hand: { enabled: true },
body: { enabled: true },
object: { enabled: true },
};
async function getImage(human) {
const img = await canvasJS.loadImage(input);
const canvas = canvasJS.createCanvas(img.width, img.height);
const ctx = canvas.getContext('2d');
ctx.drawImage(img, 0, 0, img.width, img.height);
const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
const tensor = human.tf.tensor(Array.from(imageData.data), [canvas.height, canvas.width, 4], 'int32'); // create rgba image tensor from flat array
log.info('Image:', input, tensor.shape);
return tensor;
}
async function main() {
log.header();
const human = new Human(myConfig);
await human.tf.ready();
log.info('Human:', human.version);
await human.load();
const loaded = Object.keys(human.models).filter((a) => human.models[a]);
log.info('Loaded:', loaded);
log.info('Memory state:', human.tf.engine().memory());
const tensor = await getImage(human);
log.state('Processing:', tensor['shape']);
const t0 = human.now();
await human.detect(tensor, myConfig);
const t1 = human.now();
log.state('Backend:', human.tf.getBackend());
log.data('Warmup:', Math.round(t1 - t0));
for (let i = 0; i < loop; i++) await human.detect(tensor, myConfig);
const t2 = human.now();
log.data('Average:', Math.round((t2 - t1) / loop));
}
main();

View File

@ -1,160 +0,0 @@
/**
* Human demo for browsers
*
* Demo for face detection
*/
/** @type {Human} */
import { Human } from '../../dist/human.esm.js';
let loader;
const humanConfig = { // user configuration for human, used to fine-tune behavior
cacheSensitivity: 0,
debug: true,
modelBasePath: 'https://vladmandic.github.io/human-models/models/',
filter: { enabled: true, equalization: false, flip: false },
face: {
enabled: true,
detector: { rotation: false, maxDetected: 100, minConfidence: 0.2, return: true, square: false },
iris: { enabled: true },
description: { enabled: true },
emotion: { enabled: true },
antispoof: { enabled: true },
liveness: { enabled: true },
},
body: { enabled: false },
hand: { enabled: false },
object: { enabled: false },
gesture: { enabled: false },
segmentation: { enabled: false },
};
const human = new Human(humanConfig); // new instance of human
export const showLoader = (msg) => { loader.setAttribute('msg', msg); loader.style.display = 'block'; };
export const hideLoader = () => loader.style.display = 'none';
class ComponentLoader extends HTMLElement { // watch for attributes
message = document.createElement('div');
static get observedAttributes() { return ['msg']; }
attributeChangedCallback(_name, _prevVal, currVal) {
this.message.innerHTML = currVal;
}
connectedCallback() { // triggered on insert
this.attachShadow({ mode: 'open' });
const css = document.createElement('style');
css.innerHTML = `
.loader-container { top: 450px; justify-content: center; position: fixed; width: 100%; }
.loader-message { font-size: 1.5rem; padding: 1rem; }
.loader { width: 300px; height: 300px; border: 3px solid transparent; border-radius: 50%; border-top: 4px solid #f15e41; animation: spin 4s linear infinite; position: relative; }
.loader::before, .loader::after { content: ""; position: absolute; top: 6px; bottom: 6px; left: 6px; right: 6px; border-radius: 50%; border: 4px solid transparent; }
.loader::before { border-top-color: #bad375; animation: 3s spin linear infinite; }
.loader::after { border-top-color: #26a9e0; animation: spin 1.5s linear infinite; }
@keyframes spin { from { transform: rotate(0deg); } to { transform: rotate(360deg); } }
`;
const container = document.createElement('div');
container.id = 'loader-container';
container.className = 'loader-container';
loader = document.createElement('div');
loader.id = 'loader';
loader.className = 'loader';
this.message.id = 'loader-message';
this.message.className = 'loader-message';
this.message.innerHTML = '';
container.appendChild(this.message);
container.appendChild(loader);
this.shadowRoot?.append(css, container);
loader = this; // eslint-disable-line @typescript-eslint/no-this-alias
}
}
customElements.define('component-loader', ComponentLoader);
function addFace(face, source) {
const deg = (rad) => Math.round((rad || 0) * 180 / Math.PI);
const canvas = document.createElement('canvas');
const emotion = face.emotion?.map((e) => `${Math.round(100 * e.score)}% ${e.emotion}`) || [];
const rotation = `pitch ${deg(face.rotation?.angle.pitch)}° | roll ${deg(face.rotation?.angle.roll)}° | yaw ${deg(face.rotation?.angle.yaw)}°`;
const gaze = `direction ${deg(face.rotation?.gaze.bearing)}° strength ${Math.round(100 * (face.rotation.gaze.strength || 0))}%`;
canvas.title = `
source: ${source}
score: ${Math.round(100 * face.boxScore)}% detection ${Math.round(100 * face.faceScore)}% analysis
age: ${face.age} years | gender: ${face.gender} score ${Math.round(100 * face.genderScore)}%
emotion: ${emotion.join(' | ')}
head rotation: ${rotation}
eyes gaze: ${gaze}
camera distance: ${face.distance}m | ${Math.round(100 * face.distance / 2.54)}in
check: ${Math.round(100 * face.real)}% real ${Math.round(100 * face.live)}% live
`.replace(/ /g, ' ');
canvas.onclick = (e) => {
e.preventDefault();
document.getElementById('description').innerHTML = canvas.title;
};
human.draw.tensor(face.tensor, canvas);
human.tf.dispose(face.tensor);
return canvas;
}
async function addFaces(imgEl) {
showLoader('human: busy');
const faceEl = document.getElementById('faces');
faceEl.innerHTML = '';
const res = await human.detect(imgEl);
console.log(res); // eslint-disable-line no-console
document.getElementById('description').innerHTML = `detected ${res.face.length} faces`;
for (const face of res.face) {
const canvas = addFace(face, imgEl.src.substring(0, 64));
faceEl.appendChild(canvas);
}
hideLoader();
}
function addImage(imageUri) {
const imgEl = new Image(256, 256);
imgEl.onload = () => {
const images = document.getElementById('images');
images.appendChild(imgEl); // add image if loaded ok
images.scroll(images?.offsetWidth, 0);
};
imgEl.onerror = () => console.error('addImage', { imageUri }); // eslint-disable-line no-console
imgEl.onclick = () => addFaces(imgEl);
imgEl.title = imageUri.substring(0, 64);
imgEl.src = encodeURI(imageUri);
}
async function initDragAndDrop() {
const reader = new FileReader();
reader.onload = async (e) => {
if (e.target.result.startsWith('data:image')) await addImage(e.target.result);
};
document.body.addEventListener('dragenter', (evt) => evt.preventDefault());
document.body.addEventListener('dragleave', (evt) => evt.preventDefault());
document.body.addEventListener('dragover', (evt) => evt.preventDefault());
document.body.addEventListener('drop', async (evt) => {
evt.preventDefault();
evt.dataTransfer.dropEffect = 'copy';
for (const f of evt.dataTransfer.files) reader.readAsDataURL(f);
});
document.body.onclick = (e) => {
if (e.target.localName !== 'canvas') document.getElementById('description').innerHTML = '';
};
}
async function main() {
showLoader('loading models');
await human.load();
showLoader('compiling models');
await human.warmup();
showLoader('loading images');
const images = ['group-1.jpg', 'group-2.jpg', 'group-3.jpg', 'group-4.jpg', 'group-5.jpg', 'group-6.jpg', 'group-7.jpg', 'solvay1927.jpg', 'stock-group-1.jpg', 'stock-group-2.jpg', 'stock-models-6.jpg', 'stock-models-7.jpg'];
const imageUris = images.map((a) => `../../samples/in/${a}`);
for (let i = 0; i < imageUris.length; i++) addImage(imageUris[i]);
initDragAndDrop();
hideLoader();
}
window.onload = main;

View File

@ -1,43 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Human</title>
<!-- <meta http-equiv="content-type" content="text/html; charset=utf-8"> -->
<meta name="viewport" content="width=device-width, shrink-to-fit=yes">
<meta name="keywords" content="Human">
<meta name="application-name" content="Human">
<meta name="description" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="msapplication-tooltip" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="theme-color" content="#000000">
<link rel="manifest" href="../manifest.webmanifest">
<link rel="shortcut icon" href="../../favicon.ico" type="image/x-icon">
<link rel="apple-touch-icon" href="../../assets/icon.png">
<script src="./facedetect.js" type="module"></script>
<style>
img { object-fit: contain; }
img:hover { filter: grayscale(1); transform: scale(1.08); transition : all 0.3s ease; }
@font-face { font-family: 'Lato'; font-display: swap; font-style: normal; font-weight: 100; src: local('Lato'), url('../../assets/lato-light.woff2') }
html { font-family: 'Lato', 'Segoe UI'; font-size: 24px; font-variant: small-caps; }
body { margin: 24px; background: black; color: white; overflow: hidden; text-align: -webkit-center; width: 100vw; height: 100vh; }
::-webkit-scrollbar { height: 8px; border: 0; border-radius: 0; }
::-webkit-scrollbar-thumb { background: grey }
::-webkit-scrollbar-track { margin: 3px; }
canvas { width: 192px; height: 192px; margin: 2px; padding: 2px; cursor: grab; transform: scale(1.00); transition : all 0.3s ease; }
canvas:hover { filter: grayscale(1); transform: scale(1.08); transition : all 0.3s ease; }
</style>
</head>
<body>
<component-loader></component-loader>
<div style="display: flex">
<div>
<div style="margin: 24px">select image to show detected faces<br>drag & drop to add your images</div>
<div id="images" style="display: flex; width: 98vw; overflow-x: auto; overflow-y: hidden; scroll-behavior: smooth"></div>
</div>
</div>
<div id="list" style="height: 10px"></div>
<div style="margin: 24px">hover or click on face to show details</div>
<div id="faces" style="overflow-y: auto"></div>
<div id="description" style="white-space: pre;"></div>
</body>
</html>

View File

@ -38,5 +38,4 @@ designed to serve as a quick check when used together with other indicators:
**FaceID** is compatible with **FaceID** is compatible with
- `faceres.json` (default) perfoms combined age/gender/descriptor analysis - `faceres.json` (default) perfoms combined age/gender/descriptor analysis
- `faceres-deep.json` higher resolution variation of `faceres` - `faceres-deep.json` higher resolution variation of `faceres`
- `insightface` alternative model for face descriptor analysis
- `mobilefacenet` alternative model for face descriptor analysis - `mobilefacenet` alternative model for face descriptor analysis

View File

@ -19,23 +19,14 @@
body { margin: 0; padding: 16px; background: black; color: white; overflow-x: hidden; width: 100vw; height: 100vh; } body { margin: 0; padding: 16px; background: black; color: white; overflow-x: hidden; width: 100vw; height: 100vh; }
body::-webkit-scrollbar { display: none; } body::-webkit-scrollbar { display: none; }
.button { padding: 2px; cursor: pointer; box-shadow: 2px 2px black; width: 64px; text-align: center; place-content: center; margin-left: 16px; height: 16px; display: none } .button { padding: 2px; cursor: pointer; box-shadow: 2px 2px black; width: 64px; text-align: center; place-content: center; margin-left: 16px; height: 16px; display: none }
.ok { position: absolute; top: 64px; right: 20px; width: 150px; background-color: grey; padding: 4px; color: black; font-size: 14px } .ok { position: absolute; top: 64px; right: 20px; width: 100px; background-color: grey; padding: 4px; color: black; font-size: 14px }
</style> </style>
</head> </head>
<body> <body>
<div style="padding: 8px">
<h1 style="margin: 0">faceid demo using human library</h1>
look directly at camera and make sure that detection passes all of the required tests noted on the right hand side of the screen<br>
if input does not satisfies tests within specific timeout, no image will be selected<br>
once face image is approved, it will be compared with existing face database<br>
you can also store face descriptor with label in a browser's indexdb for future usage<br>
<br>
<i>note: this is not equivalent to full faceid methods as used by modern mobile phones or windows hello<br>
as they rely on additional infrared sensors and depth-sensing and not just camera image for additional levels of security</i>
</div>
<canvas id="canvas" style="padding: 8px"></canvas> <canvas id="canvas" style="padding: 8px"></canvas>
<canvas id="source" style="padding: 8px"></canvas> <canvas id="source" style="padding: 8px"></canvas>
<video id="video" playsinline style="display: none"></video> <video id="video" playsinline style="display: none"></video>
<pre id="fps" style="position: absolute; bottom: 16px; right: 20px; background-color: grey; padding: 8px; box-shadow: 2px 2px black"></pre>
<pre id="log" style="padding: 8px"></pre> <pre id="log" style="padding: 8px"></pre>
<div id="match" style="display: none; padding: 8px"> <div id="match" style="display: none; padding: 8px">
<label for="name">name:</label> <label for="name">name:</label>
@ -43,7 +34,7 @@
<span id="save" class="button" style="background-color: royalblue">save</span> <span id="save" class="button" style="background-color: royalblue">save</span>
<span id="delete" class="button" style="background-color: lightcoral">delete</span> <span id="delete" class="button" style="background-color: lightcoral">delete</span>
</div> </div>
<div id="retry" class="button" style="background-color: darkslategray; width: 93%; margin-top: 32px; padding: 12px">retry</div> <div id="retry" class="button" style="background-color: darkslategray; width: 350px; margin-top: 32px; padding: 4px">retry</div>
<div id="ok"></div> <div id="ok"></div>
</body> </body>
</html> </html>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -7,20 +7,17 @@
* @license MIT * @license MIT
*/ */
import * as H from '../../dist/human.esm.js'; // equivalent of @vladmandic/Human import { Human, TensorLike, FaceResult } from '../../dist/human.esm.js'; // equivalent of @vladmandic/Human
import * as indexDb from './indexdb'; // methods to deal with indexdb import * as indexDb from './indexdb'; // methods to deal with indexdb
const humanConfig = { // user configuration for human, used to fine-tune behavior const humanConfig = { // user configuration for human, used to fine-tune behavior
cacheSensitivity: 0.01,
modelBasePath: '../../models', modelBasePath: '../../models',
filter: { enabled: true, equalization: true }, // lets run with histogram equilizer filter: { equalization: true }, // lets run with histogram equilizer
debug: true,
face: { face: {
enabled: true, enabled: true,
detector: { rotation: true, return: true, mask: false }, // return tensor is used to get detected face image detector: { rotation: true, return: true, cropFactor: 1.6, mask: false }, // return tensor is used to get detected face image
description: { enabled: true }, // default model for face descriptor extraction is faceres description: { enabled: true }, // default model for face descriptor extraction is faceres
// mobilefacenet: { enabled: true, modelPath: 'https://vladmandic.github.io/human-models/models/mobilefacenet.json' }, // alternative model mobilefacenet: { enabled: false, modelPath: 'https://vladmandic.github.io/human-models/models/mobilefacenet.json' }, // alternative model
// insightface: { enabled: true, modelPath: 'https://vladmandic.github.io/insightface/models/insightface-mobilenet-swish.json' }, // alternative model
iris: { enabled: true }, // needed to determine gaze direction iris: { enabled: true }, // needed to determine gaze direction
emotion: { enabled: false }, // not needed emotion: { enabled: false }, // not needed
antispoof: { enabled: true }, // enable optional antispoof module antispoof: { enabled: true }, // enable optional antispoof module
@ -38,50 +35,29 @@ const matchOptions = { order: 2, multiplier: 25, min: 0.2, max: 0.8 }; // for fa
const options = { const options = {
minConfidence: 0.6, // overal face confidence for box, face, gender, real, live minConfidence: 0.6, // overal face confidence for box, face, gender, real, live
minSize: 224, // min input to face descriptor model before degradation minSize: 224, // min input to face descriptor model before degradation
maxTime: 30000, // max time before giving up maxTime: 10000, // max time before giving up
blinkMin: 10, // minimum duration of a valid blink blinkMin: 10, // minimum duration of a valid blink
blinkMax: 800, // maximum duration of a valid blink blinkMax: 800, // maximum duration of a valid blink
threshold: 0.5, // minimum similarity threshold: 0.5, // minimum similarity
distanceMin: 0.4, // closest that face is allowed to be to the cammera in cm
distanceMax: 1.0, // farthest that face is allowed to be to the cammera in cm
mask: humanConfig.face.detector.mask, mask: humanConfig.face.detector.mask,
rotation: humanConfig.face.detector.rotation, rotation: humanConfig.face.detector.rotation,
cropFactor: humanConfig.face.detector.cropFactor,
...matchOptions, ...matchOptions,
}; };
const ok: Record<string, { status: boolean | undefined, val: number }> = { // must meet all rules const ok = { // must meet all rules
faceCount: { status: false, val: 0 }, faceCount: false,
faceConfidence: { status: false, val: 0 }, faceConfidence: false,
facingCenter: { status: false, val: 0 }, facingCenter: false,
lookingCenter: { status: false, val: 0 }, lookingCenter: false,
blinkDetected: { status: false, val: 0 }, blinkDetected: false,
faceSize: { status: false, val: 0 }, faceSize: false,
antispoofCheck: { status: false, val: 0 }, antispoofCheck: false,
livenessCheck: { status: false, val: 0 }, livenessCheck: false,
distance: { status: false, val: 0 }, elapsedMs: 0, // total time while waiting for valid face
age: { status: false, val: 0 },
gender: { status: false, val: 0 },
timeout: { status: true, val: 0 },
descriptor: { status: false, val: 0 },
elapsedMs: { status: undefined, val: 0 }, // total time while waiting for valid face
detectFPS: { status: undefined, val: 0 }, // mark detection fps performance
drawFPS: { status: undefined, val: 0 }, // mark redraw fps performance
}; };
const allOk = () => ok.faceCount && ok.faceSize && ok.blinkDetected && ok.facingCenter && ok.lookingCenter && ok.faceConfidence && ok.antispoofCheck && ok.livenessCheck;
const allOk = () => ok.faceCount.status const current: { face: FaceResult | null, record: indexDb.FaceRecord | null } = { face: null, record: null }; // current face record and matched database record
&& ok.faceSize.status
&& ok.blinkDetected.status
&& ok.facingCenter.status
&& ok.lookingCenter.status
&& ok.faceConfidence.status
&& ok.antispoofCheck.status
&& ok.livenessCheck.status
&& ok.distance.status
&& ok.descriptor.status
&& ok.age.status
&& ok.gender.status;
const current: { face: H.FaceResult | null, record: indexDb.FaceRecord | null } = { face: null, record: null }; // current face record and matched database record
const blink = { // internal timers for blink start/end/duration const blink = { // internal timers for blink start/end/duration
start: 0, start: 0,
@ -90,9 +66,9 @@ const blink = { // internal timers for blink start/end/duration
}; };
// let db: Array<{ name: string, source: string, embedding: number[] }> = []; // holds loaded face descriptor database // let db: Array<{ name: string, source: string, embedding: number[] }> = []; // holds loaded face descriptor database
const human = new H.Human(humanConfig); // create instance of human with overrides from user configuration const human = new Human(humanConfig); // create instance of human with overrides from user configuration
human.env.perfadd = false; // is performance data showing instant or total values human.env['perfadd'] = false; // is performance data showing instant or total values
human.draw.options.font = 'small-caps 18px "Lato"'; // set font used to draw labels when using draw methods human.draw.options.font = 'small-caps 18px "Lato"'; // set font used to draw labels when using draw methods
human.draw.options.lineHeight = 20; human.draw.options.lineHeight = 20;
@ -110,110 +86,97 @@ const dom = { // grab instances of dom objects so we dont have to look them up l
ok: document.getElementById('ok') as HTMLDivElement, ok: document.getElementById('ok') as HTMLDivElement,
}; };
const timestamp = { detect: 0, draw: 0 }; // holds information used to calculate performance and possible memory leaks const timestamp = { detect: 0, draw: 0 }; // holds information used to calculate performance and possible memory leaks
const fps = { detect: 0, draw: 0 }; // holds calculated fps information for both detect and screen refresh
let startTime = 0; let startTime = 0;
const log = (...msg) => { // helper method to output messages const log = (...msg) => { // helper method to output messages
dom.log.innerText += msg.join(' ') + '\n'; dom.log.innerText += msg.join(' ') + '\n';
console.log(...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.log(...msg);
}; };
const printFPS = (msg) => dom.fps.innerText = msg; // print status element
async function webCam() { // initialize webcam async function webCam() { // initialize webcam
printFPS('starting webcam...');
// @ts-ignore resizeMode is not yet defined in tslib // @ts-ignore resizeMode is not yet defined in tslib
const cameraOptions: MediaStreamConstraints = { audio: false, video: { facingMode: 'user', resizeMode: 'none', width: { ideal: document.body.clientWidth } } }; const cameraOptions: MediaStreamConstraints = { audio: false, video: { facingMode: 'user', resizeMode: 'none', width: { ideal: document.body.clientWidth } } };
const stream: MediaStream = await navigator.mediaDevices.getUserMedia(cameraOptions); const stream: MediaStream = await navigator.mediaDevices.getUserMedia(cameraOptions);
const ready = new Promise((resolve) => { dom.video.onloadeddata = () => resolve(true); }); const ready = new Promise((resolve) => { dom.video.onloadeddata = () => resolve(true); });
dom.video.srcObject = stream; dom.video.srcObject = stream;
void dom.video.play(); dom.video.play();
await ready; await ready;
dom.canvas.width = dom.video.videoWidth; dom.canvas.width = dom.video.videoWidth;
dom.canvas.height = dom.video.videoHeight; dom.canvas.height = dom.video.videoHeight;
dom.canvas.style.width = '50%';
dom.canvas.style.height = '50%';
if (human.env.initial) log('video:', dom.video.videoWidth, dom.video.videoHeight, '|', stream.getVideoTracks()[0].label); if (human.env.initial) log('video:', dom.video.videoWidth, dom.video.videoHeight, '|', stream.getVideoTracks()[0].label);
dom.canvas.onclick = () => { // pause when clicked on screen and resume on next click dom.canvas.onclick = () => { // pause when clicked on screen and resume on next click
if (dom.video.paused) void dom.video.play(); if (dom.video.paused) dom.video.play();
else dom.video.pause(); else dom.video.pause();
}; };
} }
async function detectionLoop() { // main detection loop async function detectionLoop() { // main detection loop
if (!dom.video.paused) { if (!dom.video.paused) {
if (current.face?.tensor) human.tf.dispose(current.face.tensor); // dispose previous tensor if (current.face && current.face.tensor) human.tf.dispose(current.face.tensor); // dispose previous tensor
await human.detect(dom.video); // actual detection; were not capturing output in a local variable as it can also be reached via human.result await human.detect(dom.video); // actual detection; were not capturing output in a local variable as it can also be reached via human.result
const now = human.now(); const now = human.now();
ok.detectFPS.val = Math.round(10000 / (now - timestamp.detect)) / 10; fps.detect = 1000 / (now - timestamp.detect);
timestamp.detect = now; timestamp.detect = now;
requestAnimationFrame(detectionLoop); // start new frame immediately requestAnimationFrame(detectionLoop); // start new frame immediately
} }
} }
function drawValidationTests() { async function validationLoop(): Promise<FaceResult> { // main screen refresh loop
const interpolated = await human.next(human.result); // smoothen result using last-known results
await human.draw.canvas(dom.video, dom.canvas); // draw canvas to screen
await human.draw.all(dom.canvas, interpolated); // draw labels, boxes, lines, etc.
const now = human.now();
fps.draw = 1000 / (now - timestamp.draw);
timestamp.draw = now;
printFPS(`fps: ${fps.detect.toFixed(1).padStart(5, ' ')} detect | ${fps.draw.toFixed(1).padStart(5, ' ')} draw`); // write status
ok.faceCount = human.result.face.length === 1; // must be exactly detected face
if (ok.faceCount) { // skip the rest if no face
const gestures: string[] = Object.values(human.result.gesture).map((gesture) => gesture.gesture); // flatten all gestures
if (gestures.includes('blink left eye') || gestures.includes('blink right eye')) blink.start = human.now(); // blink starts when eyes get closed
if (blink.start > 0 && !gestures.includes('blink left eye') && !gestures.includes('blink right eye')) blink.end = human.now(); // if blink started how long until eyes are back open
ok.blinkDetected = ok.blinkDetected || (Math.abs(blink.end - blink.start) > options.blinkMin && Math.abs(blink.end - blink.start) < options.blinkMax);
if (ok.blinkDetected && blink.time === 0) blink.time = Math.trunc(blink.end - blink.start);
ok.facingCenter = gestures.includes('facing center');
ok.lookingCenter = gestures.includes('looking center'); // must face camera and look at camera
ok.faceConfidence = (human.result.face[0].boxScore || 0) > options.minConfidence && (human.result.face[0].faceScore || 0) > options.minConfidence && (human.result.face[0].genderScore || 0) > options.minConfidence;
ok.antispoofCheck = (human.result.face[0].real || 0) > options.minConfidence;
ok.livenessCheck = (human.result.face[0].live || 0) > options.minConfidence;
ok.faceSize = human.result.face[0].box[2] >= options.minSize && human.result.face[0].box[3] >= options.minSize;
}
let y = 32; let y = 32;
for (const [key, val] of Object.entries(ok)) { for (const [key, val] of Object.entries(ok)) {
let el = document.getElementById(`ok-${key}`); let el = document.getElementById(`ok-${key}`);
if (!el) { if (!el) {
el = document.createElement('div'); el = document.createElement('div');
el.id = `ok-${key}`;
el.innerText = key; el.innerText = key;
el.className = 'ok'; el.className = 'ok';
el.style.top = `${y}px`; el.style.top = `${y}px`;
dom.ok.appendChild(el); dom.ok.appendChild(el);
} }
if (typeof val.status === 'boolean') el.style.backgroundColor = val.status ? 'lightgreen' : 'lightcoral'; if (typeof val === 'boolean') el.style.backgroundColor = val ? 'lightgreen' : 'lightcoral';
const status = val.status ? 'ok' : 'fail'; else el.innerText = `${key}:${val}`;
el.innerText = `${key}: ${val.val === 0 ? status : val.val}`;
y += 28; y += 28;
} }
} if (allOk()) { // all criteria met
async function validationLoop(): Promise<H.FaceResult> { // main screen refresh loop
const interpolated = human.next(human.result); // smoothen result using last-known results
human.draw.canvas(dom.video, dom.canvas); // draw canvas to screen
await human.draw.all(dom.canvas, interpolated); // draw labels, boxes, lines, etc.
const now = human.now();
ok.drawFPS.val = Math.round(10000 / (now - timestamp.draw)) / 10;
timestamp.draw = now;
ok.faceCount.val = human.result.face.length;
ok.faceCount.status = ok.faceCount.val === 1; // must be exactly detected face
if (ok.faceCount.status) { // skip the rest if no face
const gestures: string[] = Object.values(human.result.gesture).map((gesture: H.GestureResult) => gesture.gesture); // flatten all gestures
if (gestures.includes('blink left eye') || gestures.includes('blink right eye')) blink.start = human.now(); // blink starts when eyes get closed
if (blink.start > 0 && !gestures.includes('blink left eye') && !gestures.includes('blink right eye')) blink.end = human.now(); // if blink started how long until eyes are back open
ok.blinkDetected.status = ok.blinkDetected.status || (Math.abs(blink.end - blink.start) > options.blinkMin && Math.abs(blink.end - blink.start) < options.blinkMax);
if (ok.blinkDetected.status && blink.time === 0) blink.time = Math.trunc(blink.end - blink.start);
ok.facingCenter.status = gestures.includes('facing center');
ok.lookingCenter.status = gestures.includes('looking center'); // must face camera and look at camera
ok.faceConfidence.val = human.result.face[0].faceScore || human.result.face[0].boxScore || 0;
ok.faceConfidence.status = ok.faceConfidence.val >= options.minConfidence;
ok.antispoofCheck.val = human.result.face[0].real || 0;
ok.antispoofCheck.status = ok.antispoofCheck.val >= options.minConfidence;
ok.livenessCheck.val = human.result.face[0].live || 0;
ok.livenessCheck.status = ok.livenessCheck.val >= options.minConfidence;
ok.faceSize.val = Math.min(human.result.face[0].box[2], human.result.face[0].box[3]);
ok.faceSize.status = ok.faceSize.val >= options.minSize;
ok.distance.val = human.result.face[0].distance || 0;
ok.distance.status = (ok.distance.val >= options.distanceMin) && (ok.distance.val <= options.distanceMax);
ok.descriptor.val = human.result.face[0].embedding?.length || 0;
ok.descriptor.status = ok.descriptor.val > 0;
ok.age.val = human.result.face[0].age || 0;
ok.age.status = ok.age.val > 0;
ok.gender.val = human.result.face[0].genderScore || 0;
ok.gender.status = ok.gender.val >= options.minConfidence;
}
// run again
ok.timeout.status = ok.elapsedMs.val <= options.maxTime;
drawValidationTests();
if (allOk() || !ok.timeout.status) { // all criteria met
dom.video.pause(); dom.video.pause();
return human.result.face[0]; return human.result.face[0];
} }
ok.elapsedMs.val = Math.trunc(human.now() - startTime); if (ok.elapsedMs > options.maxTime) { // give up
dom.video.pause();
return human.result.face[0];
} else { // run again
ok.elapsedMs = Math.trunc(human.now() - startTime);
return new Promise((resolve) => { return new Promise((resolve) => {
setTimeout(async () => { setTimeout(async () => {
await validationLoop(); // run validation loop until conditions are met const res = await validationLoop(); // run validation loop until conditions are met
resolve(human.result.face[0]); // recursive promise resolve if (res) resolve(human.result.face[0]); // recursive promise resolve
}, 30); // use to slow down refresh from max refresh rate to target of 30 fps }, 30); // use to slow down refresh from max refresh rate to target of 30 fps
}); });
}
} }
async function saveRecords() { async function saveRecords() {
@ -221,8 +184,7 @@ async function saveRecords() {
const image = dom.canvas.getContext('2d')?.getImageData(0, 0, dom.canvas.width, dom.canvas.height) as ImageData; const image = dom.canvas.getContext('2d')?.getImageData(0, 0, dom.canvas.width, dom.canvas.height) as ImageData;
const rec = { id: 0, name: dom.name.value, descriptor: current.face?.embedding as number[], image }; const rec = { id: 0, name: dom.name.value, descriptor: current.face?.embedding as number[], image };
await indexDb.save(rec); await indexDb.save(rec);
log('saved face record:', rec.name, 'descriptor length:', current.face?.embedding?.length); log('saved face record:', rec.name);
log('known face records:', await indexDb.count());
} else { } else {
log('invalid name'); log('invalid name');
} }
@ -235,21 +197,20 @@ async function deleteRecord() {
} }
async function detectFace() { async function detectFace() {
dom.canvas.style.height = '';
dom.canvas.getContext('2d')?.clearRect(0, 0, options.minSize, options.minSize); dom.canvas.getContext('2d')?.clearRect(0, 0, options.minSize, options.minSize);
if (!current?.face?.tensor || !current?.face?.embedding) return false; if (!current.face || !current.face.tensor || !current.face.embedding) return false;
console.log('face record:', current.face); // eslint-disable-line no-console // eslint-disable-next-line no-console
log(`detected face: ${current.face.gender} ${current.face.age || 0}y distance ${100 * (current.face.distance || 0)}cm/${Math.round(100 * (current.face.distance || 0) / 2.54)}in`); console.log('face record:', current.face);
await human.draw.tensor(current.face.tensor, dom.canvas); human.tf.browser.toPixels(current.face.tensor as unknown as TensorLike, dom.canvas);
if (await indexDb.count() === 0) { if (await indexDb.count() === 0) {
log('face database is empty: nothing to compare face with'); log('face database is empty');
document.body.style.background = 'black'; document.body.style.background = 'black';
dom.delete.style.display = 'none'; dom.delete.style.display = 'none';
return false; return false;
} }
const db = await indexDb.load(); const db = await indexDb.load();
const descriptors = db.map((rec) => rec.descriptor).filter((desc) => desc.length > 0); const descriptors = db.map((rec) => rec.descriptor);
const res = human.match.find(current.face.embedding, descriptors, matchOptions); const res = await human.match(current.face.embedding, descriptors, matchOptions);
current.record = db[res.index] || null; current.record = db[res.index] || null;
if (current.record) { if (current.record) {
log(`best match: ${current.record.name} | id: ${current.record.id} | similarity: ${Math.round(1000 * res.similarity) / 10}%`); log(`best match: ${current.record.name} | id: ${current.record.id} | similarity: ${Math.round(1000 * res.similarity) / 10}%`);
@ -262,20 +223,17 @@ async function detectFace() {
} }
async function main() { // main entry point async function main() { // main entry point
ok.faceCount.status = false; ok.faceCount = false;
ok.faceConfidence.status = false; ok.faceConfidence = false;
ok.facingCenter.status = false; ok.facingCenter = false;
ok.blinkDetected.status = false; ok.blinkDetected = false;
ok.faceSize.status = false; ok.faceSize = false;
ok.antispoofCheck.status = false; ok.antispoofCheck = false;
ok.livenessCheck.status = false; ok.livenessCheck = false;
ok.age.status = false; ok.elapsedMs = 0;
ok.gender.status = false;
ok.elapsedMs.val = 0;
dom.match.style.display = 'none'; dom.match.style.display = 'none';
dom.retry.style.display = 'none'; dom.retry.style.display = 'none';
dom.source.style.display = 'none'; dom.source.style.display = 'none';
dom.canvas.style.height = '50%';
document.body.style.background = 'black'; document.body.style.background = 'black';
await webCam(); await webCam();
await detectionLoop(); // start detection loop await detectionLoop(); // start detection loop
@ -293,21 +251,19 @@ async function main() { // main entry point
if (!allOk()) { // is all criteria met? if (!allOk()) { // is all criteria met?
log('did not find valid face'); log('did not find valid face');
return false; return false;
} } else {
return detectFace(); return detectFace();
}
} }
async function init() { async function init() {
log('human version:', human.version, '| tfjs version:', human.tf.version['tfjs-core']); log('human version:', human.version, '| tfjs version:', human.tf.version['tfjs-core']);
log('options:', JSON.stringify(options).replace(/{|}|"|\[|\]/g, '').replace(/,/g, ' ')); log('options:', JSON.stringify(options).replace(/{|}|"|\[|\]/g, '').replace(/,/g, ' '));
log('initializing webcam...'); printFPS('loading...');
await webCam(); // start webcam
log('loading human models...');
await human.load(); // preload all models
log('initializing human...');
log('face embedding model:', humanConfig.face.description.enabled ? 'faceres' : '', humanConfig.face['mobilefacenet']?.enabled ? 'mobilefacenet' : '', humanConfig.face['insightface']?.enabled ? 'insightface' : '');
log('loading face database...');
log('known face records:', await indexDb.count()); log('known face records:', await indexDb.count());
await webCam(); // start webcam
await human.load(); // preload all models
printFPS('initializing...');
dom.retry.addEventListener('click', main); dom.retry.addEventListener('click', main);
dom.save.addEventListener('click', saveRecords); dom.save.addEventListener('click', saveRecords);
dom.delete.addEventListener('click', deleteRecord); dom.delete.addEventListener('click', deleteRecord);

View File

@ -3,9 +3,10 @@ let db: IDBDatabase; // instance of indexdb
const database = 'human'; const database = 'human';
const table = 'person'; const table = 'person';
export interface FaceRecord { id: number, name: string, descriptor: number[], image: ImageData } export type FaceRecord = { id: number, name: string, descriptor: number[], image: ImageData };
const log = (...msg) => console.log('indexdb', ...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
const log = (...msg) => console.log('indexdb', ...msg);
export async function open() { export async function open() {
if (db) return true; if (db) return true;
@ -18,7 +19,7 @@ export async function open() {
db.createObjectStore(table, { keyPath: 'id', autoIncrement: true }); db.createObjectStore(table, { keyPath: 'id', autoIncrement: true });
}; };
request.onsuccess = (evt) => { // open request.onsuccess = (evt) => { // open
db = (evt.target as IDBOpenDBRequest).result; db = (evt.target as IDBOpenDBRequest).result as IDBDatabase;
log('open:', db); log('open:', db);
resolve(true); resolve(true);
}; };
@ -26,7 +27,7 @@ export async function open() {
} }
export async function load(): Promise<FaceRecord[]> { export async function load(): Promise<FaceRecord[]> {
const faceDB: FaceRecord[] = []; const faceDB: Array<FaceRecord> = [];
if (!db) await open(); // open or create if not already done if (!db) await open(); // open or create if not already done
return new Promise((resolve) => { return new Promise((resolve) => {
const cursor: IDBRequest = db.transaction([table], 'readwrite').objectStore(table).openCursor(null, 'next'); const cursor: IDBRequest = db.transaction([table], 'readwrite').objectStore(table).openCursor(null, 'next');

View File

@ -11,7 +11,7 @@
## Browser Face Recognition Demo ## Browser Face Recognition Demo
- `demo/facematch`: Demo for Browsers that uses all face description and embedding features to - `demo/facematch`: Demo for Browsers that uses all face description and embedding features to
detect, extract and identify all faces plus calculate similarity between them detect, extract and identify all faces plus calculate simmilarity between them
It highlights functionality such as: It highlights functionality such as:
@ -72,13 +72,12 @@ Non-linear performance that increases with number of worker threads due to commu
> node node-match > node node-match
<!-- eslint-skip -->
```js ```js
INFO: options: { dbFile: './faces.json', dbMax: 10000, threadPoolSize: 6, workerSrc: './node-match-worker.js', debug: false, minThreshold: 0.9, descLength: 1024 } 2021-10-13 07:53:36 INFO: options: { dbFile: './faces.json', dbMax: 10000, threadPoolSize: 6, workerSrc: './node-match-worker.js', debug: false, minThreshold: 0.9, descLength: 1024 }
DATA: created shared buffer: { maxDescriptors: 10000, totalBytes: 40960000, totalElements: 10240000 } 2021-10-13 07:53:36 DATA: created shared buffer: { maxDescriptors: 10000, totalBytes: 40960000, totalElements: 10240000 }
DATA: db loaded: { existingRecords: 0, newRecords: 5700 } 2021-10-13 07:53:36 DATA: db loaded: { existingRecords: 0, newRecords: 5700 }
INFO: starting worker thread pool: { totalWorkers: 6, alreadyActive: 0 } 2021-10-13 07:53:36 INFO: starting worker thread pool: { totalWorkers: 6, alreadyActive: 0 }
STATE: submitted: { matchJobs: 100, poolSize: 6, activeWorkers: 6 } 2021-10-13 07:53:36 STATE: submitted: { matchJobs: 100, poolSize: 6, activeWorkers: 6 }
STATE: { matchJobsFinished: 100, totalTimeMs: 1769, averageTimeMs: 17.69 } 2021-10-13 07:53:38 STATE: { matchJobsFinished: 100, totalTimeMs: 1769, averageTimeMs: 17.69 }
INFO: closing workers: { poolSize: 6, activeWorkers: 6 } 2021-10-13 07:53:38 INFO: closing workers: { poolSize: 6, activeWorkers: 6 }
``` ```

View File

@ -1,17 +1,18 @@
// @ts-nocheck
/** /**
* Human demo for browsers * Human demo for browsers
* *
* Demo for face descriptor analysis and face similarity analysis * Demo for face descriptor analysis and face simmilarity analysis
*/ */
/** @type {Human} */ /** @type {Human} */
import { Human } from '../../dist/human.esm.js'; import Human from '../../dist/human.esm.js';
const userConfig = { const userConfig = {
backend: 'humangl', backend: 'humangl',
async: true, async: true,
warmup: 'none', warmup: 'none',
cacheSensitivity: 0.01, cacheSensitivity: 0,
debug: true, debug: true,
modelBasePath: '../../models/', modelBasePath: '../../models/',
deallocate: true, deallocate: true,
@ -22,6 +23,7 @@ const userConfig = {
}, },
face: { face: {
enabled: true, enabled: true,
// detector: { rotation: false, return: true, maxDetected: 50, iouThreshold: 0.206, minConfidence: 0.122 },
detector: { return: true, rotation: true, maxDetected: 50, iouThreshold: 0.01, minConfidence: 0.2 }, detector: { return: true, rotation: true, maxDetected: 50, iouThreshold: 0.01, minConfidence: 0.2 },
mesh: { enabled: true }, mesh: { enabled: true },
iris: { enabled: false }, iris: { enabled: false },
@ -44,7 +46,8 @@ const minScore = 0.4;
function log(...msg) { function log(...msg) {
const dt = new Date(); const dt = new Date();
const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`; const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`;
console.log(ts, ...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.log(ts, ...msg);
} }
function title(msg) { function title(msg) {
@ -63,16 +66,25 @@ async function loadFaceMatchDB() {
} }
} }
async function selectFaceCanvas(face) { async function SelectFaceCanvas(face) {
// if we have face image tensor, enhance it and display it // if we have face image tensor, enhance it and display it
let embedding; let embedding;
document.getElementById('orig').style.filter = 'blur(16px)'; document.getElementById('orig').style.filter = 'blur(16px)';
if (face.tensor) { if (face.tensor) {
title('Sorting Faces by Similarity'); title('Sorting Faces by Similarity');
const enhanced = human.enhance(face);
if (enhanced) {
const c = document.getElementById('orig'); const c = document.getElementById('orig');
await human.draw.tensor(face.tensor, c); const squeeze = human.tf.squeeze(enhanced);
const normalize = human.tf.div(squeeze, 255);
await human.tf.browser.toPixels(normalize, c);
human.tf.dispose([enhanced, squeeze, normalize]);
const ctx = c.getContext('2d');
ctx.font = 'small-caps 0.4rem "Lato"';
ctx.fillStyle = 'rgba(255, 255, 255, 1)';
}
const arr = db.map((rec) => rec.embedding); const arr = db.map((rec) => rec.embedding);
const res = await human.match.find(face.embedding, arr); const res = await human.match(face.embedding, arr);
log('Match:', db[res.index].name); log('Match:', db[res.index].name);
const emotion = face.emotion[0] ? `${Math.round(100 * face.emotion[0].score)}% ${face.emotion[0].emotion}` : 'N/A'; const emotion = face.emotion[0] ? `${Math.round(100 * face.emotion[0].score)}% ${face.emotion[0].emotion}` : 'N/A';
document.getElementById('desc').innerHTML = ` document.getElementById('desc').innerHTML = `
@ -93,11 +105,11 @@ async function selectFaceCanvas(face) {
for (const canvas of canvases) { for (const canvas of canvases) {
// calculate similarity from selected face to current one in the loop // calculate similarity from selected face to current one in the loop
const current = all[canvas.tag.sample][canvas.tag.face]; const current = all[canvas.tag.sample][canvas.tag.face];
const similarity = human.match.similarity(face.embedding, current.embedding); const similarity = human.similarity(face.embedding, current.embedding);
canvas.tag.similarity = similarity; canvas.tag.similarity = similarity;
// get best match // get best match
// draw the canvas // draw the canvas
await human.draw.tensor(current.tensor, canvas); await human.tf.browser.toPixels(current.tensor, canvas);
const ctx = canvas.getContext('2d'); const ctx = canvas.getContext('2d');
ctx.font = 'small-caps 1rem "Lato"'; ctx.font = 'small-caps 1rem "Lato"';
ctx.fillStyle = 'rgba(0, 0, 0, 1)'; ctx.fillStyle = 'rgba(0, 0, 0, 1)';
@ -110,7 +122,7 @@ async function selectFaceCanvas(face) {
ctx.font = 'small-caps 1rem "Lato"'; ctx.font = 'small-caps 1rem "Lato"';
const start = human.now(); const start = human.now();
const arr = db.map((rec) => rec.embedding); const arr = db.map((rec) => rec.embedding);
const res = await human.match.find(current.embedding, arr); const res = await human.match(current.embedding, arr);
time += (human.now() - start); time += (human.now() - start);
if (res.similarity > minScore) ctx.fillText(`DB: ${(100 * res.similarity).toFixed(1)}% ${db[res.index].name}`, 4, canvas.height - 30); if (res.similarity > minScore) ctx.fillText(`DB: ${(100 * res.similarity).toFixed(1)}% ${db[res.index].name}`, 4, canvas.height - 30);
} }
@ -125,7 +137,7 @@ async function selectFaceCanvas(face) {
title('Selected Face'); title('Selected Face');
} }
async function addFaceCanvas(index, res, fileName) { async function AddFaceCanvas(index, res, fileName) {
all[index] = res.face; all[index] = res.face;
for (const i in res.face) { for (const i in res.face) {
if (!res.face[i].tensor) continue; // did not get valid results if (!res.face[i].tensor) continue; // did not get valid results
@ -144,37 +156,35 @@ async function addFaceCanvas(index, res, fileName) {
gender: ${Math.round(100 * res.face[i].genderScore)}% ${res.face[i].gender} gender: ${Math.round(100 * res.face[i].genderScore)}% ${res.face[i].gender}
emotion: ${emotion} emotion: ${emotion}
`.replace(/ /g, ' '); `.replace(/ /g, ' ');
await human.draw.tensor(res.face[i].tensor, canvas); await human.tf.browser.toPixels(res.face[i].tensor, canvas);
const ctx = canvas.getContext('2d'); const ctx = canvas.getContext('2d');
if (!ctx) return; if (!ctx) return false;
ctx.font = 'small-caps 0.8rem "Lato"'; ctx.font = 'small-caps 0.8rem "Lato"';
ctx.fillStyle = 'rgba(255, 255, 255, 1)'; ctx.fillStyle = 'rgba(255, 255, 255, 1)';
ctx.fillText(`${res.face[i].age}y ${(100 * (res.face[i].genderScore || 0)).toFixed(1)}% ${res.face[i].gender}`, 4, canvas.height - 6); ctx.fillText(`${res.face[i].age}y ${(100 * (res.face[i].genderScore || 0)).toFixed(1)}% ${res.face[i].gender}`, 4, canvas.height - 6);
const arr = db.map((rec) => rec.embedding); const arr = db.map((rec) => rec.embedding);
const result = human.match.find(res.face[i].embedding, arr); const result = human.match(res.face[i].embedding, arr);
ctx.font = 'small-caps 1rem "Lato"'; ctx.font = 'small-caps 1rem "Lato"';
if (result.similarity && res.similarity > minScore) ctx.fillText(`${(100 * result.similarity).toFixed(1)}% ${db[result.index].name}`, 4, canvas.height - 30); if (result.similarity && res.similarity > minScore) ctx.fillText(`${(100 * result.similarity).toFixed(1)}% ${db[result.index].name}`, 4, canvas.height - 30);
document.getElementById('faces').appendChild(canvas); document.getElementById('faces').appendChild(canvas);
canvas.addEventListener('click', (evt) => { canvas.addEventListener('click', (evt) => {
log('Select:', 'Image:', evt.target.tag.sample, 'Face:', evt.target.tag.face, 'Source:', evt.target.tag.source, all[evt.target.tag.sample][evt.target.tag.face]); log('Select:', 'Image:', evt.target.tag.sample, 'Face:', evt.target.tag.face, 'Source:', evt.target.tag.source, all[evt.target.tag.sample][evt.target.tag.face]);
selectFaceCanvas(all[evt.target.tag.sample][evt.target.tag.face]); SelectFaceCanvas(all[evt.target.tag.sample][evt.target.tag.face]);
}); });
} }
} }
async function addImageElement(index, image, length) { async function AddImageElement(index, image, length) {
const faces = all.reduce((prev, curr) => prev += curr.length, 0); const faces = all.reduce((prev, curr) => prev += curr.length, 0);
title(`Analyzing Input Images<br> ${Math.round(100 * index / length)}% [${index} / ${length}]<br>Found ${faces} Faces`); title(`Analyzing Input Images<br> ${Math.round(100 * index / length)}% [${index} / ${length}]<br>Found ${faces} Faces`);
return new Promise((resolve) => { return new Promise((resolve) => {
const img = new Image(128, 128); const img = new Image(128, 128);
img.onload = () => { // must wait until image is loaded img.onload = () => { // must wait until image is loaded
document.getElementById('images').appendChild(img); // and finally we can add it document.getElementById('images').appendChild(img); // and finally we can add it
human.detect(img, userConfig) human.detect(img, userConfig).then((res) => {
.then((res) => { // eslint-disable-line promise/always-return AddFaceCanvas(index, res, image); // then wait until image is analyzed
addFaceCanvas(index, res, image); // then wait until image is analyzed
resolve(true); resolve(true);
}) });
.catch(() => log('human detect error'));
}; };
img.onerror = () => { img.onerror = () => {
log('Add image error:', index + 1, image); log('Add image error:', index + 1, image);
@ -212,23 +222,18 @@ async function main() {
// could not dynamically enumerate images so using static list // could not dynamically enumerate images so using static list
if (images.length === 0) { if (images.length === 0) {
images = [ images = [
'ai-face.jpg', 'ai-upper.jpg', 'ai-body.jpg', 'solvay1927.jpg', 'ai-body.jpg', 'solvay1927.jpg', 'ai-upper.jpg',
'person-carolina.jpg', 'person-celeste.jpg', 'person-leila1.jpg', 'person-leila2.jpg', 'person-lexi.jpg', 'person-linda.jpg', 'person-nicole.jpg', 'person-tasia.jpg',
'person-tetiana.jpg', 'person-vlado1.jpg', 'person-vlado5.jpg', 'person-vlado.jpg', 'person-christina.jpg', 'person-lauren.jpg',
'group-1.jpg', 'group-2.jpg', 'group-3.jpg', 'group-4.jpg', 'group-5.jpg', 'group-6.jpg', 'group-7.jpg', 'group-1.jpg', 'group-2.jpg', 'group-3.jpg', 'group-4.jpg', 'group-5.jpg', 'group-6.jpg', 'group-7.jpg',
'person-celeste.jpg', 'person-christina.jpg', 'person-lauren.jpg', 'person-lexi.jpg', 'person-linda.jpg', 'person-nicole.jpg', 'person-tasia.jpg', 'person-tetiana.jpg', 'person-vlado.jpg', 'person-vlado1.jpg', 'person-vlado5.jpg', 'daz3d-brianna.jpg', 'daz3d-chiyo.jpg', 'daz3d-cody.jpg', 'daz3d-drew-01.jpg', 'daz3d-drew-02.jpg', 'daz3d-ella-01.jpg', 'daz3d-ella-02.jpg', 'daz3d-gillian.jpg',
'stock-group-1.jpg', 'stock-group-2.jpg', 'daz3d-hye-01.jpg', 'daz3d-hye-02.jpg', 'daz3d-kaia.jpg', 'daz3d-karen.jpg', 'daz3d-kiaria-01.jpg', 'daz3d-kiaria-02.jpg', 'daz3d-lilah-01.jpg', 'daz3d-lilah-02.jpg',
'stock-models-1.jpg', 'stock-models-2.jpg', 'stock-models-3.jpg', 'stock-models-4.jpg', 'stock-models-5.jpg', 'stock-models-6.jpg', 'stock-models-7.jpg', 'stock-models-8.jpg', 'stock-models-9.jpg', 'daz3d-lilah-03.jpg', 'daz3d-lila.jpg', 'daz3d-lindsey.jpg', 'daz3d-megah.jpg', 'daz3d-selina-01.jpg', 'daz3d-selina-02.jpg', 'daz3d-snow.jpg',
'stock-teen-1.jpg', 'stock-teen-2.jpg', 'stock-teen-3.jpg', 'stock-teen-4.jpg', 'stock-teen-5.jpg', 'stock-teen-6.jpg', 'stock-teen-7.jpg', 'stock-teen-8.jpg', 'daz3d-sunshine.jpg', 'daz3d-taia.jpg', 'daz3d-tuesday-01.jpg', 'daz3d-tuesday-02.jpg', 'daz3d-tuesday-03.jpg', 'daz3d-zoe.jpg', 'daz3d-ginnifer.jpg',
'stock-models-10.jpg', 'stock-models-11.jpg', 'stock-models-12.jpg', 'stock-models-13.jpg', 'stock-models-14.jpg', 'stock-models-15.jpg', 'stock-models-16.jpg', 'daz3d-_emotions01.jpg', 'daz3d-_emotions02.jpg', 'daz3d-_emotions03.jpg', 'daz3d-_emotions04.jpg', 'daz3d-_emotions05.jpg',
'cgi-model-1.jpg', 'cgi-model-2.jpg', 'cgi-model-3.jpg', 'cgi-model-4.jpg', 'cgi-model-5.jpg', 'cgi-model-6.jpg', 'cgi-model-7.jpg', 'cgi-model-8.jpg', 'cgi-model-9.jpg',
'cgi-model-10.jpg', 'cgi-model-11.jpg', 'cgi-model-12.jpg', 'cgi-model-13.jpg', 'cgi-model-14.jpg', 'cgi-model-15.jpg', 'cgi-model-18.jpg', 'cgi-model-19.jpg',
'cgi-model-20.jpg', 'cgi-model-21.jpg', 'cgi-model-22.jpg', 'cgi-model-23.jpg', 'cgi-model-24.jpg', 'cgi-model-25.jpg', 'cgi-model-26.jpg', 'cgi-model-27.jpg', 'cgi-model-28.jpg', 'cgi-model-29.jpg',
'cgi-model-30.jpg', 'cgi-model-31.jpg', 'cgi-model-33.jpg', 'cgi-model-34.jpg',
'cgi-multiangle-1.jpg', 'cgi-multiangle-2.jpg', 'cgi-multiangle-3.jpg', 'cgi-multiangle-4.jpg', 'cgi-multiangle-6.jpg', 'cgi-multiangle-7.jpg', 'cgi-multiangle-8.jpg', 'cgi-multiangle-9.jpg', 'cgi-multiangle-10.jpg', 'cgi-multiangle-11.jpg',
'stock-emotions-a-1.jpg', 'stock-emotions-a-2.jpg', 'stock-emotions-a-3.jpg', 'stock-emotions-a-4.jpg', 'stock-emotions-a-5.jpg', 'stock-emotions-a-6.jpg', 'stock-emotions-a-7.jpg', 'stock-emotions-a-8.jpg',
'stock-emotions-b-1.jpg', 'stock-emotions-b-2.jpg', 'stock-emotions-b-3.jpg', 'stock-emotions-b-4.jpg', 'stock-emotions-b-5.jpg', 'stock-emotions-b-6.jpg', 'stock-emotions-b-7.jpg', 'stock-emotions-b-8.jpg',
]; ];
// add prefix for gitpages // add prefix for gitpages
images = images.map((a) => `../../samples/in/${a}`); images = images.map((a) => `/human/samples/in/${a}`);
log('Adding static image list:', images); log('Adding static image list:', images);
} else { } else {
log('Discovered images:', images); log('Discovered images:', images);
@ -237,7 +242,7 @@ async function main() {
// images = ['/samples/in/person-lexi.jpg', '/samples/in/person-carolina.jpg', '/samples/in/solvay1927.jpg']; // images = ['/samples/in/person-lexi.jpg', '/samples/in/person-carolina.jpg', '/samples/in/solvay1927.jpg'];
const t0 = human.now(); const t0 = human.now();
for (let i = 0; i < images.length; i++) await addImageElement(i, images[i], images.length); for (let i = 0; i < images.length; i++) await AddImageElement(i, images[i], images.length);
const t1 = human.now(); const t1 = human.now();
// print stats // print stats
@ -251,7 +256,7 @@ async function main() {
title(''); title('');
log('Ready'); log('Ready');
human.validate(userConfig); human.validate(userConfig);
human.match.similarity([], []); human.similarity([], []);
} }
window.onload = main; window.onload = main;

View File

@ -1,8 +1,3 @@
/**
* Runs in a worker thread started by `node-match` demo app
*
*/
const threads = require('worker_threads'); const threads = require('worker_threads');
let debug = false; let debug = false;
@ -38,8 +33,7 @@ function match(descBuffer, options = { order: 2, multiplier: 20 }) {
if (best < threshold || best === 0) break; // short circuit if (best < threshold || best === 0) break; // short circuit
} }
best = (options.order === 2) ? Math.sqrt(best) : best ** (1 / options.order); best = (options.order === 2) ? Math.sqrt(best) : best ** (1 / options.order);
const similarity = Math.round(100 * Math.max(0, 100 - best) / 100.0) / 100; return { index, distance: best, similarity: Math.max(0, 100 - best) / 100.0 };
return { index, distance: best, similarity };
} }
threads.parentPort?.on('message', (msg) => { threads.parentPort?.on('message', (msg) => {
@ -53,7 +47,7 @@ threads.parentPort?.on('message', (msg) => {
if (msg instanceof SharedArrayBuffer) { // called only once to receive reference to shared array buffer if (msg instanceof SharedArrayBuffer) { // called only once to receive reference to shared array buffer
buffer = msg; buffer = msg;
view = new Float32Array(buffer); // initialize f64 view into buffer view = new Float32Array(buffer); // initialize f64 view into buffer
if (debug) threads.parentPort?.postMessage(`buffer: ${buffer.byteLength}`); if (debug) threads.parentPort?.postMessage(`buffer: ${buffer?.byteLength}`);
} }
if (typeof msg.records !== 'undefined') { // recived every time when number of records changes if (typeof msg.records !== 'undefined') { // recived every time when number of records changes
records = msg.records; records = msg.records;
@ -61,15 +55,15 @@ threads.parentPort?.on('message', (msg) => {
} }
if (typeof msg.debug !== 'undefined') { // set verbose logging if (typeof msg.debug !== 'undefined') { // set verbose logging
debug = msg.debug; debug = msg.debug;
// if (debug) threads.parentPort?.postMessage(`debug: ${debug}`); if (debug) threads.parentPort?.postMessage(`debug: ${debug}`);
} }
if (typeof msg.threshold !== 'undefined') { // set minimum similarity threshold if (typeof msg.threshold !== 'undefined') { // set minimum similarity threshold
threshold = msg.threshold; threshold = msg.threshold;
// if (debug) threads.parentPort?.postMessage(`threshold: ${threshold}`); if (debug) threads.parentPort?.postMessage(`threshold: ${threshold}`);
} }
if (typeof msg.shutdown !== 'undefined') { // got message to close worker if (typeof msg.shutdown !== 'undefined') { // got message to close worker
if (debug) threads.parentPort?.postMessage('shutting down'); if (debug) threads.parentPort?.postMessage('shutting down');
process.exit(0); // eslint-disable-line no-process-exit process.exit(0);
} }
}); });

View File

@ -1,13 +1,7 @@
/**
* Human demo app for NodeJS that generates random facial descriptors
* and uses NodeJS multi-threading to start multiple threads for face matching
* uses `node-match-worker.js` to perform actual face matching analysis
*/
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const log = require('@vladmandic/pilogger');
const threads = require('worker_threads'); const threads = require('worker_threads');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require
// global optinos // global optinos
const options = { const options = {
@ -15,7 +9,7 @@ const options = {
dbMax: 10000, // maximum number of records to hold in memory dbMax: 10000, // maximum number of records to hold in memory
threadPoolSize: 12, // number of worker threads to create in thread pool threadPoolSize: 12, // number of worker threads to create in thread pool
workerSrc: './node-match-worker.js', // code that executes in the worker thread workerSrc: './node-match-worker.js', // code that executes in the worker thread
debug: true, // verbose messages debug: false, // verbose messages
minThreshold: 0.5, // match returns first record that meets the similarity threshold, set to 0 to always scan all records minThreshold: 0.5, // match returns first record that meets the similarity threshold, set to 0 to always scan all records
descLength: 1024, // descriptor length descLength: 1024, // descriptor length
}; };
@ -160,7 +154,7 @@ async function createBuffer() {
data.buffer = new SharedArrayBuffer(4 * options.dbMax * options.descLength); // preallocate max number of records as sharedarraybuffers cannot grow data.buffer = new SharedArrayBuffer(4 * options.dbMax * options.descLength); // preallocate max number of records as sharedarraybuffers cannot grow
data.view = new Float32Array(data.buffer); // create view into buffer data.view = new Float32Array(data.buffer); // create view into buffer
data.labels.length = 0; data.labels.length = 0;
log.data('created shared buffer:', { maxDescriptors: (data.view.length || 0) / options.descLength, totalBytes: data.buffer.byteLength, totalElements: data.view.length }); log.data('created shared buffer:', { maxDescriptors: (data.view?.length || 0) / options.descLength, totalBytes: data.buffer.byteLength, totalElements: data.view?.length });
} }
async function main() { async function main() {
@ -176,7 +170,7 @@ async function main() {
data.requestID++; // increase request id data.requestID++; // increase request id
if (testOptions.fuzDescriptors) match(fuzDescriptor(descriptor)); // fuz descriptor for harder match if (testOptions.fuzDescriptors) match(fuzDescriptor(descriptor)); // fuz descriptor for harder match
else match(descriptor); else match(descriptor);
if (options.debug) log.debug('submited job', data.requestID); // we already know what we're searching for so we can compare results if (options.debug) log.info('submited job', data.requestID); // we already know what we're searching for so we can compare results
} }
log.state('submitted:', { matchJobs: testOptions.maxJobs, poolSize: data.workers.length, activeWorkers: data.workers.filter((worker) => !!worker).length }); log.state('submitted:', { matchJobs: testOptions.maxJobs, poolSize: data.workers.length, activeWorkers: data.workers.filter((worker) => !!worker).length });
} }

View File

@ -1,3 +1,4 @@
// @ts-nocheck
// based on: https://github.com/munrocket/gl-bench // based on: https://github.com/munrocket/gl-bench
const UICSS = ` const UICSS = `
@ -42,7 +43,9 @@ class GLBench {
constructor(gl, settings = {}) { constructor(gl, settings = {}) {
this.css = UICSS; this.css = UICSS;
this.svg = UISVG; this.svg = UISVG;
// eslint-disable-next-line @typescript-eslint/no-empty-function
this.paramLogger = () => {}; this.paramLogger = () => {};
// eslint-disable-next-line @typescript-eslint/no-empty-function
this.chartLogger = () => {}; this.chartLogger = () => {};
this.chartLen = 20; this.chartLen = 20;
this.chartHz = 20; this.chartHz = 20;
@ -89,6 +92,7 @@ class GLBench {
const addProfiler = (fn, self, target) => { const addProfiler = (fn, self, target) => {
const t = self.now(); const t = self.now();
// eslint-disable-next-line prefer-rest-params
fn.apply(target, arguments); fn.apply(target, arguments);
if (self.trackGPU) self.finished.push(glFinish(t, self.activeAccums.slice(0))); if (self.trackGPU) self.finished.push(glFinish(t, self.activeAccums.slice(0)));
}; };
@ -103,11 +107,13 @@ class GLBench {
if (gl[fn]) { if (gl[fn]) {
gl[fn] = addProfiler(gl[fn], this, gl); gl[fn] = addProfiler(gl[fn], this, gl);
} else { } else {
// eslint-disable-next-line no-console
console.log('bench: cannot attach to webgl function'); console.log('bench: cannot attach to webgl function');
} }
/* /*
gl.getExtension = ((fn, self) => { gl.getExtension = ((fn, self) => {
// eslint-disable-next-line prefer-rest-params
const ext = fn.apply(gl, arguments); const ext = fn.apply(gl, arguments);
if (ext) { if (ext) {
['drawElementsInstancedANGLE', 'drawBuffersWEBGL'].forEach((fn2) => { ['drawElementsInstancedANGLE', 'drawBuffersWEBGL'].forEach((fn2) => {
@ -142,6 +148,7 @@ class GLBench {
return (i, cpu, gpu, mem, fps, totalTime, frameId) => { return (i, cpu, gpu, mem, fps, totalTime, frameId) => {
nodes['gl-cpu'][i].style.strokeDasharray = (cpu * 0.27).toFixed(0) + ' 100'; nodes['gl-cpu'][i].style.strokeDasharray = (cpu * 0.27).toFixed(0) + ' 100';
nodes['gl-gpu'][i].style.strokeDasharray = (gpu * 0.27).toFixed(0) + ' 100'; nodes['gl-gpu'][i].style.strokeDasharray = (gpu * 0.27).toFixed(0) + ' 100';
// eslint-disable-next-line no-nested-ternary
nodes['gl-mem'][i].innerHTML = names[i] ? names[i] : (mem ? 'mem: ' + mem.toFixed(0) + 'mb' : ''); nodes['gl-mem'][i].innerHTML = names[i] ? names[i] : (mem ? 'mem: ' + mem.toFixed(0) + 'mb' : '');
nodes['gl-fps'][i].innerHTML = 'FPS: ' + fps.toFixed(1); nodes['gl-fps'][i].innerHTML = 'FPS: ' + fps.toFixed(1);
logger(names[i], cpu, gpu, mem, fps, totalTime, frameId); logger(names[i], cpu, gpu, mem, fps, totalTime, frameId);

View File

@ -64,7 +64,9 @@ function createNode() {
hideChildren() { hideChildren() {
if (Array.isArray(this.children)) { if (Array.isArray(this.children)) {
this.children.forEach((item) => { this.children.forEach((item) => {
// @ts-ignore
item['elem']['classList'].add('hide'); item['elem']['classList'].add('hide');
// @ts-ignore
if (item['expanded']) item.hideChildren(); if (item['expanded']) item.hideChildren();
}); });
} }
@ -72,7 +74,9 @@ function createNode() {
showChildren() { showChildren() {
if (Array.isArray(this.children)) { if (Array.isArray(this.children)) {
this.children.forEach((item) => { this.children.forEach((item) => {
// @ts-ignore
item['elem']['classList'].remove('hide'); item['elem']['classList'].remove('hide');
// @ts-ignore
if (item['expanded']) item.showChildren(); if (item['expanded']) item.showChildren();
}); });
} }

View File

@ -119,6 +119,7 @@ class Menu {
this.menu.appendChild(this.container); this.menu.appendChild(this.container);
if (typeof parent === 'object') parent.appendChild(this.menu); if (typeof parent === 'object') parent.appendChild(this.menu);
// @ts-ignore undefined
else document.getElementById(parent).appendChild(this.menu); else document.getElementById(parent).appendChild(this.menu);
} }
@ -183,6 +184,7 @@ class Menu {
this.hidden = !this.hidden; this.hidden = !this.hidden;
const all = document.getElementsByClassName('menu'); const all = document.getElementsByClassName('menu');
for (const item of all) { for (const item of all) {
// @ts-ignore
item.style.display = this.hidden ? 'none' : 'block'; item.style.display = this.hidden ? 'none' : 'block';
} }
}); });
@ -239,6 +241,7 @@ class Menu {
el.addEventListener('change', (evt) => { el.addEventListener('change', (evt) => {
if (evt.target) { if (evt.target) {
object[variable] = parseInt(evt.target['value']) === parseFloat(evt.target['value']) ? parseInt(evt.target['value']) : parseFloat(evt.target['value']); object[variable] = parseInt(evt.target['value']) === parseFloat(evt.target['value']) ? parseInt(evt.target['value']) : parseFloat(evt.target['value']);
// @ts-ignore
evt.target.setAttribute('value', evt.target['value']); evt.target.setAttribute('value', evt.target['value']);
if (callback) callback(evt.target['value']); if (callback) callback(evt.target['value']);
} }
@ -283,6 +286,7 @@ class Menu {
return el; return el;
} }
// eslint-disable-next-line class-methods-use-this
updateValue(title, val, suffix = '') { updateValue(title, val, suffix = '') {
const el = document.getElementById(`menu-val-${title}`); const el = document.getElementById(`menu-val-${title}`);
if (el) el.innerText = `${title}: ${val}${suffix}`; if (el) el.innerText = `${title}: ${val}${suffix}`;
@ -299,9 +303,11 @@ class Menu {
return el; return el;
} }
// eslint-disable-next-line class-methods-use-this
async updateChart(id, values) { async updateChart(id, values) {
if (!values || (values.length === 0)) return; if (!values || (values.length === 0)) return;
/** @type {HTMLCanvasElement} */ /** @type {HTMLCanvasElement} */
// @ts-ignore undefined
const canvas = document.getElementById(`menu-canvas-${id}`); const canvas = document.getElementById(`menu-canvas-${id}`);
if (!canvas) return; if (!canvas) return;
const ctx = canvas.getContext('2d'); const ctx = canvas.getContext('2d');

View File

@ -4,7 +4,8 @@ async function log(...msg) {
if (debug) { if (debug) {
const dt = new Date(); const dt = new Date();
const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`; const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`;
console.log(ts, 'webrtc', ...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.log(ts, 'webrtc', ...msg);
} }
} }

View File

@ -2,7 +2,6 @@
* PWA Service Worker for Human main demo * PWA Service Worker for Human main demo
*/ */
/* eslint-disable no-restricted-globals */
/// <reference lib="webworker" /> /// <reference lib="webworker" />
const skipCaching = false; const skipCaching = false;
@ -20,7 +19,8 @@ const stats = { hit: 0, miss: 0 };
const log = (...msg) => { const log = (...msg) => {
const dt = new Date(); const dt = new Date();
const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`; const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`;
console.log(ts, 'pwa', ...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.log(ts, 'pwa', ...msg);
}; };
async function updateCached(req) { async function updateCached(req) {
@ -31,7 +31,7 @@ async function updateCached(req) {
caches caches
.open(cacheName) .open(cacheName)
.then((cache) => cache.put(req, update)) .then((cache) => cache.put(req, update))
.catch((err) => log('cache update error', err)); // eslint-disable-line promise/no-nesting .catch((err) => log('cache update error', err));
} }
return true; return true;
}) })
@ -75,13 +75,14 @@ async function getCached(evt) {
} }
function cacheInit() { function cacheInit() {
// eslint-disable-next-line promise/catch-or-return
caches.open(cacheName) caches.open(cacheName)
// eslint-disable-next-line promise/no-nesting
.then((cache) => cache.addAll(cacheFiles) .then((cache) => cache.addAll(cacheFiles)
.then( // eslint-disable-line promise/no-nesting .then(
() => log('cache refresh:', cacheFiles.length, 'files'), () => log('cache refresh:', cacheFiles.length, 'files'),
(err) => log('cache error', err), (err) => log('cache error', err),
)) ));
.catch(() => log('cache error'));
} }
if (!listening) { if (!listening) {
@ -98,12 +99,14 @@ if (!listening) {
self.addEventListener('install', (evt) => { self.addEventListener('install', (evt) => {
log('install'); log('install');
// @ts-ignore scope for self is ServiceWorkerGlobalScope not Window
self.skipWaiting(); self.skipWaiting();
evt.waitUntil(cacheInit); evt.waitUntil(cacheInit);
}); });
self.addEventListener('activate', (evt) => { self.addEventListener('activate', (evt) => {
log('activate'); log('activate');
// @ts-ignore scope for self is ServiceWorkerGlobalScope not Window
evt.waitUntil(self.clients.claim()); evt.waitUntil(self.clients.claim());
}); });
@ -111,7 +114,7 @@ if (!listening) {
const uri = new URL(evt.request.url); const uri = new URL(evt.request.url);
// if (uri.pathname === '/') { log('cache skip /', evt.request); return; } // skip root access requests // if (uri.pathname === '/') { log('cache skip /', evt.request); return; } // skip root access requests
if (evt.request.cache === 'only-if-cached' && evt.request.mode !== 'same-origin') return; // required due to chrome bug if (evt.request.cache === 'only-if-cached' && evt.request.mode !== 'same-origin') return; // required due to chrome bug
if (uri.origin !== self.location.origin) return; // skip non-local requests if (uri.origin !== location.origin) return; // skip non-local requests
if (evt.request.method !== 'GET') return; // only cache get requests if (evt.request.method !== 'GET') return; // only cache get requests
if (evt.request.url.includes('/api/')) return; // don't cache api requests, failures are handled at the time of call if (evt.request.url.includes('/api/')) return; // don't cache api requests, failures are handled at the time of call
@ -126,7 +129,7 @@ if (!listening) {
log(`PWA: ${evt.type}`); log(`PWA: ${evt.type}`);
if (refreshed) return; if (refreshed) return;
refreshed = true; refreshed = true;
self.location.reload(); location.reload();
}); });
listening = true; listening = true;

View File

@ -6,9 +6,10 @@
/// <reference lib="webworker"/> /// <reference lib="webworker"/>
// load Human using IIFE script as Chome Mobile does not support Modules as Workers // load Human using IIFE script as Chome Mobile does not support Modules as Workers
self.importScripts('../dist/human.js'); // eslint-disable-line no-restricted-globals self.importScripts('../dist/human.js');
let busy = false; let busy = false;
// @ts-ignore
// eslint-disable-next-line new-cap, no-undef // eslint-disable-next-line new-cap, no-undef
const human = new Human.default(); const human = new Human.default();

View File

@ -35,7 +35,7 @@
.video { display: none; } .video { display: none; }
.canvas { margin: 0 auto; } .canvas { margin: 0 auto; }
.bench { position: absolute; right: 0; bottom: 0; } .bench { position: absolute; right: 0; bottom: 0; }
.compare-image { width: 200px; position: absolute; top: 150px; left: 30px; box-shadow: 0 0 2px 2px black; background: black; display: none; } .compare-image { width: 256px; position: absolute; top: 150px; left: 30px; box-shadow: 0 0 2px 2px black; background: black; display: none; }
.loader { width: 300px; height: 300px; border: 3px solid transparent; border-radius: 50%; border-top: 4px solid #f15e41; animation: spin 4s linear infinite; position: absolute; bottom: 15%; left: 50%; margin-left: -150px; z-index: 15; } .loader { width: 300px; height: 300px; border: 3px solid transparent; border-radius: 50%; border-top: 4px solid #f15e41; animation: spin 4s linear infinite; position: absolute; bottom: 15%; left: 50%; margin-left: -150px; z-index: 15; }
.loader::before, .loader::after { content: ""; position: absolute; top: 6px; bottom: 6px; left: 6px; right: 6px; border-radius: 50%; border: 4px solid transparent; } .loader::before, .loader::after { content: ""; position: absolute; top: 6px; bottom: 6px; left: 6px; right: 6px; border-radius: 50%; border: 4px solid transparent; }
.loader::before { border-top-color: #bad375; animation: 3s spin linear infinite; } .loader::before { border-top-color: #bad375; animation: 3s spin linear infinite; }
@ -89,9 +89,9 @@
<body> <body>
<div id="play" class="play icon-play"></div> <div id="play" class="play icon-play"></div>
<div id="background"> <div id="background">
<div class="wave one"></div> <div class='wave one'></div>
<div class="wave two"></div> <div class='wave two'></div>
<div class="wave three"></div> <div class='wave three'></div>
</div> </div>
<div id="loader" class="loader"></div> <div id="loader" class="loader"></div>
<div id="status" class="status"></div> <div id="status" class="status"></div>
@ -107,9 +107,13 @@
<video id="video" playsinline class="video"></video> <video id="video" playsinline class="video"></video>
</div> </div>
<div id="compare-container" class="compare-image"> <div id="compare-container" class="compare-image">
<canvas id="compare-canvas" width="200" height="200"></canvas> <canvas id="compare-canvas" width="256" height="256"></canvas>
<div id="similarity"></div> <div id="similarity"></div>
</div> </div>
<div id="segmentation-container" class="compare-image">
<canvas id="segmentation-mask" width="256" height="256" style="width: 256px; height: 256px;"></canvas>
<canvas id="segmentation-canvas" width="256" height="256" style="width: 256px; height: 256px;"></canvas>
</div>
<div id="samples-container" class="samples-container"></div> <div id="samples-container" class="samples-container"></div>
<div id="hint" class="hint"></div> <div id="hint" class="hint"></div>
<div id="log" class="log"></div> <div id="log" class="log"></div>

View File

@ -18,12 +18,11 @@
* ui={}: contains all variables exposed in the UI * ui={}: contains all variables exposed in the UI
*/ */
// WARNING!!! // test url <https://human.local/?worker=false&async=false&bench=false&draw=true&warmup=full&backend=humangl>
// This demo is written using older code style and a lot of manual setup
// Newer versions of Human have richer functionality allowing for much cleaner & easier usage
// It is recommended to use other demos such as `demo/typescript` for usage examples
import { Human } from '../dist/human.esm.js'; // equivalent of @vladmandic/human // @ts-nocheck // typescript checks disabled as this is pure javascript
import Human from '../dist/human.esm.js'; // equivalent of @vladmandic/human
import Menu from './helpers/menu.js'; import Menu from './helpers/menu.js';
import GLBench from './helpers/gl-bench.js'; import GLBench from './helpers/gl-bench.js';
import webRTC from './helpers/webrtc.js'; import webRTC from './helpers/webrtc.js';
@ -37,7 +36,7 @@ let userConfig = {
// hand: { enabled: false }, // hand: { enabled: false },
/* /*
warmup: 'none', warmup: 'none',
backend: 'webgl', backend: 'humangl',
debug: true, debug: true,
wasmPath: 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@3.9.0/dist/', wasmPath: 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@3.9.0/dist/',
async: false, async: false,
@ -57,12 +56,6 @@ let userConfig = {
// body: { enabled: true, modelPath: 'movenet-multipose.json' }, // body: { enabled: true, modelPath: 'movenet-multipose.json' },
segmentation: { enabled: false }, segmentation: { enabled: false },
*/ */
/*
face: { iris: { enabled: false }, emotion: { enabled: false } },
hand: { enabled: false },
body: { enabled: false },
gesture: { enabled: false },
*/
}; };
const drawOptions = { const drawOptions = {
@ -86,7 +79,7 @@ const ui = {
facing: true, // camera facing front or back facing: true, // camera facing front or back
baseBackground: 'rgba(50, 50, 50, 1)', // 'grey' baseBackground: 'rgba(50, 50, 50, 1)', // 'grey'
columns: 2, // when processing sample images create this many columns columns: 2, // when processing sample images create this many columns
useWorker: false, // use web workers for processing useWorker: true, // use web workers for processing
worker: 'index-worker.js', worker: 'index-worker.js',
maxFPSframes: 10, // keep fps history for how many frames maxFPSframes: 10, // keep fps history for how many frames
modelsPreload: false, // preload human models on startup modelsPreload: false, // preload human models on startup
@ -114,6 +107,7 @@ const ui = {
results: false, // show results tree results: false, // show results tree
lastFrame: 0, // time of last frame processing lastFrame: 0, // time of last frame processing
viewportSet: false, // internal, has custom viewport been set viewportSet: false, // internal, has custom viewport been set
background: null, // holds instance of segmentation background image
transferCanvas: null, // canvas used to transfer data to and from worker transferCanvas: null, // canvas used to transfer data to and from worker
// webrtc // webrtc
@ -153,7 +147,7 @@ let bench;
let lastDetectedResult = {}; let lastDetectedResult = {};
// helper function: async pause // helper function: async pause
// eslint-disable-next-line no-unused-vars, @typescript-eslint/no-unused-vars // eslint-disable-next-line @typescript-eslint/no-unused-vars, no-unused-vars
const delay = (ms) => new Promise((resolve) => { setTimeout(resolve, ms); }); const delay = (ms) => new Promise((resolve) => { setTimeout(resolve, ms); });
// helper function: translates json to human readable string // helper function: translates json to human readable string
@ -171,7 +165,8 @@ function str(...msg) {
function log(...msg) { function log(...msg) {
const dt = new Date(); const dt = new Date();
const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`; const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`;
if (ui.console) console.log(ts, ...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
if (ui.console) console.log(ts, ...msg);
} }
let prevStatus = ''; let prevStatus = '';
@ -185,16 +180,17 @@ function status(msg) {
prevStatus = msg; prevStatus = msg;
} else { } else {
const video = document.getElementById('video'); const video = document.getElementById('video');
const playing = isLive(video) && !video.paused; // eslint-disable-line no-use-before-define const playing = (video.srcObject !== null) && !video.paused;
document.getElementById('play').style.display = playing ? 'none' : 'block'; document.getElementById('play').style.display = playing ? 'none' : 'block';
document.getElementById('loader').style.display = 'none'; document.getElementById('loader').style.display = 'none';
div.innerText = ''; div.innerText = '';
} }
} }
async function videoPlay(videoElement = document.getElementById('video')) { async function videoPlay() {
document.getElementById('btnStartText').innerHTML = 'pause video'; document.getElementById('btnStartText').innerHTML = 'pause video';
await videoElement.play(); await document.getElementById('video').play();
// status();
} }
async function videoPause() { async function videoPause() {
@ -207,28 +203,29 @@ async function videoPause() {
const compare = { enabled: false, original: null }; const compare = { enabled: false, original: null };
async function calcSimmilarity(result) { async function calcSimmilarity(result) {
document.getElementById('compare-container').onclick = () => {
log('resetting face compare baseline:');
compare.original = null;
};
document.getElementById('compare-container').style.display = compare.enabled ? 'block' : 'none'; document.getElementById('compare-container').style.display = compare.enabled ? 'block' : 'none';
if (!compare.enabled) { if (!compare.enabled) return;
compare.original = null;
return;
}
if (!result || !result.face || !result.face[0] || !result.face[0].embedding) return; if (!result || !result.face || !result.face[0] || !result.face[0].embedding) return;
if (!(result.face.length > 0) || (result.face[0].embedding.length <= 64)) return; if (!(result.face.length > 0) || (result.face[0].embedding.length <= 64)) return;
if (!compare.original) { if (!compare.original) {
compare.original = result; compare.original = result;
log('setting face compare baseline:', result.face[0]); log('setting face compare baseline:', result.face[0]);
if (result.face[0].tensor) { if (result.face[0].tensor) {
const enhanced = human.enhance(result.face[0]);
if (enhanced) {
const c = document.getElementById('orig'); const c = document.getElementById('orig');
human.draw.tensor(result.face[0].tensor, c); const squeeze = human.tf.squeeze(enhanced);
const norm = human.tf.div(squeeze, 255);
human.tf.browser.toPixels(norm, c);
human.tf.dispose(enhanced);
human.tf.dispose(squeeze);
human.tf.dispose(norm);
}
} else { } else {
document.getElementById('compare-canvas').getContext('2d').drawImage(compare.original.canvas, 0, 0, 200, 200); document.getElementById('compare-canvas').getContext('2d').drawImage(compare.original.canvas, 0, 0, 200, 200);
} }
} }
const similarity = human.match.similarity(compare.original.face[0].embedding, result.face[0].embedding); const similarity = human.similarity(compare.original.face[0].embedding, result.face[0].embedding);
document.getElementById('similarity').innerText = `similarity: ${Math.trunc(1000 * similarity) / 10}%`; document.getElementById('similarity').innerText = `similarity: ${Math.trunc(1000 * similarity) / 10}%`;
} }
@ -256,7 +253,21 @@ async function drawResults(input) {
// draw fps chart // draw fps chart
await menu.process.updateChart('FPS', ui.detectFPS); await menu.process.updateChart('FPS', ui.detectFPS);
if (!result.canvas || ui.buffered) { // refresh with input if using buffered output or if missing canvas document.getElementById('segmentation-container').style.display = userConfig.segmentation.enabled ? 'block' : 'none';
if (userConfig.segmentation.enabled && ui.buffered) { // refresh segmentation if using buffered output
const seg = await human.segmentation(input, ui.background);
if (seg.alpha) {
const canvasSegMask = document.getElementById('segmentation-mask');
const ctxSegMask = canvasSegMask.getContext('2d');
ctxSegMask.clearRect(0, 0, canvasSegMask.width, canvasSegMask.height); // need to clear as seg.alpha is alpha based canvas so it adds
ctxSegMask.drawImage(seg.alpha, 0, 0, seg.alpha.width, seg.alpha.height, 0, 0, canvasSegMask.width, canvasSegMask.height);
const canvasSegCanvas = document.getElementById('segmentation-canvas');
const ctxSegCanvas = canvasSegCanvas.getContext('2d');
ctxSegCanvas.clearRect(0, 0, canvasSegCanvas.width, canvasSegCanvas.height); // need to clear as seg.alpha is alpha based canvas so it adds
ctxSegCanvas.drawImage(seg.canvas, 0, 0, seg.alpha.width, seg.alpha.height, 0, 0, canvasSegCanvas.width, canvasSegCanvas.height);
}
// result.canvas = seg.alpha;
} else if (!result.canvas || ui.buffered) { // refresh with input if using buffered output or if missing canvas
const image = await human.image(input, false); const image = await human.image(input, false);
result.canvas = image.canvas; result.canvas = image.canvas;
human.tf.dispose(image.tensor); human.tf.dispose(image.tensor);
@ -325,11 +336,13 @@ async function drawResults(input) {
videoPause(); videoPause();
ui.drawThread = null; ui.drawThread = null;
} }
} else if (ui.drawThread) { } else {
if (ui.drawThread) {
log('stopping buffered refresh'); log('stopping buffered refresh');
cancelAnimationFrame(ui.drawThread); cancelAnimationFrame(ui.drawThread);
ui.drawThread = null; ui.drawThread = null;
} }
}
} }
// setup webcam // setup webcam
@ -419,7 +432,8 @@ async function setupCamera() {
ui.menuWidth.input.setAttribute('value', video.videoWidth); ui.menuWidth.input.setAttribute('value', video.videoWidth);
ui.menuHeight.input.setAttribute('value', video.videoHeight); ui.menuHeight.input.setAttribute('value', video.videoHeight);
if (live || ui.autoPlay) await videoPlay(); if (live || ui.autoPlay) await videoPlay();
if ((live || ui.autoPlay) && !ui.detectThread) runHumanDetect(video, canvas); // eslint-disable-line no-use-before-define // eslint-disable-next-line no-use-before-define
if ((live || ui.autoPlay) && !ui.detectThread) runHumanDetect(video, canvas);
return 'camera stream ready'; return 'camera stream ready';
} }
@ -473,7 +487,8 @@ function webWorker(input, image, canvas, timestamp) {
ui.framesDetect++; ui.framesDetect++;
if (!ui.drawThread) drawResults(input); if (!ui.drawThread) drawResults(input);
if (isLive(input)) { if (isLive(input)) {
ui.detectThread = requestAnimationFrame((now) => runHumanDetect(input, canvas, now)); // eslint-disable-line no-use-before-define // eslint-disable-next-line no-use-before-define
ui.detectThread = requestAnimationFrame((now) => runHumanDetect(input, canvas, now));
} }
}); });
} }
@ -510,9 +525,19 @@ function runHumanDetect(input, canvas, timestamp) {
// perform detection in worker // perform detection in worker
webWorker(input, data, canvas, timestamp); webWorker(input, data, canvas, timestamp);
} else { } else {
human.detect(input, userConfig) human.detect(input, userConfig).then((result) => {
.then((result) => {
status(); status();
/*
setTimeout(async () => { // simulate gl context lost 2sec after initial detection
const ext = human.gl && human.gl.gl ? human.gl.gl.getExtension('WEBGL_lose_context') : {};
if (ext && ext.loseContext) {
log('simulate context lost:', human.env.webgl, human.gl, ext);
human.gl.gl.getExtension('WEBGL_lose_context').loseContext();
await videoPause();
status('Exception: WebGL');
}
}, 2000);
*/
if (result.performance && result.performance.total) ui.detectFPS.push(1000 / result.performance.total); if (result.performance && result.performance.total) ui.detectFPS.push(1000 / result.performance.total);
if (ui.detectFPS.length > ui.maxFPSframes) ui.detectFPS.shift(); if (ui.detectFPS.length > ui.maxFPSframes) ui.detectFPS.shift();
if (ui.bench) { if (ui.bench) {
@ -529,9 +554,7 @@ function runHumanDetect(input, canvas, timestamp) {
ui.framesDetect++; ui.framesDetect++;
ui.detectThread = requestAnimationFrame((now) => runHumanDetect(input, canvas, now)); ui.detectThread = requestAnimationFrame((now) => runHumanDetect(input, canvas, now));
} }
return result; });
})
.catch(() => log('human detect error'));
} }
} }
@ -578,7 +601,8 @@ async function processImage(input, title) {
// copy to clipboard on click // copy to clipboard on click
if (typeof ClipboardItem !== 'undefined' && navigator.clipboard) { if (typeof ClipboardItem !== 'undefined' && navigator.clipboard) {
evt.target.toBlob((blob) => { evt.target.toBlob((blob) => {
const item = new ClipboardItem({ 'image/png': blob }); // eslint-disable-line no-undef // eslint-disable-next-line no-undef
const item = new ClipboardItem({ 'image/png': blob });
navigator.clipboard.write([item]); navigator.clipboard.write([item]);
log('copied image to clipboard'); log('copied image to clipboard');
}); });
@ -606,17 +630,20 @@ async function processImage(input, title) {
async function processVideo(input, title) { async function processVideo(input, title) {
status(`processing video: ${title}`); status(`processing video: ${title}`);
const video = document.getElementById('video'); const video = document.createElement('video');
const canvas = document.getElementById('canvas'); const canvas = document.getElementById('canvas');
video.id = 'video-file';
video.controls = true;
video.loop = true;
// video.onerror = async () => status(`video loading error: ${video.error.message}`);
video.addEventListener('error', () => status(`video loading error: ${video.error.message}`)); video.addEventListener('error', () => status(`video loading error: ${video.error.message}`));
video.addEventListener('canplay', async () => { video.addEventListener('canplay', async () => {
for (const m of Object.values(menu)) m.hide(); for (const m of Object.values(menu)) m.hide();
document.getElementById('samples-container').style.display = 'none'; document.getElementById('samples-container').style.display = 'none';
canvas.style.display = 'block'; canvas.style.display = 'block';
await videoPlay(); await videoPlay();
runHumanDetect(video, canvas); if (!ui.detectThread) runHumanDetect(video, canvas);
}); });
video.srcObject = null;
video.src = input; video.src = input;
} }
@ -627,8 +654,9 @@ async function detectVideo() {
const canvas = document.getElementById('canvas'); const canvas = document.getElementById('canvas');
canvas.style.display = 'block'; canvas.style.display = 'block';
cancelAnimationFrame(ui.detectThread); cancelAnimationFrame(ui.detectThread);
if (isLive(video) && !video.paused) { if ((video.srcObject !== null) && !video.paused) {
await videoPause(); await videoPause();
// if (ui.drawThread) cancelAnimationFrame(ui.drawThread);
} else { } else {
const cameraError = await setupCamera(); const cameraError = await setupCamera();
if (!cameraError) { if (!cameraError) {
@ -704,6 +732,7 @@ function setupMenu() {
menu.image.addBool('technicolor', userConfig.filter, 'technicolor', (val) => userConfig.filter.technicolor = val); menu.image.addBool('technicolor', userConfig.filter, 'technicolor', (val) => userConfig.filter.technicolor = val);
menu.image.addBool('polaroid', userConfig.filter, 'polaroid', (val) => userConfig.filter.polaroid = val); menu.image.addBool('polaroid', userConfig.filter, 'polaroid', (val) => userConfig.filter.polaroid = val);
menu.image.addHTML('<input type="file" id="file-input" class="input-file"></input> &nbsp input'); menu.image.addHTML('<input type="file" id="file-input" class="input-file"></input> &nbsp input');
menu.image.addHTML('<input type="file" id="file-background" class="input-file"></input> &nbsp background');
menu.process = new Menu(document.body, '', { top, left: x[2] }); menu.process = new Menu(document.body, '', { top, left: x[2] });
menu.process.addList('backend', ['cpu', 'webgl', 'wasm', 'humangl'], userConfig.backend, (val) => userConfig.backend = val); menu.process.addList('backend', ['cpu', 'webgl', 'wasm', 'humangl'], userConfig.backend, (val) => userConfig.backend = val);
@ -751,6 +780,8 @@ function setupMenu() {
menu.models.addHTML('<hr style="border-style: inset; border-color: dimgray">'); menu.models.addHTML('<hr style="border-style: inset; border-color: dimgray">');
menu.models.addBool('gestures', userConfig.gesture, 'enabled', (val) => userConfig.gesture.enabled = val); menu.models.addBool('gestures', userConfig.gesture, 'enabled', (val) => userConfig.gesture.enabled = val);
menu.models.addHTML('<hr style="border-style: inset; border-color: dimgray">'); menu.models.addHTML('<hr style="border-style: inset; border-color: dimgray">');
menu.models.addBool('body segmentation', userConfig.segmentation, 'enabled', (val) => userConfig.segmentation.enabled = val);
menu.models.addHTML('<hr style="border-style: inset; border-color: dimgray">');
menu.models.addBool('object detection', userConfig.object, 'enabled', (val) => userConfig.object.enabled = val); menu.models.addBool('object detection', userConfig.object, 'enabled', (val) => userConfig.object.enabled = val);
menu.models.addHTML('<hr style="border-style: inset; border-color: dimgray">'); menu.models.addHTML('<hr style="border-style: inset; border-color: dimgray">');
menu.models.addBool('face compare', compare, 'enabled', (val) => { menu.models.addBool('face compare', compare, 'enabled', (val) => {
@ -770,7 +801,6 @@ function setupMenu() {
async function resize() { async function resize() {
window.onresize = null; window.onresize = null;
log('resize');
// best setting for mobile, ignored for desktop // best setting for mobile, ignored for desktop
// can set dynamic value such as Math.min(1, Math.round(100 * window.innerWidth / 960) / 100); // can set dynamic value such as Math.min(1, Math.round(100 * window.innerWidth / 960) / 100);
const viewportScale = 0.7; const viewportScale = 0.7;
@ -819,12 +849,42 @@ async function processDataURL(f, action) {
if (e.target.result.startsWith('data:video')) await processVideo(e.target.result, f.name); if (e.target.result.startsWith('data:video')) await processVideo(e.target.result, f.name);
document.getElementById('canvas').style.display = 'none'; document.getElementById('canvas').style.display = 'none';
} }
if (action === 'background') {
const image = new Image();
image.onerror = async () => status('image loading error');
image.onload = async () => {
ui.background = image;
if (document.getElementById('canvas').style.display === 'block') { // replace canvas used for video
const canvas = document.getElementById('canvas');
const ctx = canvas.getContext('2d');
const seg = await human.segmentation(canvas, ui.background, userConfig);
if (seg.canvas) ctx.drawImage(seg.canvas, 0, 0);
} else {
const canvases = document.getElementById('samples-container').children; // replace loaded images
for (const canvas of canvases) {
const ctx = canvas.getContext('2d');
const seg = await human.segmentation(canvas, ui.background, userConfig);
if (seg.canvas) ctx.drawImage(seg.canvas, 0, 0);
}
}
};
image.src = e.target.result;
}
resolve(true); resolve(true);
}; };
reader.readAsDataURL(f); reader.readAsDataURL(f);
}); });
} }
async function runSegmentation() {
document.getElementById('file-background').onchange = async (evt) => {
userConfig.segmentation.enabled = true;
evt.preventDefault();
if (evt.target.files.length < 2) ui.columns = 1;
for (const f of evt.target.files) await processDataURL(f, 'background');
};
}
async function dragAndDrop() { async function dragAndDrop() {
document.body.addEventListener('dragenter', (evt) => evt.preventDefault()); document.body.addEventListener('dragenter', (evt) => evt.preventDefault());
document.body.addEventListener('dragleave', (evt) => evt.preventDefault()); document.body.addEventListener('dragleave', (evt) => evt.preventDefault());
@ -862,10 +922,10 @@ async function pwaRegister() {
const regs = await navigator.serviceWorker.getRegistrations(); const regs = await navigator.serviceWorker.getRegistrations();
for (const reg of regs) { for (const reg of regs) {
log('pwa found:', reg.scope); log('pwa found:', reg.scope);
if (reg.scope.startsWith(window.location.origin)) found = reg; if (reg.scope.startsWith(location.origin)) found = reg;
} }
if (!found) { if (!found) {
const reg = await navigator.serviceWorker.register(pwa.scriptFile, { scope: window.location.pathname }); const reg = await navigator.serviceWorker.register(pwa.scriptFile, { scope: location.pathname });
found = reg; found = reg;
log('pwa registered:', reg.scope); log('pwa registered:', reg.scope);
} }
@ -897,7 +957,8 @@ async function main() {
if (ui.detectThread) cancelAnimationFrame(ui.detectThread); if (ui.detectThread) cancelAnimationFrame(ui.detectThread);
if (ui.drawThread) cancelAnimationFrame(ui.drawThread); if (ui.drawThread) cancelAnimationFrame(ui.drawThread);
const msg = evt.reason.message || evt.reason || evt; const msg = evt.reason.message || evt.reason || evt;
console.error(msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.error(msg);
document.getElementById('log').innerHTML = msg; document.getElementById('log').innerHTML = msg;
status(`exception: ${msg}`); status(`exception: ${msg}`);
evt.preventDefault(); evt.preventDefault();
@ -920,7 +981,7 @@ async function main() {
await pwaRegister(); await pwaRegister();
// parse url search params // parse url search params
const params = new URLSearchParams(window.location.search); const params = new URLSearchParams(location.search);
log('url options:', params.toString()); log('url options:', params.toString());
if (params.has('worker')) { if (params.has('worker')) {
ui.useWorker = JSON.parse(params.get('worker')); ui.useWorker = JSON.parse(params.get('worker'));
@ -963,8 +1024,10 @@ async function main() {
// we've merged human defaults with user config and now lets store it back so it can be accessed by methods such as menu // we've merged human defaults with user config and now lets store it back so it can be accessed by methods such as menu
userConfig = human.config; userConfig = human.config;
if (typeof tf !== 'undefined') { if (typeof tf !== 'undefined') {
log('TensorFlow external version:', tf.version); // eslint-disable-line no-undef // eslint-disable-next-line no-undef
human.tf = tf; // eslint-disable-line no-undef log('TensorFlow external version:', tf.version);
// eslint-disable-next-line no-undef
human.tf = tf; // use externally loaded version of tfjs
} }
log('tfjs version:', human.tf.version.tfjs); log('tfjs version:', human.tf.version.tfjs);
@ -977,7 +1040,8 @@ async function main() {
if (ui.modelsPreload && !ui.useWorker) { if (ui.modelsPreload && !ui.useWorker) {
status('loading'); status('loading');
await human.load(userConfig); // this is not required, just pre-loads all models await human.load(userConfig); // this is not required, just pre-loads all models
log('demo loaded models:', human.models.loaded()); const loaded = Object.keys(human.models).filter((a) => human.models[a]);
log('demo loaded models:', loaded);
} else { } else {
await human.init(); await human.init();
} }
@ -999,6 +1063,9 @@ async function main() {
// init drag & drop // init drag & drop
await dragAndDrop(); await dragAndDrop();
// init segmentation
await runSegmentation();
if (params.has('image')) { if (params.has('image')) {
try { try {
const image = JSON.parse(params.get('image')); const image = JSON.parse(params.get('image'));
@ -1019,7 +1086,7 @@ async function main() {
} }
if (human.config.debug) log('environment:', human.env); if (human.config.debug) log('environment:', human.env);
if (human.config.backend === 'webgl' && human.config.debug) log('backend:', human.gl); if (human.config.backend === 'humangl' && human.config.debug) log('backend:', human.gl);
} }
window.onload = main; window.onload = main;

View File

@ -16,11 +16,10 @@
node demo/nodejs/node-multiprocess.js node demo/nodejs/node-multiprocess.js
``` ```
<!-- eslint-skip -->
```json ```json
2021-06-01 08:54:19 INFO: @vladmandic/human version 2.0.0 2021-06-01 08:54:19 INFO: @vladmandic/human version 2.0.0
2021-06-01 08:54:19 INFO: User: vlado Platform: linux Arch: x64 Node: v16.0.0 2021-06-01 08:54:19 INFO: User: vlado Platform: linux Arch: x64 Node: v16.0.0
2021-06-01 08:54:19 INFO: Human multi-process test 2021-06-01 08:54:19 INFO: FaceAPI multi-process test
2021-06-01 08:54:19 STATE: Enumerated images: ./assets 15 2021-06-01 08:54:19 STATE: Enumerated images: ./assets 15
2021-06-01 08:54:19 STATE: Main: started worker: 130362 2021-06-01 08:54:19 STATE: Main: started worker: 130362
2021-06-01 08:54:19 STATE: Main: started worker: 130363 2021-06-01 08:54:19 STATE: Main: started worker: 130363

View File

@ -9,10 +9,10 @@
<meta name="description" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>"> <meta name="description" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="msapplication-tooltip" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>"> <meta name="msapplication-tooltip" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="theme-color" content="#000000"> <meta name="theme-color" content="#000000">
<link rel="manifest" href="../../manifest.webmanifest"> <link rel="manifest" href="../manifest.webmanifest">
<link rel="shortcut icon" href="../../favicon.ico" type="image/x-icon"> <link rel="shortcut icon" href="../../favicon.ico" type="image/x-icon">
<link rel="apple-touch-icon" href="../../assets/icon.png"> <link rel="apple-touch-icon" href="../../assets/icon.png">
<script src="../multithread/index.js" type="module"></script> <script src="./index.js" type="module"></script>
<style> <style>
@font-face { font-family: 'Lato'; font-display: swap; font-style: normal; font-weight: 100; src: local('Lato'), url('../../assets/lato-light.woff2') } @font-face { font-family: 'Lato'; font-display: swap; font-style: normal; font-weight: 100; src: local('Lato'), url('../../assets/lato-light.woff2') }
html { font-family: 'Lato', 'Segoe UI'; font-size: 16px; font-variant: small-caps; } html { font-family: 'Lato', 'Segoe UI'; font-size: 16px; font-variant: small-caps; }

View File

@ -5,15 +5,15 @@
* *
*/ */
import { Human } from '../../dist/human.esm.js'; // equivalent of @vladmandic/human import Human from '../../dist/human.esm.js'; // equivalent of @vladmandic/human
import GLBench from '../helpers/gl-bench.js'; import GLBench from '../helpers/gl-bench.js';
const workerJS = '../multithread/worker.js'; const workerJS = './worker.js';
const config = { const config = {
main: { // processes input and runs gesture analysis main: { // processes input and runs gesture analysis
warmup: 'none', warmup: 'none',
backend: 'webgl', backend: 'humangl',
modelBasePath: '../../models/', modelBasePath: '../../models/',
async: false, async: false,
filter: { enabled: true }, filter: { enabled: true },
@ -26,7 +26,7 @@ const config = {
}, },
face: { // runs all face models face: { // runs all face models
warmup: 'none', warmup: 'none',
backend: 'webgl', backend: 'humangl',
modelBasePath: '../../models/', modelBasePath: '../../models/',
async: false, async: false,
filter: { enabled: false }, filter: { enabled: false },
@ -39,7 +39,7 @@ const config = {
}, },
body: { // runs body model body: { // runs body model
warmup: 'none', warmup: 'none',
backend: 'webgl', backend: 'humangl',
modelBasePath: '../../models/', modelBasePath: '../../models/',
async: false, async: false,
filter: { enabled: false }, filter: { enabled: false },
@ -52,7 +52,7 @@ const config = {
}, },
hand: { // runs hands model hand: { // runs hands model
warmup: 'none', warmup: 'none',
backend: 'webgl', backend: 'humangl',
modelBasePath: '../../models/', modelBasePath: '../../models/',
async: false, async: false,
filter: { enabled: false }, filter: { enabled: false },
@ -65,7 +65,7 @@ const config = {
}, },
object: { // runs object model object: { // runs object model
warmup: 'none', warmup: 'none',
backend: 'webgl', backend: 'humangl',
modelBasePath: '../../models/', modelBasePath: '../../models/',
async: false, async: false,
filter: { enabled: false }, filter: { enabled: false },
@ -130,7 +130,8 @@ const result = { // initialize empty result object which will be partially fille
function log(...msg) { function log(...msg) {
const dt = new Date(); const dt = new Date();
const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`; const ts = `${dt.getHours().toString().padStart(2, '0')}:${dt.getMinutes().toString().padStart(2, '0')}:${dt.getSeconds().toString().padStart(2, '0')}.${dt.getMilliseconds().toString().padStart(3, '0')}`;
console.log(ts, ...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.log(ts, ...msg);
} }
async function drawResults() { async function drawResults() {
@ -202,9 +203,7 @@ async function setupCamera() {
}, },
}; };
// enumerate devices for diag purposes // enumerate devices for diag purposes
navigator.mediaDevices.enumerateDevices() navigator.mediaDevices.enumerateDevices().then((devices) => log('enumerated devices:', devices));
.then((devices) => log('enumerated devices:', devices))
.catch(() => log('mediaDevices error'));
log('camera constraints', constraints); log('camera constraints', constraints);
try { try {
stream = await navigator.mediaDevices.getUserMedia(constraints); stream = await navigator.mediaDevices.getUserMedia(constraints);
@ -231,7 +230,7 @@ async function setupCamera() {
}; };
}); });
// attach input to video element // attach input to video element
if (stream && video) video.srcObject = stream; if (stream && video) video['srcObject'] = stream;
return promise; return promise;
} }

View File

@ -6,10 +6,11 @@
*/ */
const fs = require('fs'); const fs = require('fs');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require const log = require('@vladmandic/pilogger');
// workers actual import tfjs and human modules // workers actual import tfjs and faceapi modules
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require // eslint-disable-next-line no-unused-vars, @typescript-eslint/no-unused-vars
const tf = require('@tensorflow/tfjs-node');
const Human = require('../../dist/human.node.js').default; // or const Human = require('../dist/human.node-gpu.js').default; const Human = require('../../dist/human.node.js').default; // or const Human = require('../dist/human.node-gpu.js').default;
let human = null; let human = null;
@ -35,7 +36,7 @@ const myConfig = {
object: { enabled: true }, object: { enabled: true },
}; };
// read image from a file and create tensor to be used by human // read image from a file and create tensor to be used by faceapi
// this way we don't need any monkey patches // this way we don't need any monkey patches
// you can add any pre-proocessing here such as resizing, etc. // you can add any pre-proocessing here such as resizing, etc.
async function image(img) { async function image(img) {
@ -44,7 +45,7 @@ async function image(img) {
return tensor; return tensor;
} }
// actual human detection // actual faceapi detection
async function detect(img) { async function detect(img) {
const tensor = await image(img); const tensor = await image(img);
const result = await human.detect(tensor); const result = await human.detect(tensor);
@ -63,9 +64,11 @@ async function main() {
// on worker start first initialize message handler so we don't miss any messages // on worker start first initialize message handler so we don't miss any messages
process.on('message', (msg) => { process.on('message', (msg) => {
// if main told worker to exit // @ts-ignore
if (msg.exit && process.exit) process.exit(); // eslint-disable-line no-process-exit if (msg.exit && process.exit) process.exit(); // if main told worker to exit
// @ts-ignore
if (msg.test && process.send) process.send({ test: true }); if (msg.test && process.send) process.send({ test: true });
// @ts-ignore
if (msg.image) detect(msg.image); // if main told worker to process image if (msg.image) detect(msg.image); // if main told worker to process image
log.data('Worker received message:', process.pid, msg); // generic log log.data('Worker received message:', process.pid, msg); // generic log
}); });

View File

@ -8,12 +8,13 @@
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const childProcess = require('child_process'); // eslint-disable-line camelcase // eslint-disable-next-line import/no-extraneous-dependencies, node/no-unpublished-require
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require const log = require('@vladmandic/pilogger'); // this is my simple logger with few extra features
const child_process = require('child_process');
// note that main process does not import human or tfjs at all, it's all done from worker process // note that main process does not import human or tfjs at all, it's all done from worker process
const workerFile = 'demo/multithread/node-multiprocess-worker.js'; const workerFile = 'demo/nodejs/node-multiprocess-worker.js';
const imgPathRoot = './samples/in'; // modify to include your sample images const imgPathRoot = './assets'; // modify to include your sample images
const numWorkers = 4; // how many workers will be started const numWorkers = 4; // how many workers will be started
const workers = []; // this holds worker processes const workers = []; // this holds worker processes
const images = []; // this holds queue of enumerated images const images = []; // this holds queue of enumerated images
@ -22,7 +23,7 @@ let numImages;
// trigered by main when worker sends ready message // trigered by main when worker sends ready message
// if image pool is empty, signal worker to exit otherwise dispatch image to worker and remove image from queue // if image pool is empty, signal worker to exit otherwise dispatch image to worker and remove image from queue
async function submitDetect(worker) { async function detect(worker) {
if (!t[2]) t[2] = process.hrtime.bigint(); // first time do a timestamp so we can measure initial latency if (!t[2]) t[2] = process.hrtime.bigint(); // first time do a timestamp so we can measure initial latency
if (images.length === numImages) worker.send({ test: true }); // for first image in queue just measure latency if (images.length === numImages) worker.send({ test: true }); // for first image in queue just measure latency
if (images.length === 0) worker.send({ exit: true }); // nothing left in queue if (images.length === 0) worker.send({ exit: true }); // nothing left in queue
@ -57,7 +58,7 @@ async function main() {
}); });
log.header(); log.header();
log.info('Human multi-process test'); log.info('FaceAPI multi-process test');
// enumerate all images into queue // enumerate all images into queue
const dir = fs.readdirSync(imgPathRoot); const dir = fs.readdirSync(imgPathRoot);
@ -73,13 +74,13 @@ async function main() {
// manage worker processes // manage worker processes
for (let i = 0; i < numWorkers; i++) { for (let i = 0; i < numWorkers; i++) {
// create worker process // create worker process
workers[i] = await childProcess.fork(workerFile, ['special']); workers[i] = await child_process.fork(workerFile, ['special']);
// parse message that worker process sends back to main // parse message that worker process sends back to main
// if message is ready, dispatch next image in queue // if message is ready, dispatch next image in queue
// if message is processing result, just print how many faces were detected // if message is processing result, just print how many faces were detected
// otherwise it's an unknown message // otherwise it's an unknown message
workers[i].on('message', (msg) => { workers[i].on('message', (msg) => {
if (msg.ready) submitDetect(workers[i]); if (msg.ready) detect(workers[i]);
else if (msg.image) log.data('Main: worker finished:', workers[i].pid, 'detected faces:', msg.detected.face?.length, 'bodies:', msg.detected.body?.length, 'hands:', msg.detected.hand?.length, 'objects:', msg.detected.object?.length); else if (msg.image) log.data('Main: worker finished:', workers[i].pid, 'detected faces:', msg.detected.face?.length, 'bodies:', msg.detected.body?.length, 'hands:', msg.detected.hand?.length, 'objects:', msg.detected.object?.length);
else if (msg.test) measureLatency(); else if (msg.test) measureLatency();
else log.data('Main: worker message:', workers[i].pid, msg); else log.data('Main: worker message:', workers[i].pid, msg);

View File

@ -1,7 +1,7 @@
/// <reference lib="webworker" /> /// <reference lib="webworker" />
// load Human using IIFE script as Chome Mobile does not support Modules as Workers // load Human using IIFE script as Chome Mobile does not support Modules as Workers
self.importScripts('../../dist/human.js'); // eslint-disable-line no-restricted-globals self.importScripts('../../dist/human.js');
let human; let human;
@ -9,8 +9,9 @@ onmessage = async (msg) => {
// received from index.js using: // received from index.js using:
// worker.postMessage({ image: image.data.buffer, width: canvas.width, height: canvas.height, config }, [image.data.buffer]); // worker.postMessage({ image: image.data.buffer, width: canvas.width, height: canvas.height, config }, [image.data.buffer]);
// Human is registered as global namespace using IIFE script // @ts-ignore // Human is registered as global namespace using IIFE script
if (!human) human = new Human.default(msg.data.config); // eslint-disable-line no-undef, new-cap // eslint-disable-next-line no-undef, new-cap
if (!human) human = new Human.default(msg.data.config);
const image = new ImageData(new Uint8ClampedArray(msg.data.image), msg.data.width, msg.data.height); const image = new ImageData(new Uint8ClampedArray(msg.data.image), msg.data.width, msg.data.height);
let result = {}; let result = {};
result = await human.detect(image, msg.data.config); result = await human.detect(image, msg.data.config);

View File

@ -28,8 +28,7 @@ or you can pass a path to image to analyze, either on local filesystem or using
node demo/nodejs/node.js node demo/nodejs/node.js
``` ```
<!-- eslint-skip --> ```json
```js
2021-06-01 08:52:15 INFO: @vladmandic/human version 2.0.0 2021-06-01 08:52:15 INFO: @vladmandic/human version 2.0.0
2021-06-01 08:52:15 INFO: User: vlado Platform: linux Arch: x64 Node: v16.0.0 2021-06-01 08:52:15 INFO: User: vlado Platform: linux Arch: x64 Node: v16.0.0
2021-06-01 08:52:15 INFO: Current folder: /home/vlado/dev/human 2021-06-01 08:52:15 INFO: Current folder: /home/vlado/dev/human
@ -83,7 +82,7 @@ node demo/nodejs/node.js
detector: { modelPath: 'handdetect.json' }, detector: { modelPath: 'handdetect.json' },
skeleton: { modelPath: 'handskeleton.json' } skeleton: { modelPath: 'handskeleton.json' }
}, },
object: { enabled: true, modelPath: 'centernet.json', minConfidence: 0.2, iouThreshold: 0.4, maxDetected: 10, skipFrames: 19 } object: { enabled: true, modelPath: 'mb3-centernet.json', minConfidence: 0.2, iouThreshold: 0.4, maxDetected: 10, skipFrames: 19 }
} }
08:52:15.673 Human: version: 2.0.0 08:52:15.673 Human: version: 2.0.0
08:52:15.674 Human: tfjs version: 3.6.0 08:52:15.674 Human: tfjs version: 3.6.0
@ -97,7 +96,7 @@ node demo/nodejs/node.js
08:52:15.847 Human: load model: file://models/handdetect.json 08:52:15.847 Human: load model: file://models/handdetect.json
08:52:15.847 Human: load model: file://models/handskeleton.json 08:52:15.847 Human: load model: file://models/handskeleton.json
08:52:15.914 Human: load model: file://models/movenet-lightning.json 08:52:15.914 Human: load model: file://models/movenet-lightning.json
08:52:15.957 Human: load model: file://models/centernet.json 08:52:15.957 Human: load model: file://models/mb3-centernet.json
08:52:16.015 Human: load model: file://models/faceres.json 08:52:16.015 Human: load model: file://models/faceres.json
08:52:16.015 Human: tf engine state: 50796152 bytes 1318 tensors 08:52:16.015 Human: tf engine state: 50796152 bytes 1318 tensors
2021-06-01 08:52:16 INFO: Loaded: [ 'face', 'movenet', 'handpose', 'emotion', 'centernet', 'faceres', [length]: 6 ] 2021-06-01 08:52:16 INFO: Loaded: [ 'face', 'movenet', 'handpose', 'emotion', 'centernet', 'faceres', [length]: 6 ]

View File

@ -1,70 +0,0 @@
const fs = require('fs');
const process = require('process');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require
const Human = require('../../dist/human.node.js');
const humanConfig = {
debug: false,
face: {
enabled: true,
detector: { enabled: true, modelPath: 'blazeface.json' },
description: { enabled: true, modelPath: 'faceres.json' },
// gear: { enabled: true, modelPath: '/home/vlado/dev/human-models/models/gear.json' },
// ssrnet: { enabled: true, modelPathAge: '/home/vlado/dev/human-models/models/age.json', modelPathGender: '/home/vlado/dev/human-models/models/gender.json' },
emotion: { enabled: false },
mesh: { enabled: false },
iris: { enabled: false },
antispoof: { enabled: false },
liveness: { enabled: false },
},
body: { enabled: false },
hand: { enabled: false },
gesture: { enabled: false },
};
const human = new Human.Human(humanConfig);
const ageThreshold = 18;
async function detect(inputFile) {
try {
const buffer = fs.readFileSync(inputFile);
const tensor = human.tf.node.decodeImage(buffer);
const result = await human.detect(tensor);
human.tf.dispose(tensor);
if (!result || !result.face || result.face.length === 0) return false;
let msg = ` file=${inputFile} resolution=${tensor.shape}`;
for (const face of result.face) {
msg = ` file=${inputFile} resolution=${tensor.shape} age=${face.age} gender=${face.gender} confidence=${face.genderScore}`;
if (face.age < ageThreshold) {
log.warn('fail:' + msg);
return true;
}
}
log.info('pass: ' + msg);
return false;
} catch (err) {
log.error(`error: file=${inputFile}: ${err}`);
}
return false;
}
async function main() {
log.info(`Human: version=${human.version} tf=${tf.version_core}`);
process.noDeprecation = true;
if (process.argv.length < 3) return;
await human.load();
await human.warmup();
const t0 = performance.now();
const args = process.argv.slice(2);
let pass = 0;
let fail = 0;
for (const arg of args) {
const ok = await detect(arg);
if (ok) pass++;
else fail++;
}
const t1 = performance.now();
log.info(`Human: files=${args.length} pass=${pass} fail=${fail} time=${Math.round(t1 - t0)} fps=${Math.round(10000 * args.length / (t1 - t0)) / 10}`);
}
main();

View File

@ -1,66 +0,0 @@
/**
* Human simple demo for NodeJS
*/
const childProcess = require('child_process'); // eslint-disable-line camelcase
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require
const canvas = require('canvas'); // eslint-disable-line node/no-unpublished-require
const config = {
cacheSensitivity: 0.01,
wasmPlatformFetch: true,
modelBasePath: 'https://vladmandic.github.io/human-models/models/',
};
const count = 10;
async function loadImage(input) {
const inputImage = await canvas.loadImage(input);
const inputCanvas = new canvas.Canvas(inputImage.width, inputImage.height);
const inputCtx = inputCanvas.getContext('2d');
inputCtx.drawImage(inputImage, 0, 0);
const imageData = inputCtx.getImageData(0, 0, inputCanvas.width, inputCanvas.height);
process.send({ input, resolution: [inputImage.width, inputImage.height] });
return imageData;
}
async function runHuman(module, backend) {
if (backend === 'wasm') require('@tensorflow/tfjs-backend-wasm'); // eslint-disable-line node/no-unpublished-require, global-require
const Human = require('../../dist/' + module); // eslint-disable-line global-require, import/no-dynamic-require
config.backend = backend;
const human = new Human.Human(config);
human.env.Canvas = canvas.Canvas;
human.env.Image = canvas.Image;
human.env.ImageData = canvas.ImageData;
process.send({ human: human.version, module });
await human.init();
process.send({ desired: human.config.backend, wasm: human.env.wasm, tfjs: human.tf.version.tfjs, tensorflow: human.env.tensorflow });
const imageData = await loadImage('samples/in/ai-body.jpg');
const t0 = human.now();
await human.load();
const t1 = human.now();
await human.warmup();
const t2 = human.now();
for (let i = 0; i < count; i++) await human.detect(imageData);
const t3 = human.now();
process.send({ backend: human.tf.getBackend(), load: Math.round(t1 - t0), warmup: Math.round(t2 - t1), detect: Math.round(t3 - t2), count, memory: human.tf.memory().numBytes });
}
async function executeWorker(args) {
return new Promise((resolve) => {
const worker = childProcess.fork(process.argv[1], args);
worker.on('message', (msg) => log.data(msg));
worker.on('exit', () => resolve(true));
});
}
async function main() {
if (process.argv[2]) {
await runHuman(process.argv[2], process.argv[3]);
} else {
await executeWorker(['human.node.js', 'tensorflow']);
await executeWorker(['human.node-gpu.js', 'tensorflow']);
await executeWorker(['human.node-wasm.js', 'wasm']);
}
}
main();

View File

@ -1,22 +1,21 @@
/** /**
* Human demo for NodeJS using Canvas library * Human demo for NodeJS using Canvas library
*
* Requires [canvas](https://www.npmjs.com/package/canvas) to provide Canvas functionality in NodeJS environment
*/ */
const fs = require('fs'); const fs = require('fs');
const process = require('process'); const process = require('process');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require const log = require('@vladmandic/pilogger');
// in nodejs environments tfjs-node is required to be loaded before human const canvas = require('canvas');
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require
const canvas = require('canvas'); // eslint-disable-line node/no-unpublished-require // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
// const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
const config = { // just enable all and leave default settings const config = { // just enable all and leave default settings
debug: false, debug: false,
face: { enabled: true, detector: { maxDetected: 10 } }, // includes mesh, iris, emotion, descriptor face: { enabled: true }, // includes mesh, iris, emotion, descriptor
hand: { enabled: true, maxDetected: 20, minConfidence: 0.5, detector: { modelPath: 'handtrack.json' } }, // use alternative hand model hand: { enabled: true, maxDetected: 2, minConfidence: 0.5, detector: { modelPath: 'handtrack.json' } }, // use alternative hand model
body: { enabled: true }, body: { enabled: true },
object: { enabled: true }, object: { enabled: true },
gestures: { enabled: true }, gestures: { enabled: true },
@ -24,7 +23,6 @@ const config = { // just enable all and leave default settings
async function main() { async function main() {
log.header(); log.header();
process.noDeprecation = true;
globalThis.Canvas = canvas.Canvas; // patch global namespace with canvas library globalThis.Canvas = canvas.Canvas; // patch global namespace with canvas library
globalThis.ImageData = canvas.ImageData; // patch global namespace with canvas library globalThis.ImageData = canvas.ImageData; // patch global namespace with canvas library
@ -33,16 +31,15 @@ async function main() {
// init // init
const human = new Human.Human(config); // create instance of human const human = new Human.Human(config); // create instance of human
log.info('Human:', human.version, 'TF:', tf.version_core); log.info('Human:', human.version);
await human.load(); // pre-load models await human.load(); // pre-load models
log.info('Loaded models:', human.models.loaded()); log.info('Loaded models:', Object.keys(human.models).filter((a) => human.models[a]));
log.info('Memory state:', human.tf.engine().memory()); log.info('Memory state:', human.tf.engine().memory());
// parse cmdline // parse cmdline
const input = process.argv[2]; const input = process.argv[2];
let output = process.argv[3]; const output = process.argv[3];
if (!output.toLowerCase().endsWith('.jpg')) output += '.jpg';
if (process.argv.length !== 4) log.error('Parameters: <input-image> <output-image> missing'); if (process.argv.length !== 4) log.error('Parameters: <input-image> <output-image> missing');
else if (!fs.existsSync(input) && !input.startsWith('http')) log.error(`File not found: ${process.argv[2]}`); else if (!fs.existsSync(input) && !input.startsWith('http')) log.error(`File not found: ${process.argv[2]}`);
else { else {
@ -56,6 +53,9 @@ async function main() {
// run detection // run detection
const result = await human.detect(imageData); const result = await human.detect(imageData);
// run segmentation
// const seg = await human.segmentation(inputCanvas);
// log.data('Segmentation:', { data: seg.data.length, alpha: typeof seg.alpha, canvas: typeof seg.canvas });
// print results summary // print results summary
const persons = result.persons; // invoke persons getter, only used to print summary on console const persons = result.persons; // invoke persons getter, only used to print summary on console
@ -63,7 +63,7 @@ async function main() {
const face = persons[i].face; const face = persons[i].face;
const faceTxt = face ? `score:${face.score} age:${face.age} gender:${face.gender} iris:${face.iris}` : null; const faceTxt = face ? `score:${face.score} age:${face.age} gender:${face.gender} iris:${face.iris}` : null;
const body = persons[i].body; const body = persons[i].body;
const bodyTxt = body ? `score:${body.score} keypoints:${body.keypoints.length}` : null; const bodyTxt = body ? `score:${body.score} keypoints:${body.keypoints?.length}` : null;
log.data(`Detected: #${i}: Face:${faceTxt} Body:${bodyTxt} LeftHand:${persons[i].hands.left ? 'yes' : 'no'} RightHand:${persons[i].hands.right ? 'yes' : 'no'} Gestures:${persons[i].gestures.length}`); log.data(`Detected: #${i}: Face:${faceTxt} Body:${bodyTxt} LeftHand:${persons[i].hands.left ? 'yes' : 'no'} RightHand:${persons[i].hands.right ? 'yes' : 'no'} Gestures:${persons[i].gestures.length}`);
} }
@ -71,6 +71,7 @@ async function main() {
const outputCanvas = new canvas.Canvas(inputImage.width, inputImage.height); // create canvas const outputCanvas = new canvas.Canvas(inputImage.width, inputImage.height); // create canvas
const outputCtx = outputCanvas.getContext('2d'); const outputCtx = outputCanvas.getContext('2d');
outputCtx.drawImage(result.canvas || inputImage, 0, 0); // draw input image onto canvas outputCtx.drawImage(result.canvas || inputImage, 0, 0); // draw input image onto canvas
// @ts-ignore canvas is not checked for typedefs
human.draw.all(outputCanvas, result); // use human build-in method to draw results as overlays on canvas human.draw.all(outputCanvas, result); // use human build-in method to draw results as overlays on canvas
const outFile = fs.createWriteStream(output); // write canvas to new image file const outFile = fs.createWriteStream(output); // write canvas to new image file
outFile.on('finish', () => log.state('Output image:', output, outputCanvas.width, outputCanvas.height)); outFile.on('finish', () => log.state('Output image:', output, outputCanvas.width, outputCanvas.height));

View File

@ -2,13 +2,15 @@
* Human demo for NodeJS * Human demo for NodeJS
*/ */
const log = require('@vladmandic/pilogger');
const fs = require('fs'); const fs = require('fs');
const process = require('process'); const process = require('process');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require let fetch; // fetch is dynamically imported later
// in nodejs environments tfjs-node is required to be loaded before human
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
// const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
let human = null; let human = null;
@ -36,13 +38,13 @@ async function detect(input) {
let buffer; let buffer;
log.info('Loading image:', input); log.info('Loading image:', input);
if (input.startsWith('http:') || input.startsWith('https:')) { if (input.startsWith('http:') || input.startsWith('https:')) {
fetch = (await import('node-fetch')).default;
const res = await fetch(input); const res = await fetch(input);
if (res && res.ok) buffer = Buffer.from(await res.arrayBuffer()); if (res && res.ok) buffer = await res.buffer();
else log.error('Invalid image URL:', input, res.status, res.statusText, res.headers.get('content-type')); else log.error('Invalid image URL:', input, res.status, res.statusText, res.headers.get('content-type'));
} else { } else {
buffer = fs.readFileSync(input); buffer = fs.readFileSync(input);
} }
log.data('Image bytes:', buffer?.length, 'buffer:', buffer?.slice(0, 32));
// decode image using tfjs-node so we don't need external depenencies // decode image using tfjs-node so we don't need external depenencies
if (!buffer) return; if (!buffer) return;
@ -57,7 +59,6 @@ async function main() {
log.header(); log.header();
human = new Human.Human(myConfig); human = new Human.Human(myConfig);
log.info('Human:', human.version, 'TF:', tf.version_core);
if (human.events) { if (human.events) {
human.events.addEventListener('warmup', () => { human.events.addEventListener('warmup', () => {
@ -65,7 +66,8 @@ async function main() {
}); });
human.events.addEventListener('load', () => { human.events.addEventListener('load', () => {
log.info('Event Loaded:', human.models.loaded(), human.tf.engine().memory()); const loaded = Object.keys(human.models).filter((a) => human.models[a]);
log.info('Event Loaded:', loaded, human.tf.engine().memory());
}); });
human.events.addEventListener('image', () => { human.events.addEventListener('image', () => {
@ -77,7 +79,7 @@ async function main() {
const persons = human.result.persons; const persons = human.result.persons;
for (let i = 0; i < persons.length; i++) { for (let i = 0; i < persons.length; i++) {
const face = persons[i].face; const face = persons[i].face;
const faceTxt = face ? `score:${face.score} age:${face.age} gender:${face.gender} iris:${face.distance}` : null; const faceTxt = face ? `score:${face.score} age:${face.age} gender:${face.gender} iris:${face.iris}` : null;
const body = persons[i].body; const body = persons[i].body;
const bodyTxt = body ? `score:${body.score} keypoints:${body.keypoints?.length}` : null; const bodyTxt = body ? `score:${body.score} keypoints:${body.keypoints?.length}` : null;
log.data(` #${i}: Face:${faceTxt} Body:${bodyTxt} LeftHand:${persons[i].hands.left ? 'yes' : 'no'} RightHand:${persons[i].hands.right ? 'yes' : 'no'} Gestures:${persons[i].gestures.length}`); log.data(` #${i}: Face:${faceTxt} Body:${bodyTxt} LeftHand:${persons[i].hands.left ? 'yes' : 'no'} RightHand:${persons[i].hands.right ? 'yes' : 'no'} Gestures:${persons[i].gestures.length}`);

View File

@ -1,14 +1,8 @@
/**
* Human demo for NodeJS using http fetch to get image file
*
* Requires [node-fetch](https://www.npmjs.com/package/node-fetch) to provide `fetch` functionality in NodeJS environment
*/
const fs = require('fs'); const fs = require('fs');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require
// in nodejs environments tfjs-node is required to be loaded before human // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) // const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
const humanConfig = { const humanConfig = {
@ -16,15 +10,16 @@ const humanConfig = {
}; };
async function main(inputFile) { async function main(inputFile) {
global.fetch = (await import('node-fetch')).default; // eslint-disable-line node/no-unpublished-import, import/no-unresolved, node/no-missing-import, node/no-extraneous-import // @ts-ignore
global.fetch = (await import('node-fetch')).default;
const human = new Human.Human(humanConfig); // create instance of human using default configuration const human = new Human.Human(humanConfig); // create instance of human using default configuration
log.info('Human:', human.version, 'TF:', tf.version_core);
await human.load(); // optional as models would be loaded on-demand first time they are required await human.load(); // optional as models would be loaded on-demand first time they are required
await human.warmup(); // optional as model warmup is performed on-demand first time its executed await human.warmup(); // optional as model warmup is performed on-demand first time its executed
const buffer = fs.readFileSync(inputFile); // read file data into buffer const buffer = fs.readFileSync(inputFile); // read file data into buffer
const tensor = human.tf.node.decodeImage(buffer); // decode jpg data const tensor = human.tf.node.decodeImage(buffer); // decode jpg data
const result = await human.detect(tensor); // run detection; will initialize backend and on-demand load models const result = await human.detect(tensor); // run detection; will initialize backend and on-demand load models
log.data(result.gesture); // eslint-disable-next-line no-console
console.log(result.gesture);
} }
main('samples/in/ai-body.jpg'); main('samples/in/ai-body.jpg');

View File

@ -2,13 +2,13 @@
* Human Person Similarity test for NodeJS * Human Person Similarity test for NodeJS
*/ */
const log = require('@vladmandic/pilogger');
const fs = require('fs'); const fs = require('fs');
const process = require('process'); const process = require('process');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
// in nodejs environments tfjs-node is required to be loaded before human const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require // const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
let human = null; let human = null;
@ -25,19 +25,21 @@ const myConfig = {
async function init() { async function init() {
human = new Human.Human(myConfig); human = new Human.Human(myConfig);
await human.tf.ready(); await human.tf.ready();
log.info('Human:', human.version, 'TF:', tf.version_core); log.info('Human:', human.version);
await human.load(); await human.load();
log.info('Loaded:', human.models.loaded()); const loaded = Object.keys(human.models).filter((a) => human.models[a]);
log.info('Loaded:', loaded);
log.info('Memory state:', human.tf.engine().memory()); log.info('Memory state:', human.tf.engine().memory());
} }
async function detect(input) { async function detect(input) {
if (!fs.existsSync(input)) { if (!fs.existsSync(input)) {
throw new Error('Cannot load image:', input); log.error('Cannot load image:', input);
process.exit(1);
} }
const buffer = fs.readFileSync(input); const buffer = fs.readFileSync(input);
const tensor = human.tf.node.decodeImage(buffer, 3); const tensor = human.tf.node.decodeImage(buffer, 3);
log.state('Loaded image:', input, tensor.shape); log.state('Loaded image:', input, tensor['shape']);
const result = await human.detect(tensor, myConfig); const result = await human.detect(tensor, myConfig);
human.tf.dispose(tensor); human.tf.dispose(tensor);
log.state('Detected faces:', result.face.length); log.state('Detected faces:', result.face.length);
@ -45,20 +47,20 @@ async function detect(input) {
} }
async function main() { async function main() {
process.noDeprecation = true;
log.configure({ inspect: { breakLength: 265 } }); log.configure({ inspect: { breakLength: 265 } });
log.header(); log.header();
if (process.argv.length !== 4) { if (process.argv.length !== 4) {
log.error('Parameters: <first image> <second image> missing'); log.error('Parameters: <first image> <second image> missing');
return; process.exit(1);
} }
await init(); await init();
const res1 = await detect(process.argv[2]); const res1 = await detect(process.argv[2]);
const res2 = await detect(process.argv[3]); const res2 = await detect(process.argv[3]);
if (!res1 || !res1.face || res1.face.length === 0 || !res2 || !res2.face || res2.face.length === 0) { if (!res1 || !res1.face || res1.face.length === 0 || !res2 || !res2.face || res2.face.length === 0) {
throw new Error('Could not detect face descriptors'); log.error('Could not detect face descriptors');
process.exit(1);
} }
const similarity = human.match.similarity(res1.face[0].embedding, res2.face[0].embedding, { order: 2 }); const similarity = human.similarity(res1.face[0].embedding, res2.face[0].embedding, { order: 2 });
log.data('Similarity: ', similarity); log.data('Similarity: ', similarity);
} }

View File

@ -1,33 +1,19 @@
/**
* Human simple demo for NodeJS
*/
const fs = require('fs'); const fs = require('fs');
const process = require('process');
// in nodejs environments tfjs-node is required to be loaded before human // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) // const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
const humanConfig = { async function main(inputFile) {
// add any custom config here const human = new Human.Human(); // create instance of human using default configuration
debug: true,
body: { enabled: false },
};
async function detect(inputFile) {
process.noDeprecation = true;
const human = new Human.Human(humanConfig); // create instance of human using default configuration
console.log('Human:', human.version, 'TF:', tf.version_core); // eslint-disable-line no-console
await human.load(); // optional as models would be loaded on-demand first time they are required await human.load(); // optional as models would be loaded on-demand first time they are required
await human.warmup(); // optional as model warmup is performed on-demand first time its executed await human.warmup(); // optional as model warmup is performed on-demand first time its executed
const buffer = fs.readFileSync(inputFile); // read file data into buffer const buffer = fs.readFileSync(inputFile); // read file data into buffer
const tensor = human.tf.node.decodeImage(buffer); // decode jpg data const tensor = human.tf.node.decodeImage(buffer); // decode jpg data
console.log('loaded input file:', inputFile, 'resolution:', tensor.shape); // eslint-disable-line no-console
const result = await human.detect(tensor); // run detection; will initialize backend and on-demand load models const result = await human.detect(tensor); // run detection; will initialize backend and on-demand load models
console.log(result); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.log(result);
} }
if (process.argv.length === 3) detect(process.argv[2]); // if input file is provided as cmdline parameter use it main('samples/in/ai-body.jpg');
else detect('samples/in/ai-body.jpg'); // else use built-in test inputfile

View File

@ -7,24 +7,24 @@
* If you want process at specific intervals, set output fps to some value * If you want process at specific intervals, set output fps to some value
* If you want to process an input stream, set real-time flag and set input as required * If you want to process an input stream, set real-time flag and set input as required
* *
* Note that [pipe2jpeg](https://www.npmjs.com/package/pipe2jpeg) is not part of Human dependencies and should be installed manually * Note that pipe2jpeg is not part of Human dependencies and should be installed manually
* Working version of `ffmpeg` must be present on the system * Working version of ffmpeg must be present on the system
*/ */
const process = require('process');
const spawn = require('child_process').spawn; const spawn = require('child_process').spawn;
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require const log = require('@vladmandic/pilogger');
// in nodejs environments tfjs-node is required to be loaded before human // @ts-ignore pipe2jpeg is not installed by default
// const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require // eslint-disable-next-line node/no-missing-require
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) const Pipe2Jpeg = require('pipe2jpeg');
const Pipe2Jpeg = require('pipe2jpeg'); // eslint-disable-line node/no-missing-require, import/no-unresolved
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
// const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
let count = 0; // counter let count = 0; // counter
let busy = false; // busy flag let busy = false; // busy flag
let inputFile = './test.mp4'; const inputFile = './test.mp4';
if (process.argv.length === 3) inputFile = process.argv[2];
const humanConfig = { const humanConfig = {
modelBasePath: 'file://models/', modelBasePath: 'file://models/',
@ -61,27 +61,24 @@ const ffmpegParams = [
'pipe:1', // output to unix pipe that is then captured by pipe2jpeg 'pipe:1', // output to unix pipe that is then captured by pipe2jpeg
]; ];
async function detect(jpegBuffer) { async function process(jpegBuffer) {
if (busy) return; // skip processing if busy if (busy) return; // skip processing if busy
busy = true; busy = true;
const tensor = human.tf.node.decodeJpeg(jpegBuffer, 3); // decode jpeg buffer to raw tensor const tensor = human.tf.node.decodeJpeg(jpegBuffer, 3); // decode jpeg buffer to raw tensor
log.state('input frame:', ++count, 'size:', jpegBuffer.length, 'decoded shape:', tensor.shape);
const res = await human.detect(tensor); const res = await human.detect(tensor);
human.tf.dispose(tensor); // must dispose tensor log.data('gesture', JSON.stringify(res.gesture));
// start custom processing here // do processing here
log.data('frame', { frame: ++count, size: jpegBuffer.length, shape: tensor.shape, face: res?.face?.length, body: res?.body?.length, hand: res?.hand?.length, gesture: res?.gesture?.length }); tf.dispose(tensor); // must dispose tensor
if (res?.face?.[0]) log.data('person', { score: [res.face[0].boxScore, res.face[0].faceScore], age: res.face[0].age || 0, gender: [res.face[0].genderScore || 0, res.face[0].gender], emotion: res.face[0].emotion?.[0] });
// at the of processing mark loop as not busy so it can process next frame
busy = false; busy = false;
} }
async function main() { async function main() {
log.header(); log.header();
process.noDeprecation = true;
await human.tf.ready(); await human.tf.ready();
// pre-load models // pre-load models
log.info({ human: human.version, tf: human.tf.version_core }); log.info('human:', human.version);
log.info({ input: inputFile }); pipe2jpeg.on('jpeg', (jpegBuffer) => process(jpegBuffer));
pipe2jpeg.on('data', (jpegBuffer) => detect(jpegBuffer));
const ffmpeg = spawn('ffmpeg', ffmpegParams, { stdio: ['ignore', 'pipe', 'ignore'] }); const ffmpeg = spawn('ffmpeg', ffmpegParams, { stdio: ['ignore', 'pipe', 'ignore'] });
ffmpeg.on('error', (error) => log.error('ffmpeg error:', error)); ffmpeg.on('error', (error) => log.error('ffmpeg error:', error));

View File

@ -2,17 +2,19 @@
* Human demo for NodeJS * Human demo for NodeJS
* Unsupported sample of using external utility fswebcam to capture screenshot from attached webcam in regular intervals and process it using Human * Unsupported sample of using external utility fswebcam to capture screenshot from attached webcam in regular intervals and process it using Human
* *
* Note that [node-webcam](https://www.npmjs.com/package/node-webcam) is not part of Human dependencies and should be installed manually * Note that node-webcam is not part of Human dependencies and should be installed manually
* Working version of `fswebcam` must be present on the system * Working version of fswebcam must be present on the system
*/ */
let initial = true; // remember if this is the first run to print additional details let initial = true; // remember if this is the first run to print additional details
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require const log = require('@vladmandic/pilogger');
const nodeWebCam = require('node-webcam'); // eslint-disable-line import/no-unresolved, node/no-missing-require // @ts-ignore node-webcam is not installed by default
// eslint-disable-next-line node/no-missing-require
const nodeWebCam = require('node-webcam');
// in nodejs environments tfjs-node is required to be loaded before human // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) // const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
// options for node-webcam // options for node-webcam
@ -27,7 +29,6 @@ const camera = nodeWebCam.create(optionsCamera);
const optionsHuman = { const optionsHuman = {
modelBasePath: 'file://models/', modelBasePath: 'file://models/',
}; };
const human = new Human.Human(optionsHuman); const human = new Human.Human(optionsHuman);
function buffer2tensor(buffer) { function buffer2tensor(buffer) {
@ -60,8 +61,8 @@ async function detect() {
} else { } else {
const tensor = buffer2tensor(data); // create tensor from image buffer const tensor = buffer2tensor(data); // create tensor from image buffer
if (initial) log.data('input tensor:', tensor.shape); if (initial) log.data('input tensor:', tensor.shape);
human.detect(tensor) // eslint-disable-line promise/no-promise-in-callback // eslint-disable-next-line promise/no-promise-in-callback
.then((result) => { human.detect(tensor).then((result) => {
if (result && result.face && result.face.length > 0) { if (result && result.face && result.face.length > 0) {
for (let i = 0; i < result.face.length; i++) { for (let i = 0; i < result.face.length; i++) {
const face = result.face[i]; const face = result.face[i];
@ -71,9 +72,7 @@ async function detect() {
} else { } else {
log.data(' Face: N/A'); log.data(' Face: N/A');
} }
return result; });
})
.catch(() => log.error('human detect error'));
} }
initial = false; initial = false;
}); });
@ -82,7 +81,6 @@ async function detect() {
} }
async function main() { async function main() {
log.info('human:', human.version, 'tf:', tf.version_core);
camera.list((list) => { camera.list((list) => {
log.data('detected camera:', list); log.data('detected camera:', list);
}); });

View File

@ -1,15 +1,17 @@
/** /**
* Human demo for NodeJS * Human demo for NodeJS
*/ */
const log = require('@vladmandic/pilogger');
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const process = require('process'); const process = require('process');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require
// in nodejs environments tfjs-node is required to be loaded before human let fetch; // fetch is dynamically imported later
const tf = require('@tensorflow/tfjs-node'); // eslint-disable-line node/no-unpublished-require
// const human = require('@vladmandic/human'); // use this when human is installed as module (majority of use cases) // eslint-disable-next-line import/no-extraneous-dependencies, no-unused-vars, @typescript-eslint/no-unused-vars
const tf = require('@tensorflow/tfjs-node'); // in nodejs environments tfjs-node is required to be loaded before human
// const faceapi = require('@vladmandic/face-api'); // use this when human is installed as module (majority of use cases)
const Human = require('../../dist/human.node.js'); // use this when using human in dev mode const Human = require('../../dist/human.node.js'); // use this when using human in dev mode
let human = null; let human = null;
@ -44,14 +46,14 @@ async function init() {
human = new Human.Human(myConfig); human = new Human.Human(myConfig);
// wait until tf is ready // wait until tf is ready
await human.tf.ready(); await human.tf.ready();
log.info('human:', human.version, 'tf:', tf.version_core);
// pre-load models // pre-load models
log.info('Human:', human.version); log.info('Human:', human.version);
// log.info('Active Configuration', human.config); // log.info('Active Configuration', human.config);
await human.load(); await human.load();
log.info('Loaded:', human.models.loaded()); const loaded = Object.keys(human.models).filter((a) => human.models[a]);
log.info('Loaded:', loaded);
// log.info('Memory state:', human.tf.engine().memory()); // log.info('Memory state:', human.tf.engine().memory());
log.data(tf.backend().binding ? tf.backend().binding.TF_Version : null); log.data(tf.backend()['binding'] ? tf.backend()['binding']['TF_Version'] : null);
} }
async function detect(input) { async function detect(input) {
@ -60,12 +62,11 @@ async function detect(input) {
log.info('Loading image:', input); log.info('Loading image:', input);
if (input.startsWith('http:') || input.startsWith('https:')) { if (input.startsWith('http:') || input.startsWith('https:')) {
const res = await fetch(input); const res = await fetch(input);
if (res && res.ok) buffer = Buffer.from(await res.arrayBuffer()); if (res && res.ok) buffer = await res.buffer();
else log.error('Invalid image URL:', input, res.status, res.statusText, res.headers.get('content-type')); else log.error('Invalid image URL:', input, res.status, res.statusText, res.headers.get('content-type'));
} else { } else {
buffer = fs.readFileSync(input); buffer = fs.readFileSync(input);
} }
log.data('Image bytes:', buffer?.length, 'buffer:', buffer?.slice(0, 32));
// decode image using tfjs-node so we don't need external depenencies // decode image using tfjs-node so we don't need external depenencies
// can also be done using canvas.js or some other 3rd party image library // can also be done using canvas.js or some other 3rd party image library
@ -85,14 +86,14 @@ async function detect(input) {
}); });
// image shape contains image dimensions and depth // image shape contains image dimensions and depth
log.state('Processing:', tensor.shape); log.state('Processing:', tensor['shape']);
// run actual detection // run actual detection
let result; let result;
try { try {
result = await human.detect(tensor, myConfig); result = await human.detect(tensor, myConfig);
} catch (err) { } catch (err) {
log.error('caught', err); log.error('caught');
} }
// dispose image tensor as we no longer need it // dispose image tensor as we no longer need it
@ -104,7 +105,7 @@ async function detect(input) {
for (let i = 0; i < result.face.length; i++) { for (let i = 0; i < result.face.length; i++) {
const face = result.face[i]; const face = result.face[i];
const emotion = face.emotion.reduce((prev, curr) => (prev.score > curr.score ? prev : curr)); const emotion = face.emotion.reduce((prev, curr) => (prev.score > curr.score ? prev : curr));
log.data(` Face: #${i} boxScore:${face.boxScore} faceScore:${face.faceScore} age:${face.age} genderScore:${face.genderScore} gender:${face.gender} emotionScore:${emotion.score} emotion:${emotion.emotion} distance:${face.distance}`); log.data(` Face: #${i} boxScore:${face.boxScore} faceScore:${face.faceScore} age:${face.age} genderScore:${face.genderScore} gender:${face.gender} emotionScore:${emotion.score} emotion:${emotion.emotion} iris:${face.iris}`);
} }
} else { } else {
log.data(' Face: N/A'); log.data(' Face: N/A');
@ -187,8 +188,8 @@ async function test() {
async function main() { async function main() {
log.configure({ inspect: { breakLength: 265 } }); log.configure({ inspect: { breakLength: 265 } });
log.header(); log.header();
process.noDeprecation = true;
log.info('Current folder:', process.env.PWD); log.info('Current folder:', process.env.PWD);
fetch = (await import('node-fetch')).default;
await init(); await init();
const f = process.argv[2]; const f = process.argv[2];
if (process.argv.length !== 3) { if (process.argv.length !== 3) {
@ -196,7 +197,8 @@ async function main() {
await test(); await test();
} else if (!fs.existsSync(f) && !f.startsWith('http')) { } else if (!fs.existsSync(f) && !f.startsWith('http')) {
log.error(`File not found: ${process.argv[2]}`); log.error(`File not found: ${process.argv[2]}`);
} else if (fs.existsSync(f)) { } else {
if (fs.existsSync(f)) {
const stat = fs.statSync(f); const stat = fs.statSync(f);
if (stat.isDirectory()) { if (stat.isDirectory()) {
const dir = fs.readdirSync(f); const dir = fs.readdirSync(f);
@ -209,6 +211,7 @@ async function main() {
} else { } else {
await detect(f); await detect(f);
} }
}
} }
main(); main();

View File

@ -1,64 +1,56 @@
/**
* Human demo for NodeJS
*
* Takes input and output folder names parameters and processes all images
* found in input folder and creates annotated images in output folder
*
* Requires [canvas](https://www.npmjs.com/package/canvas) to provide Canvas functionality in NodeJS environment
*/
const fs = require('fs'); const fs = require('fs');
const path = require('path'); const path = require('path');
const process = require('process'); const process = require('process');
const log = require('@vladmandic/pilogger'); // eslint-disable-line node/no-unpublished-require const log = require('@vladmandic/pilogger');
const canvas = require('canvas'); // eslint-disable-line node/no-unpublished-require const canvas = require('canvas');
// for nodejs, `tfjs-node` or `tfjs-node-gpu` should be loaded before using Human // const tf = require('@tensorflow/tfjs-node-gpu'); // for nodejs, `tfjs-node` or `tfjs-node-gpu` should be loaded before using Human
const tf = require('@tensorflow/tfjs-node-gpu'); // eslint-disable-line node/no-unpublished-require
const Human = require('../../dist/human.node-gpu.js'); // this is 'const Human = require('../dist/human.node-gpu.js').default;' const Human = require('../../dist/human.node-gpu.js'); // this is 'const Human = require('../dist/human.node-gpu.js').default;'
const config = { // just enable all and leave default settings const config = { // just enable all and leave default settings
modelBasePath: 'file://models',
debug: true, debug: true,
softwareKernels: true, // slower but enhanced precision since face rotation can work in software mode in nodejs environments async: false,
cacheSensitivity: 0.01, cacheSensitivity: 0,
face: { enabled: true, detector: { maxDetected: 100, minConfidence: 0.1 } }, face: { enabled: true, detector: { maxDetected: 20 } },
object: { enabled: true, maxDetected: 100, minConfidence: 0.1 }, object: { enabled: true },
gesture: { enabled: true }, gesture: { enabled: true },
hand: { enabled: true, maxDetected: 100, minConfidence: 0.2 }, hand: { enabled: true },
body: { enabled: true, maxDetected: 100, minConfidence: 0.1, modelPath: 'https://vladmandic.github.io/human-models/models/movenet-multipose.json' }, body: { enabled: true, modelPath: 'https://vladmandic.github.io/human-models/models/movenet-multipose.json' },
}; };
const poolSize = 4; async function main() {
log.header();
const human = new Human.Human(config); // create instance of human globalThis.Canvas = canvas.Canvas; // patch global namespace with canvas library
globalThis.ImageData = canvas.ImageData; // patch global namespace with canvas library
async function saveFile(shape, buffer, result, outFile) { const human = new Human.Human(config); // create instance of human
return new Promise(async (resolve, reject) => { // eslint-disable-line no-async-promise-executor log.info('Human:', human.version);
const outputCanvas = new canvas.Canvas(shape[2], shape[1]); // create canvas const configErrors = await human.validate();
const outputCtx = outputCanvas.getContext('2d'); if (configErrors.length > 0) log.error('Configuration errors:', configErrors);
const inputImage = await canvas.loadImage(buffer); // load image using canvas library await human.load(); // pre-load models
outputCtx.drawImage(inputImage, 0, 0); // draw input image onto canvas log.info('Loaded models:', Object.keys(human.models).filter((a) => human.models[a]));
human.draw.all(outputCanvas, result); // use human build-in method to draw results as overlays on canvas
const outStream = fs.createWriteStream(outFile); // write canvas to new image file
outStream.on('finish', () => {
log.data('Output image:', outFile, outputCanvas.width, outputCanvas.height);
resolve();
});
outStream.on('error', (err) => {
log.error('Output error:', outFile, err);
reject();
});
const stream = outputCanvas.createJPEGStream({ quality: 0.5, progressive: true, chromaSubsampling: true });
stream.pipe(outStream);
});
}
async function processFile(image, inFile, outFile) { const inDir = process.argv[2];
const outDir = process.argv[3];
if (process.argv.length !== 4) {
log.error('Parameters: <input-directory> <output-directory> missing');
return;
}
if (!fs.existsSync(inDir) || !fs.statSync(inDir).isDirectory() || !fs.existsSync(outDir) || !fs.statSync(outDir).isDirectory()) {
log.error('Invalid directory specified:', 'input:', fs.existsSync(inDir) ?? fs.statSync(inDir).isDirectory(), 'output:', fs.existsSync(outDir) ?? fs.statSync(outDir).isDirectory());
return;
}
const dir = fs.readdirSync(inDir);
const images = dir.filter((f) => fs.statSync(path.join(inDir, f)).isFile() && (f.toLocaleLowerCase().endsWith('.jpg') || f.toLocaleLowerCase().endsWith('.jpeg')));
log.info(`Processing folder: ${inDir} entries:`, dir.length, 'images', images.length);
for (const image of images) {
const inFile = path.join(inDir, image);
const buffer = fs.readFileSync(inFile); const buffer = fs.readFileSync(inFile);
const tensor = tf.tidy(() => { const tensor = human.tf.tidy(() => {
const decode = tf.node.decodeImage(buffer, 3); const decode = human.tf.node.decodeImage(buffer, 3);
const expand = tf.expandDims(decode, 0); const expand = human.tf.expandDims(decode, 0);
const cast = tf.cast(expand, 'float32'); const cast = human.tf.cast(expand, 'float32');
return cast; return cast;
}); });
log.state('Loaded image:', inFile, tensor.shape); log.state('Loaded image:', inFile, tensor.shape);
@ -67,54 +59,20 @@ async function processFile(image, inFile, outFile) {
human.tf.dispose(tensor); human.tf.dispose(tensor);
log.data(`Detected: ${image}:`, 'Face:', result.face.length, 'Body:', result.body.length, 'Hand:', result.hand.length, 'Objects:', result.object.length, 'Gestures:', result.gesture.length); log.data(`Detected: ${image}:`, 'Face:', result.face.length, 'Body:', result.body.length, 'Hand:', result.hand.length, 'Objects:', result.object.length, 'Gestures:', result.gesture.length);
if (outFile) await saveFile(tensor.shape, buffer, result, outFile); const outputCanvas = new canvas.Canvas(tensor.shape[2], tensor.shape[1]); // create canvas
} const outputCtx = outputCanvas.getContext('2d');
const inputImage = await canvas.loadImage(buffer); // load image using canvas library
async function main() { outputCtx.drawImage(inputImage, 0, 0); // draw input image onto canvas
log.header(); // @ts-ignore
human.draw.all(outputCanvas, result); // use human build-in method to draw results as overlays on canvas
globalThis.Canvas = canvas.Canvas; // patch global namespace with canvas library const outFile = path.join(outDir, image);
globalThis.ImageData = canvas.ImageData; // patch global namespace with canvas library const outStream = fs.createWriteStream(outFile); // write canvas to new image file
outStream.on('finish', () => log.state('Output image:', outFile, outputCanvas.width, outputCanvas.height));
log.info('Human:', human.version, 'TF:', tf.version_core); outStream.on('error', (err) => log.error('Output error:', outFile, err));
process.noDeprecation = true; const stream = outputCanvas.createJPEGStream({ quality: 0.5, progressive: true, chromaSubsampling: true });
const configErrors = await human.validate(); // @ts-ignore
if (configErrors.length > 0) log.error('Configuration errors:', configErrors); stream.pipe(outStream);
await human.load(); // pre-load models
log.info('Loaded models:', human.models.loaded());
const inDir = process.argv[2];
const outDir = process.argv[3];
if (!inDir) {
log.error('Parameters: <input-directory> missing');
return;
} }
if (inDir && (!fs.existsSync(inDir) || !fs.statSync(inDir).isDirectory())) {
log.error('Invalid input directory:', fs.existsSync(inDir) ?? fs.statSync(inDir).isDirectory());
return;
}
if (!outDir) {
log.info('Parameters: <output-directory> missing, images will not be saved');
}
if (outDir && (!fs.existsSync(outDir) || !fs.statSync(outDir).isDirectory())) {
log.error('Invalid output directory:', fs.existsSync(outDir) ?? fs.statSync(outDir).isDirectory());
return;
}
const dir = fs.readdirSync(inDir);
const images = dir.filter((f) => fs.statSync(path.join(inDir, f)).isFile() && (f.toLocaleLowerCase().endsWith('.jpg') || f.toLocaleLowerCase().endsWith('.jpeg')));
log.info(`Processing folder: ${inDir} entries:`, dir.length, 'images', images.length);
const t0 = performance.now();
const promises = [];
for (let i = 0; i < images.length; i++) {
const inFile = path.join(inDir, images[i]);
const outFile = outDir ? path.join(outDir, images[i]) : null;
promises.push(processFile(images[i], inFile, outFile));
if (i % poolSize === 0) await Promise.all(promises);
}
await Promise.all(promises);
const t1 = performance.now();
log.info(`Processed ${images.length} images in ${Math.round(t1 - t0)} ms`);
} }
main(); main();

View File

@ -24,13 +24,13 @@
a:hover { color: lightskyblue; text-decoration: none; } a:hover { color: lightskyblue; text-decoration: none; }
.row { width: 90vw; margin: auto; margin-top: 100px; text-align: center; } .row { width: 90vw; margin: auto; margin-top: 100px; text-align: center; }
</style> </style>
</head> </head>
<body> <body>
<div class="row text-center"> <div class="row text-center">
<h1> <h1>
<a href="/">Human: Offline</a><br> <a href="/">Human: Offline</a><br>
<img alt="icon" src="../assets/icon.png"> <img alt="icon" src="../assets/icon.png">
</h1> </h1>
</div> </div>
</body> </body>
</html> </html>

View File

@ -1,61 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<title>Human Demo</title>
<meta name="viewport" content="width=device-width, shrink-to-fit=yes">
<meta name="mobile-web-app-capable" content="yes">
<meta name="application-name" content="Human Demo">
<meta name="keywords" content="Human Demo">
<meta name="description" content="Human Demo; Author: Vladimir Mandic <mandic00@live.com>">
<link rel="manifest" href="../manifest.webmanifest">
<link rel="shortcut icon" href="../favicon.ico" type="image/x-icon">
<link rel="icon" sizes="256x256" href="../assets/icons/dash-256.png">
<link rel="apple-touch-icon" href="../assets/icons/dash-256.png">
<link rel="apple-touch-startup-image" href="../assets/icons/dash-256.png">
<style>
@font-face { font-family: 'CenturyGothic'; font-display: swap; font-style: normal; font-weight: 400; src: local('CenturyGothic'), url('../assets/century-gothic.ttf') format('truetype'); }
html { font-size: 18px; }
body { font-size: 1rem; font-family: "CenturyGothic", "Segoe UI", sans-serif; font-variant: small-caps; width: -webkit-fill-available; height: 100%; background: black; color: white; overflow: hidden; margin: 0; }
select { font-size: 1rem; font-family: "CenturyGothic", "Segoe UI", sans-serif; font-variant: small-caps; background: gray; color: white; border: none; }
</style>
<script src="../segmentation/index.js" type="module"></script>
</head>
<body>
<noscript><h1>javascript is required</h1></noscript>
<nav>
<div id="nav" class="nav"></div>
</nav>
<header>
<div id="header" class="header" style="position: fixed; top: 0; right: 0; padding: 4px; margin: 16px; background: rgba(0, 0, 0, 0.5); z-index: 10; line-height: 2rem;">
<label for="mode">mode</label>
<select id="mode" name="mode">
<option value="default">remove background</option>
<option value="alpha">draw alpha channel</option>
<option value="foreground">full foreground</option>
<option value="state">recurrent state</option>
</select><br>
<label for="composite">composite</label>
<select id="composite" name="composite"></select><br>
<label for="ratio">downsample ratio</label>
<input type="range" name="ratio" id="ratio" min="0.1" max="1" value="0.5" step="0.05">
<div id="fps" style="margin-top: 8px"></div>
</div>
</header>
<main>
<div id="main" class="main">
<video id="webcam" style="position: fixed; top: 0; left: 0; width: 50vw; height: 50vh"></video>
<img id="background" alt="background" style="position: fixed; top: 0; right: 0; width: 50vw; height: 50vh" controls></img>
<canvas id="output" style="position: fixed; bottom: 0; left: 0; height: 50vh"></canvas>
<canvas id="merge" style="position: fixed; bottom: 0; right: 0; height: 50vh"></canvas>
</div>
</main>
<footer>
<div id="footer" class="footer"></div>
</footer>
<aside>
<div id="aside" class="aside"></div>
</aside>
</body>
</html>

View File

@ -1,99 +0,0 @@
/**
* Human demo for browsers
* @default Human Library
* @summary <https://github.com/vladmandic/human>
* @author <https://github.com/vladmandic>
* @copyright <https://github.com/vladmandic>
* @license MIT
*/
import * as H from '../../dist/human.esm.js'; // equivalent of @vladmandic/Human
const humanConfig = { // user configuration for human, used to fine-tune behavior
modelBasePath: 'https://vladmandic.github.io/human-models/models/',
filter: { enabled: true, equalization: false, flip: false },
face: { enabled: false },
body: { enabled: false },
hand: { enabled: false },
object: { enabled: false },
gesture: { enabled: false },
segmentation: {
enabled: true,
modelPath: 'rvm.json', // can use rvm, selfie or meet
ratio: 0.5,
mode: 'default',
},
};
const backgroundImage = '../../samples/in/background.jpg';
const human = new H.Human(humanConfig); // create instance of human with overrides from user configuration
const log = (...msg) => console.log(...msg); // eslint-disable-line no-console
async function main() {
// gather dom elements
const dom = {
background: document.getElementById('background'),
webcam: document.getElementById('webcam'),
output: document.getElementById('output'),
merge: document.getElementById('merge'),
mode: document.getElementById('mode'),
composite: document.getElementById('composite'),
ratio: document.getElementById('ratio'),
fps: document.getElementById('fps'),
};
// set defaults
dom.fps.innerText = 'initializing';
dom.ratio.valueAsNumber = human.config.segmentation.ratio;
dom.background.src = backgroundImage;
dom.composite.innerHTML = ['source-atop', 'color', 'color-burn', 'color-dodge', 'copy', 'darken', 'destination-atop', 'destination-in', 'destination-out', 'destination-over', 'difference', 'exclusion', 'hard-light', 'hue', 'lighten', 'lighter', 'luminosity', 'multiply', 'overlay', 'saturation', 'screen', 'soft-light', 'source-in', 'source-out', 'source-over', 'xor'].map((gco) => `<option value="${gco}">${gco}</option>`).join(''); // eslint-disable-line max-len
const ctxMerge = dom.merge.getContext('2d');
log('human version:', human.version, '| tfjs version:', human.tf.version['tfjs-core']);
log('platform:', human.env.platform, '| agent:', human.env.agent);
await human.load(); // preload all models
log('backend:', human.tf.getBackend(), '| available:', human.env.backends);
log('models stats:', human.models.stats());
log('models loaded:', human.models.loaded());
await human.warmup(); // warmup function to initialize backend for future faster detection
const numTensors = human.tf.engine().state.numTensors;
// initialize webcam
dom.webcam.onplay = () => { // start processing on video play
log('start processing');
dom.output.width = human.webcam.width;
dom.output.height = human.webcam.height;
dom.merge.width = human.webcam.width;
dom.merge.height = human.webcam.height;
loop(); // eslint-disable-line no-use-before-define
};
await human.webcam.start({ element: dom.webcam, crop: true, width: window.innerWidth / 2, height: window.innerHeight / 2 }); // use human webcam helper methods and associate webcam stream with a dom element
if (!human.webcam.track) dom.fps.innerText = 'webcam error';
// processing loop
async function loop() {
if (!human.webcam.element || human.webcam.paused) return; // check if webcam is valid and playing
human.config.segmentation.mode = dom.mode.value; // get segmentation mode from ui
human.config.segmentation.ratio = dom.ratio.valueAsNumber; // get segmentation downsample ratio from ui
const t0 = Date.now();
const rgba = await human.segmentation(human.webcam.element, human.config); // run model and process results
const t1 = Date.now();
if (!rgba) {
dom.fps.innerText = 'error';
return;
}
dom.fps.innerText = `fps: ${Math.round(10000 / (t1 - t0)) / 10}`; // mark performance
human.draw.tensor(rgba, dom.output); // draw raw output
human.tf.dispose(rgba); // dispose tensors
ctxMerge.globalCompositeOperation = 'source-over';
ctxMerge.drawImage(dom.background, 0, 0); // draw original video to first stacked canvas
ctxMerge.globalCompositeOperation = dom.composite.value;
ctxMerge.drawImage(dom.output, 0, 0); // draw processed output to second stacked canvas
if (numTensors !== human.tf.engine().state.numTensors) log({ leak: human.tf.engine().state.numTensors - numTensors }); // check for memory leaks
requestAnimationFrame(loop);
}
}
window.onload = main;

View File

@ -1,28 +0,0 @@
## Tracker
### Based on
<https://github.com/opendatacam/node-moving-things-tracker>
### Build
- remove reference to `lodash`:
> `isEqual` in <tracker.js>
- replace external lib:
> curl https://raw.githubusercontent.com/ubilabs/kd-tree-javascript/master/kdTree.js -o lib/kdTree-min.js
- build with `esbuild`:
> node_modules/.bin/esbuild --bundle tracker.js --format=esm --platform=browser --target=esnext --keep-names --tree-shaking=false --analyze --outfile=/home/vlado/dev/human/demo/tracker/tracker.js --banner:js="/* eslint-disable */"
### Usage
computeDistance(item1, item2)
disableKeepInMemory()
enableKeepInMemory()
getAllTrackedItems()
getJSONDebugOfTrackedItems(roundInt = true)
getJSONOfAllTrackedItems()
getJSONOfTrackedItems(roundInt = true)
getTrackedItemsInMOTFormat(frameNb)
reset()
setParams(newParams)
updateTrackedItemsWithNewFrame(detectionsOfThisFrame, frameNb)

View File

@ -1,65 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Human</title>
<meta name="viewport" content="width=device-width" id="viewport">
<meta name="keywords" content="Human">
<meta name="application-name" content="Human">
<meta name="description" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="msapplication-tooltip" content="Human: 3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition; Author: Vladimir Mandic <https://github.com/vladmandic>">
<meta name="theme-color" content="#000000">
<link rel="manifest" href="../manifest.webmanifest">
<link rel="shortcut icon" href="../../favicon.ico" type="image/x-icon">
<link rel="apple-touch-icon" href="../../assets/icon.png">
<script src="./index.js" type="module"></script>
<style>
html { font-family: 'Segoe UI'; font-size: 16px; font-variant: small-caps; }
body { margin: 0; background: black; color: white; overflow-x: hidden; width: 100vw; height: 100vh; }
body::-webkit-scrollbar { display: none; }
input[type="file"] { font-family: 'Segoe UI'; font-size: 14px; font-variant: small-caps; }
::-webkit-file-upload-button { background: #333333; color: white; border: 0; border-radius: 0; padding: 6px 16px; box-shadow: 4px 4px 4px #222222; font-family: 'Segoe UI'; font-size: 14px; font-variant: small-caps; }
</style>
</head>
<body>
<div style="display: flex">
<video id="video" playsinline style="width: 25vw" controls controlslist="nofullscreen nodownload noremoteplayback" disablepictureinpicture loop></video>
<canvas id="canvas" style="width: 75vw"></canvas>
</div>
<div class="uploader" style="padding: 8px">
<input type="file" name="inputvideo" id="inputvideo" accept="video/*"></input>
<input type="checkbox" id="interpolation" name="interpolation"></input>
<label for="tracker">interpolation</label>
</div>
<form id="config" style="padding: 8px; line-height: 1.6rem;">
tracker |
<input type="checkbox" id="tracker" name="tracker" checked></input>
<label for="tracker">enabled</label> |
<input type="checkbox" id="keepInMemory" name="keepInMemory"></input>
<label for="keepInMemory">keepInMemory</label> |
<br>
tracker source |
<input type="radio" id="box-face" name="box" value="face" checked>
<label for="box-face">face</label> |
<input type="radio" id="box-body" name="box" value="body">
<label for="box-face">body</label> |
<input type="radio" id="box-object" name="box" value="object">
<label for="box-face">object</label> |
<br>
tracker config |
<input type="range" id="unMatchedFramesTolerance" name="unMatchedFramesTolerance" min="0" max="300" step="1", value="60"></input>
<label for="unMatchedFramesTolerance">unMatchedFramesTolerance</label> |
<input type="range" id="iouLimit" name="unMatchedFramesTolerance" min="0" max="1" step="0.01", value="0.1"></input>
<label for="iouLimit">iouLimit</label> |
<input type="range" id="distanceLimit" name="unMatchedFramesTolerance" min="0" max="1" step="0.01", value="0.1"></input>
<label for="distanceLimit">distanceLimit</label> |
<input type="radio" id="matchingAlgorithm-kdTree" name="matchingAlgorithm" value="kdTree" checked>
<label for="matchingAlgorithm-kdTree">kdTree</label> |
<input type="radio" id="matchingAlgorithm-munkres" name="matchingAlgorithm" value="munkres">
<label for="matchingAlgorithm-kdTree">munkres</label> |
</form>
<pre id="status" style="position: absolute; top: 12px; right: 20px; background-color: grey; padding: 8px; box-shadow: 2px 2px black"></pre>
<pre id="log" style="padding: 8px"></pre>
<div id="performance" style="position: absolute; bottom: 0; width: 100%; padding: 8px; font-size: 0.8rem;"></div>
</body>
</html>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -1,208 +0,0 @@
/**
* Human demo for browsers
* @default Human Library
* @summary <https://github.com/vladmandic/human>
* @author <https://github.com/vladmandic>
* @copyright <https://github.com/vladmandic>
* @license MIT
*/
import * as H from '../../dist/human.esm.js'; // equivalent of @vladmandic/Human
import tracker from './tracker.js';
const humanConfig: Partial<H.Config> = { // user configuration for human, used to fine-tune behavior
debug: true,
backend: 'webgl',
// cacheSensitivity: 0,
// cacheModels: false,
// warmup: 'none',
modelBasePath: 'https://vladmandic.github.io/human-models/models',
filter: { enabled: true, equalization: false, flip: false },
face: {
enabled: true,
detector: { rotation: false, maxDetected: 10, minConfidence: 0.3 },
mesh: { enabled: true },
attention: { enabled: false },
iris: { enabled: false },
description: { enabled: false },
emotion: { enabled: false },
antispoof: { enabled: false },
liveness: { enabled: false },
},
body: { enabled: false, maxDetected: 6, modelPath: 'movenet-multipose.json' },
hand: { enabled: false },
object: { enabled: false, maxDetected: 10 },
segmentation: { enabled: false },
gesture: { enabled: false },
};
interface TrackerConfig {
unMatchedFramesTolerance: number, // number of frame when an object is not matched before considering it gone; ignored if fastDelete is set
iouLimit: number, // exclude things from beeing matched if their IOU less than; 1 means total overlap; 0 means no overlap
fastDelete: boolean, // remove new objects immediately if they could not be matched in the next frames; if set, ignores unMatchedFramesTolerance
distanceLimit: number, // distance limit for matching; if values need to be excluded from matching set their distance to something greater than the distance limit
matchingAlgorithm: 'kdTree' | 'munkres', // algorithm used to match tracks with new detections
}
interface TrackerResult {
id: number,
confidence: number,
bearing: number,
isZombie: boolean,
name: string,
x: number,
y: number,
w: number,
h: number,
}
const trackerConfig: TrackerConfig = {
unMatchedFramesTolerance: 100,
iouLimit: 0.05,
fastDelete: false,
distanceLimit: 1e4,
matchingAlgorithm: 'kdTree',
};
const human = new H.Human(humanConfig); // create instance of human with overrides from user configuration
const dom = { // grab instances of dom objects so we dont have to look them up later
video: document.getElementById('video') as HTMLVideoElement,
canvas: document.getElementById('canvas') as HTMLCanvasElement,
log: document.getElementById('log') as HTMLPreElement,
fps: document.getElementById('status') as HTMLPreElement,
tracker: document.getElementById('tracker') as HTMLInputElement,
interpolation: document.getElementById('interpolation') as HTMLInputElement,
config: document.getElementById('config') as HTMLFormElement,
ctx: (document.getElementById('canvas') as HTMLCanvasElement).getContext('2d') as CanvasRenderingContext2D,
};
const timestamp = { detect: 0, draw: 0, tensors: 0, start: 0 }; // holds information used to calculate performance and possible memory leaks
const fps = { detectFPS: 0, drawFPS: 0, frames: 0, averageMs: 0 }; // holds calculated fps information for both detect and screen refresh
const log = (...msg) => { // helper method to output messages
dom.log.innerText += msg.join(' ') + '\n';
console.log(...msg); // eslint-disable-line no-console
};
const status = (msg) => dom.fps.innerText = msg; // print status element
async function detectionLoop() { // main detection loop
if (!dom.video.paused && dom.video.readyState >= 2) {
if (timestamp.start === 0) timestamp.start = human.now();
// log('profiling data:', await human.profile(dom.video));
await human.detect(dom.video, humanConfig); // actual detection; were not capturing output in a local variable as it can also be reached via human.result
const tensors = human.tf.memory().numTensors; // check current tensor usage for memory leaks
if (tensors - timestamp.tensors !== 0) log('allocated tensors:', tensors - timestamp.tensors); // printed on start and each time there is a tensor leak
timestamp.tensors = tensors;
fps.detectFPS = Math.round(1000 * 1000 / (human.now() - timestamp.detect)) / 1000;
fps.frames++;
fps.averageMs = Math.round(1000 * (human.now() - timestamp.start) / fps.frames) / 1000;
}
timestamp.detect = human.now();
requestAnimationFrame(detectionLoop); // start new frame immediately
}
function drawLoop() { // main screen refresh loop
if (!dom.video.paused && dom.video.readyState >= 2) {
const res: H.Result = dom.interpolation.checked ? human.next(human.result) : human.result; // interpolate results if enabled
let tracking: H.FaceResult[] | H.BodyResult[] | H.ObjectResult[] = [];
if (human.config.face.enabled) tracking = res.face;
else if (human.config.body.enabled) tracking = res.body;
else if (human.config.object.enabled) tracking = res.object;
else log('unknown object type');
let data: TrackerResult[] = [];
if (dom.tracker.checked) {
const items = tracking.map((obj) => ({
x: obj.box[0] + obj.box[2] / 2,
y: obj.box[1] + obj.box[3] / 2,
w: obj.box[2],
h: obj.box[3],
name: obj.label || (human.config.face.enabled ? 'face' : 'body'),
confidence: obj.score,
}));
tracker.updateTrackedItemsWithNewFrame(items, fps.frames);
data = tracker.getJSONOfTrackedItems(true) as TrackerResult[];
}
human.draw.canvas(dom.video, dom.canvas); // copy input video frame to output canvas
for (let i = 0; i < tracking.length; i++) {
// @ts-ignore
const name = tracking[i].label || (human.config.face.enabled ? 'face' : 'body');
dom.ctx.strokeRect(tracking[i].box[0], tracking[i].box[1], tracking[i].box[1], tracking[i].box[2]);
dom.ctx.fillText(`id: ${tracking[i].id} ${Math.round(100 * tracking[i].score)}% ${name}`, tracking[i].box[0] + 4, tracking[i].box[1] + 16);
if (data[i]) {
dom.ctx.fillText(`t: ${data[i].id} ${Math.round(100 * data[i].confidence)}% ${data[i].name} ${data[i].isZombie ? 'zombie' : ''}`, tracking[i].box[0] + 4, tracking[i].box[1] + 34);
}
}
}
const now = human.now();
fps.drawFPS = Math.round(1000 * 1000 / (now - timestamp.draw)) / 1000;
timestamp.draw = now;
status(dom.video.paused ? 'paused' : `fps: ${fps.detectFPS.toFixed(1).padStart(5, ' ')} detect | ${fps.drawFPS.toFixed(1).padStart(5, ' ')} draw`); // write status
setTimeout(drawLoop, 30); // use to slow down refresh from max refresh rate to target of 30 fps
}
async function handleVideo(file: File) {
const url = URL.createObjectURL(file);
dom.video.src = url;
await dom.video.play();
log('loaded video:', file.name, 'resolution:', [dom.video.videoWidth, dom.video.videoHeight], 'duration:', dom.video.duration);
dom.canvas.width = dom.video.videoWidth;
dom.canvas.height = dom.video.videoHeight;
dom.ctx.strokeStyle = 'white';
dom.ctx.fillStyle = 'white';
dom.ctx.font = '16px Segoe UI';
dom.video.playbackRate = 0.25;
}
function initInput() {
document.body.addEventListener('dragenter', (evt) => evt.preventDefault());
document.body.addEventListener('dragleave', (evt) => evt.preventDefault());
document.body.addEventListener('dragover', (evt) => evt.preventDefault());
document.body.addEventListener('drop', async (evt) => {
evt.preventDefault();
if (evt.dataTransfer) evt.dataTransfer.dropEffect = 'copy';
const file = evt.dataTransfer?.files?.[0];
if (file) await handleVideo(file);
log(dom.video.readyState);
});
(document.getElementById('inputvideo') as HTMLInputElement).onchange = async (evt) => {
evt.preventDefault();
const file = evt.target?.['files']?.[0];
if (file) await handleVideo(file);
};
dom.config.onchange = () => {
trackerConfig.distanceLimit = (document.getElementById('distanceLimit') as HTMLInputElement).valueAsNumber;
trackerConfig.iouLimit = (document.getElementById('iouLimit') as HTMLInputElement).valueAsNumber;
trackerConfig.unMatchedFramesTolerance = (document.getElementById('unMatchedFramesTolerance') as HTMLInputElement).valueAsNumber;
trackerConfig.unMatchedFramesTolerance = (document.getElementById('unMatchedFramesTolerance') as HTMLInputElement).valueAsNumber;
trackerConfig.matchingAlgorithm = (document.getElementById('matchingAlgorithm-kdTree') as HTMLInputElement).checked ? 'kdTree' : 'munkres';
tracker.setParams(trackerConfig);
if ((document.getElementById('keepInMemory') as HTMLInputElement).checked) tracker.enableKeepInMemory();
else tracker.disableKeepInMemory();
tracker.reset();
log('tracker config change', JSON.stringify(trackerConfig));
humanConfig.face!.enabled = (document.getElementById('box-face') as HTMLInputElement).checked; // eslint-disable-line @typescript-eslint/no-non-null-assertion
humanConfig.body!.enabled = (document.getElementById('box-body') as HTMLInputElement).checked; // eslint-disable-line @typescript-eslint/no-non-null-assertion
humanConfig.object!.enabled = (document.getElementById('box-object') as HTMLInputElement).checked; // eslint-disable-line @typescript-eslint/no-non-null-assertion
};
dom.tracker.onchange = (evt) => {
log('tracker', (evt.target as HTMLInputElement).checked ? 'enabled' : 'disabled');
tracker.setParams(trackerConfig);
tracker.reset();
};
}
async function main() { // main entry point
log('human version:', human.version, '| tfjs version:', human.tf.version['tfjs-core']);
log('platform:', human.env.platform, '| agent:', human.env.agent);
status('loading...');
await human.load(); // preload all models
log('backend:', human.tf.getBackend(), '| available:', human.env.backends);
log('models loaded:', human.models.loaded());
status('initializing...');
await human.warmup(); // warmup function to initialize backend for future faster detection
initInput(); // initialize input
await detectionLoop(); // start detection loop
drawLoop(); // start draw loop
}
window.onload = main;

File diff suppressed because it is too large Load Diff

View File

@ -21,10 +21,10 @@
</style> </style>
</head> </head>
<body> <body>
<canvas id="canvas" style="margin: 0 auto; width: 100vw"></canvas> <canvas id="canvas" style="margin: 0 auto; width: 100%"></canvas>
<video id="video" playsinline style="display: none"></video> <video id="video" playsinline style="display: none"></video>
<pre id="status" style="position: absolute; top: 12px; right: 20px; background-color: grey; padding: 8px; box-shadow: 2px 2px black"></pre> <pre id="status" style="position: absolute; top: 12px; right: 20px; background-color: grey; padding: 8px; box-shadow: 2px 2px black"></pre>
<pre id="log" style="padding: 8px"></pre> <pre id="log" style="padding: 8px"></pre>
<div id="performance" style="position: absolute; bottom: 0; width: 100%; padding: 8px; font-size: 0.8rem;"></div> <div id="performance" style="position: absolute; bottom: 1rem; width: 100%; padding: 8px; font-size: 0.8rem;"></div>
</body> </body>
</html> </html>

View File

@ -4,6 +4,14 @@
author: <https://github.com/vladmandic>' author: <https://github.com/vladmandic>'
*/ */
import*as m from"../../dist/human.esm.js";var v=1920,b={debug:!0,backend:"webgl",modelBasePath:"https://vladmandic.github.io/human-models/models/",filter:{enabled:!0,equalization:!1,flip:!1},face:{enabled:!0,detector:{rotation:!1},mesh:{enabled:!0},attention:{enabled:!1},iris:{enabled:!0},description:{enabled:!0},emotion:{enabled:!0},antispoof:{enabled:!0},liveness:{enabled:!0}},body:{enabled:!1},hand:{enabled:!1},object:{enabled:!1},segmentation:{enabled:!1},gesture:{enabled:!0}},e=new m.Human(b);e.env.perfadd=!1;e.draw.options.font='small-caps 18px "Lato"';e.draw.options.lineHeight=20;e.draw.options.drawPoints=!0;var a={video:document.getElementById("video"),canvas:document.getElementById("canvas"),log:document.getElementById("log"),fps:document.getElementById("status"),perf:document.getElementById("performance")},n={detect:0,draw:0,tensors:0,start:0},s={detectFPS:0,drawFPS:0,frames:0,averageMs:0},o=(...t)=>{a.log.innerText+=t.join(" ")+` import{Human as p}from"../../dist/human.esm.js";var w={modelBasePath:"../../models",filter:{enabled:!0,equalization:!1},face:{enabled:!0,detector:{rotation:!1},mesh:{enabled:!0},attention:{enabled:!1},iris:{enabled:!0},description:{enabled:!0},emotion:{enabled:!0}},body:{enabled:!0},hand:{enabled:!0},object:{enabled:!1},gesture:{enabled:!0}},t=new p(w);t.env.perfadd=!1;t.draw.options.font='small-caps 18px "Lato"';t.draw.options.lineHeight=20;var e={video:document.getElementById("video"),canvas:document.getElementById("canvas"),log:document.getElementById("log"),fps:document.getElementById("status"),perf:document.getElementById("performance")},i={detect:0,draw:0,tensors:0},d={detect:0,draw:0},s=(...a)=>{e.log.innerText+=a.join(" ")+`
`,console.log(...t)},i=t=>a.fps.innerText=t,g=t=>a.perf.innerText="tensors:"+e.tf.memory().numTensors.toString()+" | performance: "+JSON.stringify(t).replace(/"|{|}/g,"").replace(/,/g," | ");async function f(){if(!a.video.paused){n.start===0&&(n.start=e.now()),await e.detect(a.video);let t=e.tf.memory().numTensors;t-n.tensors!==0&&o("allocated tensors:",t-n.tensors),n.tensors=t,s.detectFPS=Math.round(1e3*1e3/(e.now()-n.detect))/1e3,s.frames++,s.averageMs=Math.round(1e3*(e.now()-n.start)/s.frames)/1e3,s.frames%100===0&&!a.video.paused&&o("performance",{...s,tensors:n.tensors})}n.detect=e.now(),requestAnimationFrame(f)}async function u(){var d,r,c;if(!a.video.paused){let l=e.next(e.result),w=await e.image(a.video);e.draw.canvas(w.canvas,a.canvas);let p={bodyLabels:`person confidence [score] and ${(c=(r=(d=e.result)==null?void 0:d.body)==null?void 0:r[0])==null?void 0:c.keypoints.length} keypoints`};await e.draw.all(a.canvas,l,p),g(l.performance)}let t=e.now();s.drawFPS=Math.round(1e3*1e3/(t-n.draw))/1e3,n.draw=t,i(a.video.paused?"paused":`fps: ${s.detectFPS.toFixed(1).padStart(5," ")} detect | ${s.drawFPS.toFixed(1).padStart(5," ")} draw`),setTimeout(u,30)}async function h(){let d=(await e.webcam.enumerate())[0].deviceId,r=await e.webcam.start({element:a.video,crop:!1,width:v,id:d});o(r),a.canvas.width=e.webcam.width,a.canvas.height=e.webcam.height,a.canvas.onclick=async()=>{e.webcam.paused?await e.webcam.play():e.webcam.pause()}}async function y(){o("human version:",e.version,"| tfjs version:",e.tf.version["tfjs-core"]),o("platform:",e.env.platform,"| agent:",e.env.agent),i("loading..."),await e.load(),o("backend:",e.tf.getBackend(),"| available:",e.env.backends),o("models stats:",e.models.stats()),o("models loaded:",e.models.loaded()),o("environment",e.env),i("initializing..."),await e.warmup(),await h(),await f(),await u()}window.onload=y; `,console.log(...a)},r=a=>e.fps.innerText=a,b=a=>e.perf.innerText="tensors:"+t.tf.memory().numTensors+" | performance: "+JSON.stringify(a).replace(/"|{|}/g,"").replace(/,/g," | ");async function h(){r("starting webcam...");let a={audio:!1,video:{facingMode:"user",resizeMode:"none",width:{ideal:document.body.clientWidth},height:{ideal:document.body.clientHeight}}},n=await navigator.mediaDevices.getUserMedia(a),m=new Promise(f=>{e.video.onloadeddata=()=>f(!0)});e.video.srcObject=n,e.video.play(),await m,e.canvas.width=e.video.videoWidth,e.canvas.height=e.video.videoHeight;let o=n.getVideoTracks()[0],g=o.getCapabilities?o.getCapabilities():"",v=o.getSettings?o.getSettings():"",u=o.getConstraints?o.getConstraints():"";s("video:",e.video.videoWidth,e.video.videoHeight,o.label,{stream:n,track:o,settings:v,constraints:u,capabilities:g}),e.canvas.onclick=()=>{e.video.paused?e.video.play():e.video.pause()}}async function c(){if(!e.video.paused){await t.detect(e.video);let n=t.tf.memory().numTensors;n-i.tensors!==0&&s("allocated tensors:",n-i.tensors),i.tensors=n}let a=t.now();d.detect=1e3/(a-i.detect),i.detect=a,requestAnimationFrame(c)}async function l(){if(!e.video.paused){let n=await t.next(t.result);await t.draw.canvas(e.video,e.canvas),await t.draw.all(e.canvas,n),console.log(e.canvas.width,e.canvas.height),b(n.performance)}let a=t.now();d.draw=1e3/(a-i.draw),i.draw=a,r(e.video.paused?"paused":`fps: ${d.detect.toFixed(1).padStart(5," ")} detect | ${d.draw.toFixed(1).padStart(5," ")} draw`),setTimeout(l,30)}async function y(){s("human version:",t.version,"| tfjs version:",t.tf.version["tfjs-core"]),s("platform:",t.env.platform,"| agent:",t.env.agent),r("loading..."),await t.load(),s("backend:",t.tf.getBackend(),"| available:",t.env.backends),s("loaded models:",Object.values(t.models).filter(a=>a!==null).length),r("initializing..."),await t.warmup(),await h(),await c(),await l()}window.onload=y;
/**
* Human demo for browsers
* @default Human Library
* @summary <https://github.com/vladmandic/human>
* @author <https://github.com/vladmandic>
* @copyright <https://github.com/vladmandic>
* @license MIT
*/
//# sourceMappingURL=index.js.map //# sourceMappingURL=index.js.map

File diff suppressed because one or more lines are too long

View File

@ -7,34 +7,26 @@
* @license MIT * @license MIT
*/ */
import * as H from '../../dist/human.esm.js'; // equivalent of @vladmandic/Human import { Human, Config } from '../../dist/human.esm.js'; // equivalent of @vladmandic/Human
const width = 1920; // used by webcam config as well as human maximum resultion // can be anything, but resolutions higher than 4k will disable internal optimizations const humanConfig: Partial<Config> = { // user configuration for human, used to fine-tune behavior
// backend: 'webgpu' as const,
const humanConfig: Partial<H.Config> = { // user configuration for human, used to fine-tune behavior // async: true,
debug: true, modelBasePath: '../../models',
backend: 'webgl', filter: { enabled: true, equalization: false },
// cacheSensitivity: 0, // cacheSensitivity: 0,
// cacheModels: false, face: { enabled: true, detector: { rotation: false }, mesh: { enabled: true }, attention: { enabled: false }, iris: { enabled: true }, description: { enabled: true }, emotion: { enabled: true } },
// warmup: 'none', body: { enabled: true },
// modelBasePath: '../../models', hand: { enabled: true },
modelBasePath: 'https://vladmandic.github.io/human-models/models/',
filter: { enabled: true, equalization: false, flip: false },
face: { enabled: true, detector: { rotation: false }, mesh: { enabled: true }, attention: { enabled: false }, iris: { enabled: true }, description: { enabled: true }, emotion: { enabled: true }, antispoof: { enabled: true }, liveness: { enabled: true } },
body: { enabled: false },
hand: { enabled: false },
object: { enabled: false }, object: { enabled: false },
segmentation: { enabled: false },
gesture: { enabled: true }, gesture: { enabled: true },
}; };
const human = new H.Human(humanConfig); // create instance of human with overrides from user configuration const human = new Human(humanConfig); // create instance of human with overrides from user configuration
human.env.perfadd = false; // is performance data showing instant or total values human.env['perfadd'] = false; // is performance data showing instant or total values
human.draw.options.font = 'small-caps 18px "Lato"'; // set font used to draw labels when using draw methods human.draw.options.font = 'small-caps 18px "Lato"'; // set font used to draw labels when using draw methods
human.draw.options.lineHeight = 20; human.draw.options.lineHeight = 20;
human.draw.options.drawPoints = true; // draw points on face mesh
// human.draw.options.fillPolygons = true;
const dom = { // grab instances of dom objects so we dont have to look them up later const dom = { // grab instances of dom objects so we dont have to look them up later
video: document.getElementById('video') as HTMLVideoElement, video: document.getElementById('video') as HTMLVideoElement,
@ -43,72 +35,76 @@ const dom = { // grab instances of dom objects so we dont have to look them up l
fps: document.getElementById('status') as HTMLPreElement, fps: document.getElementById('status') as HTMLPreElement,
perf: document.getElementById('performance') as HTMLDivElement, perf: document.getElementById('performance') as HTMLDivElement,
}; };
const timestamp = { detect: 0, draw: 0, tensors: 0, start: 0 }; // holds information used to calculate performance and possible memory leaks const timestamp = { detect: 0, draw: 0, tensors: 0 }; // holds information used to calculate performance and possible memory leaks
const fps = { detectFPS: 0, drawFPS: 0, frames: 0, averageMs: 0 }; // holds calculated fps information for both detect and screen refresh const fps = { detect: 0, draw: 0 }; // holds calculated fps information for both detect and screen refresh
const log = (...msg) => { // helper method to output messages const log = (...msg) => { // helper method to output messages
dom.log.innerText += msg.join(' ') + '\n'; dom.log.innerText += msg.join(' ') + '\n';
console.log(...msg); // eslint-disable-line no-console // eslint-disable-next-line no-console
console.log(...msg);
}; };
const status = (msg) => dom.fps.innerText = msg; // print status element const status = (msg) => dom.fps.innerText = msg; // print status element
const perf = (msg) => dom.perf.innerText = 'tensors:' + human.tf.memory().numTensors.toString() + ' | performance: ' + JSON.stringify(msg).replace(/"|{|}/g, '').replace(/,/g, ' | '); // print performance element const perf = (msg) => dom.perf.innerText = 'tensors:' + human.tf.memory().numTensors + ' | performance: ' + JSON.stringify(msg).replace(/"|{|}/g, '').replace(/,/g, ' | '); // print performance element
async function webCam() { // initialize webcam
status('starting webcam...');
// @ts-ignore resizeMode is not yet defined in tslib
const options: MediaStreamConstraints = { audio: false, video: { facingMode: 'user', resizeMode: 'none', width: { ideal: document.body.clientWidth }, height: { ideal: document.body.clientHeight } } };
const stream: MediaStream = await navigator.mediaDevices.getUserMedia(options);
const ready = new Promise((resolve) => { dom.video.onloadeddata = () => resolve(true); });
dom.video.srcObject = stream;
dom.video.play();
await ready;
dom.canvas.width = dom.video.videoWidth;
dom.canvas.height = dom.video.videoHeight;
const track: MediaStreamTrack = stream.getVideoTracks()[0];
const capabilities: MediaTrackCapabilities | string = track.getCapabilities ? track.getCapabilities() : '';
const settings: MediaTrackSettings | string = track.getSettings ? track.getSettings() : '';
const constraints: MediaTrackConstraints | string = track.getConstraints ? track.getConstraints() : '';
log('video:', dom.video.videoWidth, dom.video.videoHeight, track.label, { stream, track, settings, constraints, capabilities });
dom.canvas.onclick = () => { // pause when clicked on screen and resume on next click
if (dom.video.paused) dom.video.play();
else dom.video.pause();
};
}
async function detectionLoop() { // main detection loop async function detectionLoop() { // main detection loop
if (!dom.video.paused) { if (!dom.video.paused) {
if (timestamp.start === 0) timestamp.start = human.now(); // console.log('profiling data:', await human.profile(dom.video));
// log('profiling data:', await human.profile(dom.video));
await human.detect(dom.video); // actual detection; were not capturing output in a local variable as it can also be reached via human.result await human.detect(dom.video); // actual detection; were not capturing output in a local variable as it can also be reached via human.result
const tensors = human.tf.memory().numTensors; // check current tensor usage for memory leaks const tensors = human.tf.memory().numTensors; // check current tensor usage for memory leaks
if (tensors - timestamp.tensors !== 0) log('allocated tensors:', tensors - timestamp.tensors); // printed on start and each time there is a tensor leak if (tensors - timestamp.tensors !== 0) log('allocated tensors:', tensors - timestamp.tensors); // printed on start and each time there is a tensor leak
timestamp.tensors = tensors; timestamp.tensors = tensors;
fps.detectFPS = Math.round(1000 * 1000 / (human.now() - timestamp.detect)) / 1000;
fps.frames++;
fps.averageMs = Math.round(1000 * (human.now() - timestamp.start) / fps.frames) / 1000;
if (fps.frames % 100 === 0 && !dom.video.paused) log('performance', { ...fps, tensors: timestamp.tensors });
} }
timestamp.detect = human.now(); const now = human.now();
fps.detect = 1000 / (now - timestamp.detect);
timestamp.detect = now;
requestAnimationFrame(detectionLoop); // start new frame immediately requestAnimationFrame(detectionLoop); // start new frame immediately
} }
async function drawLoop() { // main screen refresh loop async function drawLoop() { // main screen refresh loop
if (!dom.video.paused) { if (!dom.video.paused) {
const interpolated = human.next(human.result); // smoothen result using last-known results const interpolated = await human.next(human.result); // smoothen result using last-known results
const processed = await human.image(dom.video); // get current video frame, but enhanced with human.filters await human.draw.canvas(dom.video, dom.canvas); // draw canvas to screen
human.draw.canvas(processed.canvas as HTMLCanvasElement, dom.canvas); await human.draw.all(dom.canvas, interpolated); // draw labels, boxes, lines, etc.
console.log(dom.canvas.width, dom.canvas.height);
const opt: Partial<H.DrawOptions> = { bodyLabels: `person confidence [score] and ${human.result?.body?.[0]?.keypoints.length} keypoints` };
await human.draw.all(dom.canvas, interpolated, opt); // draw labels, boxes, lines, etc.
perf(interpolated.performance); // write performance data perf(interpolated.performance); // write performance data
} }
const now = human.now(); const now = human.now();
fps.drawFPS = Math.round(1000 * 1000 / (now - timestamp.draw)) / 1000; fps.draw = 1000 / (now - timestamp.draw);
timestamp.draw = now; timestamp.draw = now;
status(dom.video.paused ? 'paused' : `fps: ${fps.detectFPS.toFixed(1).padStart(5, ' ')} detect | ${fps.drawFPS.toFixed(1).padStart(5, ' ')} draw`); // write status status(dom.video.paused ? 'paused' : `fps: ${fps.detect.toFixed(1).padStart(5, ' ')} detect | ${fps.draw.toFixed(1).padStart(5, ' ')} draw`); // write status
// requestAnimationFrame(drawLoop); // refresh at screen refresh rate
setTimeout(drawLoop, 30); // use to slow down refresh from max refresh rate to target of 30 fps setTimeout(drawLoop, 30); // use to slow down refresh from max refresh rate to target of 30 fps
} }
async function webCam() {
const devices = await human.webcam.enumerate();
const id = devices[0].deviceId; // use first available video source
const webcamStatus = await human.webcam.start({ element: dom.video, crop: false, width, id }); // use human webcam helper methods and associate webcam stream with a dom element
log(webcamStatus);
dom.canvas.width = human.webcam.width;
dom.canvas.height = human.webcam.height;
dom.canvas.onclick = async () => { // pause when clicked on screen and resume on next click
if (human.webcam.paused) await human.webcam.play();
else human.webcam.pause();
};
}
async function main() { // main entry point async function main() { // main entry point
log('human version:', human.version, '| tfjs version:', human.tf.version['tfjs-core']); log('human version:', human.version, '| tfjs version:', human.tf.version['tfjs-core']);
log('platform:', human.env.platform, '| agent:', human.env.agent); log('platform:', human.env.platform, '| agent:', human.env.agent);
status('loading...'); status('loading...');
await human.load(); // preload all models await human.load(); // preload all models
log('backend:', human.tf.getBackend(), '| available:', human.env.backends); log('backend:', human.tf.getBackend(), '| available:', human.env.backends);
log('models stats:', human.models.stats()); log('loaded models:', Object.values(human.models).filter((model) => model !== null).length);
log('models loaded:', human.models.loaded());
log('environment', human.env);
status('initializing...'); status('initializing...');
await human.warmup(); // warmup function to initialize backend for future faster detection await human.warmup(); // warmup function to initialize backend for future faster detection
await webCam(); // start webcam await webCam(); // start webcam

View File

@ -1,58 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Human</title>
<meta name="viewport" content="width=device-width" id="viewport">
<meta name="keywords" content="Human">
<meta name="description" content="Human: Demo; Author: Vladimir Mandic <https://github.com/vladmandic>">
<link rel="manifest" href="../manifest.webmanifest">
<link rel="shortcut icon" href="../../favicon.ico" type="image/x-icon">
<style>
@font-face { font-family: 'Lato'; font-display: swap; font-style: normal; font-weight: 100; src: local('Lato'), url('../../assets/lato-light.woff2') }
body { font-family: 'Lato', 'Segoe UI'; font-size: 16px; font-variant: small-caps; margin: 0; background: black; color: white; overflow: hidden; width: 100vw; height: 100vh; }
</style>
</head>
<body>
<canvas id="canvas" style="margin: 0 auto; width: 100%"></canvas>
<pre id="log" style="padding: 8px; position: fixed; bottom: 0"></pre>
<script type="module">
import * as H from '../../dist/human.esm.js'; // equivalent of import @vladmandic/Human
const humanConfig = { // user configuration for human, used to fine-tune behavior
modelBasePath: '../../models', // models can be loaded directly from cdn as well
filter: { enabled: true, equalization: true, flip: false },
face: { enabled: true, detector: { rotation: false }, mesh: { enabled: true }, attention: { enabled: false }, iris: { enabled: true }, description: { enabled: true }, emotion: { enabled: true } },
body: { enabled: true },
hand: { enabled: true },
gesture: { enabled: true },
object: { enabled: false },
segmentation: { enabled: false },
};
const human = new H.Human(humanConfig); // create instance of human with overrides from user configuration
const canvas = document.getElementById('canvas'); // output canvas to draw both webcam and detection results
async function drawLoop() { // main screen refresh loop
const interpolated = human.next(); // get smoothened result using last-known results which are continously updated based on input webcam video
human.draw.canvas(human.webcam.element, canvas); // draw webcam video to screen canvas // better than using procesed image as this loop happens faster than processing loop
await human.draw.all(canvas, interpolated); // draw labels, boxes, lines, etc.
setTimeout(drawLoop, 30); // use to slow down refresh from max refresh rate to target of 1000/30 ~ 30 fps
}
async function main() { // main entry point
document.getElementById('log').innerHTML = `human version: ${human.version} | tfjs version: ${human.tf.version['tfjs-core']}<br>platform: ${human.env.platform} | agent ${human.env.agent}`;
await human.webcam.start({ crop: true }); // find webcam and start it
human.video(human.webcam.element); // instruct human to continously detect video frames
canvas.width = human.webcam.width; // set canvas resolution to input webcam native resolution
canvas.height = human.webcam.height;
canvas.onclick = async () => { // pause when clicked on screen and resume on next click
if (human.webcam.paused) await human.webcam.play();
else human.webcam.pause();
};
await drawLoop(); // start draw loop
}
window.onload = main;
</script>
</body>
</html>

2537
dist/human.d.ts vendored

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

7
dist/human.esm-nobundle.js.map vendored Normal file

File diff suppressed because one or more lines are too long

2537
dist/human.esm.d.ts vendored

File diff suppressed because it is too large Load Diff

61315
dist/human.esm.js vendored

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

6353
dist/human.js vendored

File diff suppressed because one or more lines are too long

2537
dist/human.node-gpu.d.ts vendored

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

2537
dist/human.node.d.ts vendored

File diff suppressed because it is too large Load Diff

44
dist/human.node.js vendored

File diff suppressed because one or more lines are too long

25
dist/tfjs.esm.d.ts vendored
View File

@ -1,25 +0,0 @@
/* eslint-disable import/no-unresolved */
/* eslint-disable import/no-extraneous-dependencies */
export * from 'types/tfjs.esm';
export declare const version: {
'tfjs-core': string;
'tfjs-backend-cpu': string;
'tfjs-backend-webgl': string;
'tfjs-data': string;
'tfjs-layers': string;
'tfjs-converter': string;
tfjs: string;
};
export * from '@tensorflow/tfjs-core';
export * from '@tensorflow/tfjs-converter';
export * from '@tensorflow/tfjs-data';
export * from '@tensorflow/tfjs-layers';
export * from '@tensorflow/tfjs-backend-cpu';
export * from '@tensorflow/tfjs-backend-wasm';
export * from '@tensorflow/tfjs-backend-webgl';
export * from '@tensorflow/tfjs-backend-webgpu';
export * from '@tensorflow/tfjs-node';
export * from '@tensorflow/tfjs-node-gpu';

6945
dist/tfjs.esm.js vendored

File diff suppressed because one or more lines are too long

View File

@ -4,4 +4,4 @@
author: <https://github.com/vladmandic>' author: <https://github.com/vladmandic>'
*/ */
var e="4.22.0";var s="4.22.0";var t="4.22.0";var n="4.22.0";var r="4.22.0";var i="4.22.0";var h={tfjs:e,"tfjs-core":e,"tfjs-converter":s,"tfjs-backend-cpu":t,"tfjs-backend-webgl":n,"tfjs-backend-wasm":r,"tfjs-backend-webgpu":i};export{h as version}; var e="3.16.0";var s="3.16.0";var t="3.16.0";var r="3.16.0";var l="3.16.0";var i="3.16.0";var a="3.16.0";var V={tfjs:e,"tfjs-core":s,"tfjs-data":t,"tfjs-layers":r,"tfjs-converter":l,"tfjs-backend-webgl":i,"tfjs-backend-wasm":a};export{V as version};

View File

@ -5,7 +5,7 @@ After=network.target network-online.target
[Service] [Service]
Type=simple Type=simple
Environment="NODE_ENV=production" Environment="NODE_ENV=production"
ExecStart=<path-to-node> <your-project-folder>/node_modules/@vladmandic/build/src/build.js --profile serve ExecStart=<path-to-node> <your-project-folder>/node_modules/@vladmandic/build/src/build.js --profile development
WorkingDirectory=<your-project-folder> WorkingDirectory=<your-project-folder>
StandardOutput=inherit StandardOutput=inherit
StandardError=inherit StandardError=inherit

View File

@ -3,3 +3,308 @@
For details see Wiki: For details see Wiki:
- [**List of Models & Credits**](https://github.com/vladmandic/human/wiki/Models) - [**List of Models & Credits**](https://github.com/vladmandic/human/wiki/Models)
## Model signatures:
```js
INFO: graph model: /home/vlado/dev/human/models/iris.json
INFO: created on: 2020-10-12T18:46:47.060Z
INFO: metadata: { generatedBy: 'https://github.com/google/mediapipe', convertedBy: 'https://github.com/vladmandic', version: undefined }
INFO: model inputs based on signature
{ name: 'input_1:0', dtype: 'DT_FLOAT', shape: [ -1, 64, 64, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'Identity:0', dytpe: 'DT_FLOAT', shape: [ -1, 1, 1, 228 ] }
INFO: tensors: 191
DATA: weights: {
files: [ 'iris.bin' ],
size: { disk: 2599092, memory: 2599092 },
count: { total: 191, float32: 189, int32: 2 },
quantized: { none: 191 },
values: { total: 649773, float32: 649764, int32: 9 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity' ],
convolution: [ '_FusedConv2D', 'DepthwiseConv2dNative', 'MaxPool' ],
arithmetic: [ 'AddV2' ],
basic_math: [ 'Prelu' ],
transformation: [ 'Pad' ],
slice_join: [ 'ConcatV2' ]
}
INFO: graph model: /home/vlado/dev/human/models/facemesh.json
INFO: created on: 2020-10-12T18:46:46.944Z
INFO: metadata: { generatedBy: 'https://github.com/google/mediapipe', convertedBy: 'https://github.com/vladmandic', version: undefined }
INFO: model inputs based on signature
{ name: 'input_1:0', dtype: 'DT_FLOAT', shape: [ 1, 192, 192, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'Identity_1:0', dytpe: 'DT_FLOAT', shape: [ 1, 266 ] }
{ id: 1, name: 'Identity_2:0', dytpe: 'DT_FLOAT', shape: [ 1, 1 ] }
{ id: 2, name: 'Identity:0', dytpe: 'DT_FLOAT', shape: [ 1, 1404 ] }
INFO: tensors: 118
DATA: weights: {
files: [ 'facemesh.bin' ],
size: { disk: 2955780, memory: 2955780 },
count: { total: 118, float32: 114, int32: 4 },
quantized: { none: 118 },
values: { total: 738945, float32: 738919, int32: 26 }
}
DATA: kernel ops: {
graph: [ 'Placeholder', 'Const', 'NoOp', 'Identity' ],
convolution: [ '_FusedConv2D', 'DepthwiseConv2dNative', 'MaxPool' ],
arithmetic: [ 'AddV2' ],
basic_math: [ 'Prelu', 'Sigmoid' ],
transformation: [ 'Pad', 'Reshape' ]
}
INFO: graph model: /home/vlado/dev/human/models/emotion.json
INFO: created on: 2020-11-05T20:11:29.740Z
INFO: metadata: { generatedBy: 'https://github.com/oarriaga/face_classification', convertedBy: 'https://github.com/vladmandic', version: undefined }
INFO: model inputs based on signature
{ name: 'input_1:0', dtype: 'DT_FLOAT', shape: [ -1, 64, 64, 1 ] }
INFO: model outputs based on signature
{ id: 0, name: 'Identity:0', dytpe: 'DT_FLOAT', shape: [ -1, 7 ] }
INFO: tensors: 23
DATA: weights: {
files: [ 'emotion.bin' ],
size: { disk: 820516, memory: 820516 },
count: { total: 23, float32: 22, int32: 1 },
quantized: { none: 23 },
values: { total: 205129, float32: 205127, int32: 2 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity' ],
convolution: [ '_FusedConv2D', 'DepthwiseConv2dNative', 'MaxPool' ],
arithmetic: [ 'AddV2' ],
basic_math: [ 'Relu' ],
reduction: [ 'Mean' ],
normalization: [ 'Softmax' ]
}
INFO: graph model: /home/vlado/dev/human/models/faceres.json
INFO: created on: 2021-03-21T14:12:59.863Z
INFO: metadata: { generatedBy: 'https://github.com/HSE-asavchenko/HSE_FaceRec_tf', convertedBy: 'https://github.com/vladmandic', version: undefined }
INFO: model inputs based on signature
{ name: 'input_1', dtype: 'DT_FLOAT', shape: [ -1, 224, 224, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'gender_pred/Sigmoid:0', dytpe: 'DT_FLOAT', shape: [ 1, 1 ] }
{ id: 1, name: 'global_pooling/Mean', dytpe: 'DT_FLOAT', shape: [ 1, 1024 ] }
{ id: 2, name: 'age_pred/Softmax:0', dytpe: 'DT_FLOAT', shape: [ 1, 100 ] }
INFO: tensors: 128
DATA: weights: {
files: [ 'faceres.bin' ],
size: { disk: 6978814, memory: 13957620 },
count: { total: 128, float32: 127, int32: 1 },
quantized: { float16: 127, none: 1 },
values: { total: 3489405, float32: 3489403, int32: 2 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder' ],
convolution: [ 'Conv2D', 'DepthwiseConv2dNative' ],
arithmetic: [ 'Add', 'Minimum', 'Maximum', 'Mul' ],
basic_math: [ 'Relu', 'Sigmoid' ],
reduction: [ 'Mean' ],
matrices: [ '_FusedMatMul' ],
normalization: [ 'Softmax' ]
}
INFO: graph model: /home/vlado/dev/human/models/blazeface.json
INFO: created on: 2020-10-15T19:57:26.419Z
INFO: metadata: { generatedBy: 'https://github.com/google/mediapipe', convertedBy: 'https://github.com/vladmandic', version: undefined }
INFO: model inputs based on signature
{ name: 'input:0', dtype: 'DT_FLOAT', shape: [ 1, 256, 256, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'Identity_3:0', dytpe: 'DT_FLOAT', shape: [ 1, 384, 16 ] }
{ id: 1, name: 'Identity:0', dytpe: 'DT_FLOAT', shape: [ 1, 512, 1 ] }
{ id: 2, name: 'Identity_1:0', dytpe: 'DT_FLOAT', shape: [ 1, 384, 1 ] }
{ id: 3, name: 'Identity_2:0', dytpe: 'DT_FLOAT', shape: [ 1, 512, 16 ] }
INFO: tensors: 112
DATA: weights: {
files: [ 'blazeface.bin' ],
size: { disk: 538928, memory: 538928 },
count: { total: 112, float32: 106, int32: 6 },
quantized: { none: 112 },
values: { total: 134732, float32: 134704, int32: 28 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity' ],
convolution: [ '_FusedConv2D', 'DepthwiseConv2dNative', 'MaxPool' ],
arithmetic: [ 'AddV2' ],
basic_math: [ 'Relu' ],
transformation: [ 'Pad', 'Reshape' ]
}
INFO: graph model: /home/vlado/dev/human/models/mb3-centernet.json
INFO: created on: 2021-05-19T11:50:13.013Z
INFO: metadata: { generatedBy: 'https://github.com/610265158/mobilenetv3_centernet', convertedBy: 'https://github.com/vladmandic', version: undefined }
INFO: model inputs based on signature
{ name: 'tower_0/images', dtype: 'DT_FLOAT', shape: [ 1, 512, 512, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'tower_0/wh', dytpe: 'DT_FLOAT', shape: [ 1, 128, 128, 4 ] }
{ id: 1, name: 'tower_0/keypoints', dytpe: 'DT_FLOAT', shape: [ 1, 128, 128, 80 ] }
{ id: 2, name: 'tower_0/detections', dytpe: 'DT_FLOAT', shape: [ 1, 100, 6 ] }
INFO: tensors: 267
DATA: weights: {
files: [ 'mb3-centernet.bin' ],
size: { disk: 4030290, memory: 8060260 },
count: { total: 267, float32: 227, int32: 40 },
quantized: { float16: 227, none: 40 },
values: { total: 2015065, float32: 2014985, int32: 80 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity' ],
convolution: [ '_FusedConv2D', 'FusedDepthwiseConv2dNative', 'DepthwiseConv2dNative', 'Conv2D', 'MaxPool' ],
arithmetic: [ 'Mul', 'Add', 'FloorDiv', 'FloorMod', 'Sub' ],
basic_math: [ 'Relu6', 'Relu', 'Sigmoid' ],
reduction: [ 'Mean' ],
image: [ 'ResizeBilinear' ],
slice_join: [ 'ConcatV2', 'GatherV2', 'StridedSlice' ],
transformation: [ 'Reshape', 'Cast', 'ExpandDims' ],
logical: [ 'Equal' ],
evaluation: [ 'TopKV2' ]
}
INFO: graph model: /home/vlado/dev/human/models/movenet-lightning.json
INFO: created on: 2021-05-29T12:26:32.994Z
INFO: metadata: { generatedBy: 'https://tfhub.dev/google/movenet/singlepose/lightning/4', convertedBy: 'https://github.com/vladmandic', version: undefined }
INFO: model inputs based on signature
{ name: 'input:0', dtype: 'DT_INT32', shape: [ 1, 192, 192, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'Identity:0', dytpe: 'DT_FLOAT', shape: [ 1, 1, 17, 3 ] }
INFO: tensors: 180
DATA: weights: {
files: [ 'movenet-lightning.bin' ],
size: { disk: 4650216, memory: 9300008 },
count: { total: 180, int32: 31, float32: 149 },
quantized: { none: 31, float16: 149 },
values: { total: 2325002, int32: 106, float32: 2324896 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity' ],
transformation: [ 'Cast', 'ExpandDims', 'Squeeze', 'Reshape' ],
slice_join: [ 'Unpack', 'Pack', 'GatherNd', 'ConcatV2' ],
arithmetic: [ 'Sub', 'Mul', 'AddV2', 'FloorDiv', 'SquaredDifference', 'RealDiv' ],
convolution: [ '_FusedConv2D', 'FusedDepthwiseConv2dNative', 'DepthwiseConv2dNative' ],
image: [ 'ResizeBilinear' ],
basic_math: [ 'Sigmoid', 'Sqrt' ],
reduction: [ 'ArgMax' ]
}
INFO: graph model: /home/vlado/dev/human/models/selfie.json
INFO: created on: 2021-06-04T13:46:56.904Z
INFO: metadata: { generatedBy: 'https://github.com/PINTO0309/PINTO_model_zoo/tree/main/109_Selfie_Segmentation', convertedBy: 'https://github.com/vladmandic', version: '561.undefined' }
INFO: model inputs based on signature
{ name: 'input_1:0', dtype: 'DT_FLOAT', shape: [ 1, 256, 256, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'activation_10:0', dytpe: 'DT_FLOAT', shape: [ 1, 256, 256, 1 ] }
INFO: tensors: 136
DATA: weights: {
files: [ 'selfie.bin' ],
size: { disk: 212886, memory: 425732 },
count: { total: 136, int32: 4, float32: 132 },
quantized: { none: 4, float16: 132 },
values: { total: 106433, int32: 10, float32: 106423 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder' ],
convolution: [ 'Conv2D', 'DepthwiseConv2dNative', 'AvgPool', 'Conv2DBackpropInput' ],
arithmetic: [ 'Add', 'Mul', 'AddV2', 'AddN' ],
basic_math: [ 'Relu6', 'Relu', 'Sigmoid' ],
image: [ 'ResizeBilinear' ]
}
INFO: graph model: /home/vlado/dev/human/models/handtrack.json
INFO: created on: 2021-09-21T12:09:47.583Z
INFO: metadata: { generatedBy: 'https://github.com/victordibia/handtracking', convertedBy: 'https://github.com/vladmandic', version: '561.undefined' }
INFO: model inputs based on signature
{ name: 'input_tensor:0', dtype: 'DT_UINT8', shape: [ 1, 320, 320, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'Identity_2:0', dytpe: 'DT_FLOAT', shape: [ 1, 100 ] }
{ id: 1, name: 'Identity_4:0', dytpe: 'DT_FLOAT', shape: [ 1, 100 ] }
{ id: 2, name: 'Identity_6:0', dytpe: 'DT_FLOAT', shape: [ 1, 12804, 4 ] }
{ id: 3, name: 'Identity_1:0', dytpe: 'DT_FLOAT', shape: [ 1, 100, 4 ] }
{ id: 4, name: 'Identity_3:0', dytpe: 'DT_FLOAT', shape: [ 1, 100, 8 ] }
{ id: 5, name: 'Identity_5:0', dytpe: 'DT_FLOAT', shape: [ 1 ] }
{ id: 6, name: 'Identity:0', dytpe: 'DT_FLOAT', shape: [ 1, 100 ] }
{ id: 7, name: 'Identity_7:0', dytpe: 'DT_FLOAT', shape: [ 1, 12804, 8 ] }
INFO: tensors: 619
DATA: weights: {
files: [ 'handtrack.bin' ],
size: { disk: 2964837, memory: 11846016 },
count: { total: 619, int32: 347, float32: 272 },
quantized: { none: 347, uint8: 272 },
values: { total: 2961504, int32: 1111, float32: 2960393 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity', 'Shape', 'NoOp' ],
control: [ 'TensorListReserve', 'Enter', 'TensorListFromTensor', 'Merge', 'LoopCond', 'Switch', 'Exit', 'TensorListStack', 'NextIteration', 'TensorListSetItem', 'TensorListGetItem' ],
logical: [ 'Less', 'LogicalAnd', 'Select', 'Greater', 'GreaterEqual' ],
convolution: [ '_FusedConv2D', 'FusedDepthwiseConv2dNative', 'DepthwiseConv2dNative' ],
arithmetic: [ 'AddV2', 'Mul', 'Sub', 'Minimum', 'Maximum' ],
transformation: [ 'Cast', 'ExpandDims', 'Squeeze', 'Reshape', 'Pad' ],
slice_join: [ 'Unpack', 'StridedSlice', 'Pack', 'ConcatV2', 'Slice', 'GatherV2', 'Split' ],
image: [ 'ResizeBilinear' ],
basic_math: [ 'Reciprocal', 'Sigmoid', 'Exp' ],
matrices: [ 'Transpose' ],
dynamic: [ 'NonMaxSuppressionV5', 'Where' ],
creation: [ 'Fill', 'Range' ],
evaluation: [ 'TopKV2' ],
reduction: [ 'Sum' ]
}
INFO: graph model: /home/vlado/dev/human/models/antispoof.json
INFO: created on: 2021-10-13T14:20:27.100Z
INFO: metadata: { generatedBy: 'https://www.kaggle.com/anku420/fake-face-detection', convertedBy: 'https://github.com/vladmandic', version: '716.undefined' }
INFO: model inputs based on signature
{ name: 'conv2d_input', dtype: 'DT_FLOAT', shape: [ -1, 128, 128, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'activation_4', dytpe: 'DT_FLOAT', shape: [ -1, 1 ] }
INFO: tensors: 11
DATA: weights: {
files: [ 'antispoof.bin' ],
size: { disk: 853098, memory: 1706188 },
count: { total: 11, float32: 10, int32: 1 },
quantized: { float16: 10, none: 1 },
values: { total: 426547, float32: 426545, int32: 2 }
}
DATA: kernel ops: { graph: [ 'Const', 'Placeholder', 'Identity' ], convolution: [ '_FusedConv2D', 'MaxPool' ], basic_math: [ 'Relu', 'Sigmoid' ], transformation: [ 'Reshape' ], matrices: [ '_FusedMatMul' ] }
INFO: graph model: /home/vlado/dev/human/models/handlandmark-full.json
INFO: created on: 2021-10-31T12:27:49.343Z
INFO: metadata: { generatedBy: 'https://github.com/google/mediapipe', convertedBy: 'https://github.com/vladmandic', version: '808.undefined' }
INFO: model inputs based on signature
{ name: 'input_1', dtype: 'DT_FLOAT', shape: [ 1, 224, 224, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'Identity_3:0', dytpe: 'DT_FLOAT', shape: [ 1, 63 ] }
{ id: 1, name: 'Identity:0', dytpe: 'DT_FLOAT', shape: [ 1, 63 ] }
{ id: 2, name: 'Identity_1:0', dytpe: 'DT_FLOAT', shape: [ 1, 1 ] }
{ id: 3, name: 'Identity_2:0', dytpe: 'DT_FLOAT', shape: [ 1, 1 ] }
INFO: tensors: 103
DATA: weights: {
files: [ 'handlandmark-full.bin' ],
size: { disk: 5431368, memory: 10862728 },
count: { total: 103, float32: 102, int32: 1 },
quantized: { float16: 102, none: 1 },
values: { total: 2715682, float32: 2715680, int32: 2 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity' ],
convolution: [ 'Conv2D', 'DepthwiseConv2dNative' ],
arithmetic: [ 'AddV2', 'AddN' ],
basic_math: [ 'Relu6', 'Sigmoid' ],
reduction: [ 'Mean' ],
matrices: [ '_FusedMatMul' ]
}
INFO: graph model: /home/vlado/dev/human/models/liveness.json
INFO: created on: 2021-11-09T12:39:11.760Z
INFO: metadata: { generatedBy: 'https://github.com/leokwu/livenessnet', convertedBy: 'https://github.com/vladmandic', version: '808.undefined' }
INFO: model inputs based on signature
{ name: 'conv2d_1_input', dtype: 'DT_FLOAT', shape: [ -1, 32, 32, 3 ] }
INFO: model outputs based on signature
{ id: 0, name: 'activation_6', dytpe: 'DT_FLOAT', shape: [ -1, 2 ] }
INFO: tensors: 23
DATA: weights: {
files: [ 'liveness.bin' ],
size: { disk: 592976, memory: 592976 },
count: { total: 23, float32: 22, int32: 1 },
quantized: { none: 23 },
values: { total: 148244, float32: 148242, int32: 2 }
}
DATA: kernel ops: {
graph: [ 'Const', 'Placeholder', 'Identity' ],
convolution: [ '_FusedConv2D', 'MaxPool' ],
arithmetic: [ 'Mul', 'Add', 'AddV2' ],
transformation: [ 'Reshape' ],
matrices: [ '_FusedMatMul' ],
normalization: [ 'Softmax' ]
}
```

Binary file not shown.

File diff suppressed because one or more lines are too long

Binary file not shown.

File diff suppressed because one or more lines are too long

Binary file not shown.

File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More