mirror of https://github.com/vladmandic/human
add notes on building custom modules
parent
de0142bcee
commit
bffa2cb040
1
Home.md
1
Home.md
|
@ -53,6 +53,7 @@ Check out [**Live Demo**](https://vladmandic.github.io/human/demo/index.html) fo
|
|||
- [**Notes on Backends**](https://github.com/vladmandic/human/wiki/Backends)
|
||||
- [**Development Server**](https://github.com/vladmandic/human/wiki/Development-Server)
|
||||
- [**Build Process**](https://github.com/vladmandic/human/wiki/Build-Process)
|
||||
- [**Adding Custom Modules**](https://github.com/vladmandic/human/wiki/Module)
|
||||
- [**Performance Notes**](https://github.com/vladmandic/human/wiki/Performance)
|
||||
- [**Performance Profiling**](https://github.com/vladmandic/human/wiki/Profiling)
|
||||
- [**Platform Support**](https://github.com/vladmandic/human/wiki/Platforms)
|
||||
|
|
|
@ -0,0 +1,74 @@
|
|||
# Custom Modules
|
||||
|
||||
## Overview
|
||||
|
||||
Each module is fully self enclosed:
|
||||
|
||||
- Defined in a separate folder under `/src/`
|
||||
- Performs `load` and `predict` functions
|
||||
- Uses global configuration object
|
||||
- Runs prediction on global input image
|
||||
- Merges results into global results object
|
||||
|
||||
<br>
|
||||
|
||||
## Define module
|
||||
|
||||
Define module that implements `load` and `predict` async methods:
|
||||
|
||||
```js
|
||||
export async function load(config: Config | any) { ... }
|
||||
```
|
||||
|
||||
- loads specific model using `modelPath` configuration
|
||||
- returns `tf.GraphModel`
|
||||
|
||||
```js
|
||||
export async function predict(image: Tensor, config: Config, idx, count) { ... }
|
||||
```
|
||||
|
||||
- input image is already preprocessed and passed to predict method as tensor
|
||||
- takes input tensor and resizes and normalizes it as needed
|
||||
- optionally implements `skipFrames` for caching of results
|
||||
- optionally uses `idx` and `count` params to map cached objects to current objects
|
||||
- uses any other optional configuration params
|
||||
- returns result object
|
||||
|
||||
<br>
|
||||
|
||||
## Enable Module
|
||||
|
||||
in `human.ts`:
|
||||
|
||||
- define model placeholder (`module.load()` stores model here) in the main constructor
|
||||
|
||||
in `config.ts`:
|
||||
|
||||
- define configuration types
|
||||
- set configuration defaul values
|
||||
|
||||
in `result.ts`:
|
||||
|
||||
- define results type
|
||||
|
||||
in `models.ts`:
|
||||
|
||||
- enable model loading sync and async by calling `module.load()`
|
||||
|
||||
<br>
|
||||
|
||||
## Execute Module
|
||||
|
||||
if model works on full image, execute from `human.ts`
|
||||
if model works on face image, execute from `face.ts`
|
||||
|
||||
- add results placeholder
|
||||
- implement calls to `module.predict()` for sync and async
|
||||
- merge results object
|
||||
|
||||
## Examples
|
||||
|
||||
- For simple module that works on pre-detected face tensor,
|
||||
follow existing module defintion is `emotion/emotion.ts`
|
||||
- For simple module that works on a full image tensor,
|
||||
follow existing module definition in `object/centernet.ts`
|
Loading…
Reference in New Issue