From bffa2cb0408f8fca1db66d461ceb1068be201a21 Mon Sep 17 00:00:00 2001
From: Vladimir Mandic <mandic00@live.com>
Date: Sat, 31 Jul 2021 20:39:36 -0400
Subject: [PATCH] add notes on building custom modules

---
 Home.md   |  1 +
 Module.md | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 75 insertions(+)
 create mode 100644 Module.md

diff --git a/Home.md b/Home.md
index 5bc164a..a0aeba7 100644
--- a/Home.md
+++ b/Home.md
@@ -53,6 +53,7 @@ Check out [**Live Demo**](https://vladmandic.github.io/human/demo/index.html) fo
 - [**Notes on Backends**](https://github.com/vladmandic/human/wiki/Backends)
 - [**Development Server**](https://github.com/vladmandic/human/wiki/Development-Server)
 - [**Build Process**](https://github.com/vladmandic/human/wiki/Build-Process)
+- [**Adding Custom Modules**](https://github.com/vladmandic/human/wiki/Module)
 - [**Performance Notes**](https://github.com/vladmandic/human/wiki/Performance)
 - [**Performance Profiling**](https://github.com/vladmandic/human/wiki/Profiling)
 - [**Platform Support**](https://github.com/vladmandic/human/wiki/Platforms)
diff --git a/Module.md b/Module.md
new file mode 100644
index 0000000..978fb3f
--- /dev/null
+++ b/Module.md
@@ -0,0 +1,74 @@
+# Custom Modules
+
+## Overview
+
+Each module is fully self enclosed:
+
+- Defined in a separate folder under `/src/`
+- Performs `load` and `predict` functions
+- Uses global configuration object
+- Runs prediction on global input image
+- Merges results into global results object
+
+<br>
+
+## Define module
+
+Define module that implements `load` and `predict` async methods:
+
+```js
+export async function load(config: Config | any) { ... }
+```
+
+- loads specific model using `modelPath` configuration
+- returns `tf.GraphModel`
+
+```js
+export async function predict(image: Tensor, config: Config, idx, count) { ... }
+```
+
+- input image is already preprocessed and passed to predict method as tensor
+- takes input tensor and resizes and normalizes it as needed
+- optionally implements `skipFrames` for caching of results
+- optionally uses `idx` and `count` params to map cached objects to current objects
+- uses any other optional configuration params
+- returns result object
+
+<br>
+
+## Enable Module
+
+in `human.ts`:
+
+- define model placeholder (`module.load()` stores model here) in the main constructor
+
+in `config.ts`:
+
+- define configuration types
+- set configuration defaul values
+
+in `result.ts`:
+
+- define results type
+
+in `models.ts`:
+
+- enable model loading sync and async by calling `module.load()`
+
+<br>
+
+## Execute Module
+
+if model works on full image, execute from `human.ts`
+if model works on face image, execute from `face.ts`
+
+- add results placeholder
+- implement calls to `module.predict()` for sync and async
+- merge results object
+
+## Examples
+
+- For simple module that works on pre-detected face tensor,  
+  follow existing module defintion is `emotion/emotion.ts`  
+- For simple module that works on a full image tensor,  
+  follow existing module definition in `object/centernet.ts`