(Work-In-Progress) Let your computer see your screen and interpret it using TFJS models

npm install

After package installations, perform the following changes:

Change \node_modules\@microsoft\customvision-tfjs-node\lib\index.js:97 to return _this._preprocess(tf.browser.fromPixels(pixels));

This will process the image by its raw pixels instead of parsing it to an image file. Screenshots return raw pixel array without image encoding.

http://robotjs.io/docs/electron

npm start

npm install node-pre-gyp -g

Used for building the TFJS libraries. Maybe due to the wrong Python version, it fails to build from npm install command, therefore we use node-pre-gyp to manually build it

node_modules/@tensorflow/tfjs-node > node-pre-gyp rebuild

node_modules/@tensorflow/tfjs-node > tensorflow/tfjs#4171 (comment)

root > npm rebuild @tensorflow/tfjs-node --build-addon-from-source

When using a model, construct the TF object within the propritary package. Can't seem to import TFJS and Custom Vision together.