TensorFlow in JavaScript

I have been working on Gesture Quest Web, and I just spent a solid few hours losing a fight with model conversion.

The setup was in place. Camera feed running, hand landmarks coming in, all of it cleaned and reshaped so it was ready for inference. The only missing piece was the model itself. To run inference in the browser, I needed it in TensorFlow.js form.

First I tried converting our quantized TFLite model straight to TFJS. Turns out the TFJS converter does not take a .tflite file at all. You are meant to convert from the original Keras or SavedModel, not the little phone-sized version at the end of the pipeline. There is supposedly a detour through ONNX, but that road is more trouble than it is worth.

So I went back to the source model, an old .h5 file, the format .keras eventually replaced. Rather than wrestle with that, I just reran the entire training from scratch to produce fresh copies in both modern formats: a .keras file and a SavedModel, the one with a .pb file and a whole pile of variable files next to it. I converted both. Both came out bad, still misreading a bunch of signs. A hand-tracking game that confuses half the alphabet is not exactly shippable.

Eventually I gave up on converting anything and just rewrote the training directly in JavaScript. Which was not even that hard. The new model trained, ran natively in the browser, and did well on our data. It performs a tiny bitty bit worse than the Python one, but it is plenty good for the game.

I am choosing to call it "not overfitted" and move on.