Workshop / Webcam AI lab

Webcam AI lab

Count fingers, find faces, track colours and read QR codes with your camera, in your browser. Each result can drive a real Arduino or ESP32 over USB.

The camera picture stays on this device. Nothing is uploaded or saved.

How it works

A hand landmark model gives the position of each joint. A finger is straight when its tip is much farther from the wrist than its middle joint, so the count works at any angle.

MediaPipe hand landmarker (7.8 MB, downloaded once). Camera frames are processed in this browser and never leave your device.

Board sketch

// Light one LED per finger the webcam sees (LEDs on pins 2 to 6, each with a 220 ohm resistor).
// The webcam AI lab sends "FINGERS n" over USB at 115200 baud.
const int LEDS[] = {2, 3, 4, 5, 6};
String line;

bool readLine() {
  while (Serial.available()) {
    char c = Serial.read();
    if (c == '\n') return true;
    if (c != '\r' && line.length() < 60) line += c;
  }
  return false;
}

void setup() {
  Serial.begin(115200);
  for (int i = 0; i < 5; i++) pinMode(LEDS[i], OUTPUT);
}

void loop() {
  if (!readLine()) return;
  if (line.startsWith("FINGERS ")) {
    int n = line.substring(8).toInt();
    for (int i = 0; i < 5; i++) digitalWrite(LEDS[i], i < n ? HIGH : LOW);
  }
  line = "";
}

Run vision on the board itself

  • Live demos: the laptop sees, the board acts
  • Train your own: a small model runs on an ESP32-S3 or ESP32-CAM, no laptop (TinyML)
  • Models fit in a few hundred KB: person or no person, or a handful of objects

Common questions

Are my camera images uploaded?

No. Every frame is processed in your browser and never leaves your device. Only the model files for the hand, face and object demos are downloaded, once, from Google's public MediaPipe model store.

Which computer vision demos are there?

Finger counting with hand landmarks, face detection, object detection for 80 everyday objects, colour tracking, motion detection by frame differencing, and QR code reading.

Can the webcam control an Arduino?

Yes. Each demo turns what it sees into a short text line such as FINGERS 3 or PAN 97. Upload the matching sketch, plug the board in by USB and connect it in desktop Chrome or Edge; the board lights LEDs, turns a servo or switches a relay from the camera.

Can the model run on the microcontroller itself?

Small models can. An ESP32-S3 or ESP32-CAM can run a person detector or a small image classifier of a few hundred kilobytes; that is TinyML. The lab links to the TinyML lessons and builds.

Can I train my own image model?

Yes. In Train your own, record pictures of two to four objects, train a small neural network in the browser in under a second and see its accuracy on pictures it never trained on. Then download an Arduino sketch with the model inside (int8 weights, about 7 KB) for a XIAO ESP32S3 Sense or an AI-Thinker ESP32-CAM; it runs on the board's camera with no ML library.