Final-year projects / Assistive tech

Sign language to speech glove

Most people do not know Indian Sign Language, so deaf and mute people struggle in shops, offices and hospitals. A glove that recognises a set of signs and speaks them aloud helps bridge that gap.

ESP32Major projectECE, CSE, BME, EIE
ESP32FIVEMPU605DFPLAYtilt 26°Featured
Parts
₹3,500
Build
14 weeks
Board
ESP32
Objectives
5

Block diagram

Inputs
  • FIVE
    Five flex sensorsMeasure each finger's bend
  • MPU605
    MPU6050Measures hand tilt and motion
Controller
  • ESP32
    ESP32Classifies the sign
Outputs
  • DFPLAY
    DFPlayer and speakerSpeak the word
  • 23.4°COLED
    OLED displayShows the word
  • PHONE
    Phone appShows readings and sends commands over Bluetooth+

Objectives

  1. To read five finger bends and hand orientation 50 times a second.
  2. To recognise at least 15 signs (letters and common words).
  3. To speak each word in English and one Indian language.
  4. To report accuracy per sign for three users with a confusion matrix.
  5. To monitor and control the system from a phone over Bluetooth.

Parts and specifications

Total about ₹3,500 at typical online prices. Tap a part for its pinout and datasheet notes.

Design calculations

Flex divider

Vout = 3.3 × 47k / (47k + Rflex). Flat 25 kΩ → 2.15 V; bent 70 kΩ → 1.33 V: about 1000 ADC counts of range.

Feature vector

Five normalised bends (0 to 1) plus roll, pitch and three acceleration means over 0.5 s = 10 features per sign.

Classifier

k-nearest neighbours (k = 3) on recorded samples, 30 per sign per user. A small neural network (TinyML) handles signs with motion better.

Pin map

SignalPin or connection
Flex sensors (thumb to little)GPIO 36, 39, 34, 35, 32 with 47 kΩ dividers
MPU6050 SDA / SCLGPIO 21 / 22
OLEDSame I²C bus
DFPlayer TX / RXGPIO 16 / 17

Firmware outline

  1. Calibrate: record each finger flat and fully bent for this user.
  2. Sample at 50 Hz; normalise; smooth with a moving average.
  3. When the hand is still for 300 ms, take the features and classify.
  4. If confidence is high, show the word, play its clip (English or Tamil/Hindi folder), and wait for a new sign.
  5. Collect training data mode: label and save samples over serial.

Test plan

TestExpected result
15 signs × 20 tries by the trained userAccuracy per sign
Two new usersAccuracy before and after their own calibration
Similar signs (for example M and N)Confusion matrix shows mix-ups
Time from sign to speechUnder 1 s

These rows start your results table when you plan the project.

Viva practice

Say your answer out loud first, then check. The first 2 answers are free; Maker opens all 5.

1. How does a flex sensor work?
2. Why add an IMU?
3. Why is calibration per user needed?
4. What is k-NN?
5. What are the limits for real ISL?

Make it yours

One or two upgrades for a mini project, two to four for a major one.

Build these first

Datasheets and sources to cite

  • Spectra Symbol flex sensor datasheet
  • Indian Sign Language Research and Training Centre (ISLRTC) dictionary
  • InvenSense MPU-6050 datasheet

About this project

How much does the sign language to speech glove project cost?

About ₹3,500 for the parts in the base list: ESP32 DevKit V1, Flex sensor 2.2 inch × 5, MPU6050 accelerometer and gyro, DFPlayer Mini MP3 module, Speaker 8 Ω 0.5 W, 0.96 inch OLED (SSD1306, I²C), 47 kΩ resistors and a cotton glove, 18650 cell 2600 mAh (branded), TP4056 charger with protection, MT3608 boost converter.

How long does it take?

About 12 weeks for the base project, built on an ESP32. Each upgrade adds time.

Which branches is it suitable for?

Electronics and Communication, Computer Science / IT (IoT, AI), Biomedical, Instrumentation, as a major project.

Which pins and design calculations does it use?

Flex sensors (thumb to little) on GPIO 36, 39, 34, 35, 32 with 47 kΩ dividers; MPU6050 SDA / SCL on GPIO 21 / 22; OLED on Same I²C bus. Worked calculations on this page: flex divider, feature vector, classifier.

How do I test it?

15 signs × 20 tries by the trained user: Accuracy per sign. Two new users: Accuracy before and after their own calibration. Similar signs (for example M and N): Confusion matrix shows mix-ups. Time from sign to speech: Under 1 s.

What are the objectives?

To read five finger bends and hand orientation 50 times a second. To recognise at least 15 signs (letters and common words). To speak each word in English and one Indian language. To report accuracy per sign for three users with a confusion matrix.