Fix-It-Flow
Most e-waste starts with a repair knowledge gap. Fix-It-Flow gives people a hands-free repair guide that sees the item, asks a few questions, and pushes toward repair before replacement.
A broken lamp, coffee maker, or small appliance often gets thrown away because the owner does not know where to start. The goal was a sustainability-first assistant that feels like a patient technician, not a generic chatbot.
Fix-It-Flow runs two connected modes: inspection to identify the item and problem, then repair to walk the user through tailored steps with hands-free voice controls.
Inspection flow
The user describes the problem while pointing the camera at the item. Gemini Vision analyzes live frames and turns visual evidence into structured context.
Sustainable reasoning
Featherless.AI running Llama 3.1 reasons over the visual context and conversation, asks up to three clarifying questions, then recommends repair, part replacement, repurpose, donation, or recycling.
Repair walkthrough
After diagnosis, the app generates a short step-by-step repair sequence tailored to the item and broken part.
Voice-first UX
ElevenLabs reads each step aloud, while Web Speech API commands like next, repeat, help, and end session let the user keep both hands on the repair.
Session state
AWS DynamoDB stores conversation history and repair progress so the flow can survive refreshes and continue from the right point.
Sequential vision-to-reasoning pipeline
Running the model calls in parallel looked faster, but it risked giving the reasoning model stale visual context. The final pipeline runs Gemini first, then passes fresh observations into Featherless.
Sustainability priority in the prompt
The model had to be steered away from default replacement advice. Recommendations are ordered around repair, replacing only the broken part, repurposing, donating, and recycling.
Voice as the primary interface
Repair is a hands-busy workflow. Speech commands and TTS made the app usable while someone is holding tools, checking parts, or looking at the device.
Built with