AI vision for construction photos

I led research and contextual inquiry, owned the tag taxonomy and vocabulary decisions, drove search interaction design and mobile-first direction, validated the AI confidence threshold model with users, and aligned SnapSoft and Carnera around a shared UX direction throughout.

SnapSoft owned the ML pipeline. Carnera executed visual UI. The decisions about what the system should communicate to users—and what it should stay quiet about—were mine.

I conducted contextual inquiry with 8 field users across 3 construction firms. Users think in terms of content, not chronology—materials, equipment, site conditions—not when or where a photo was taken. Average time to find a specific photo by scrolling: 4.2 minutes.

Field adoption of manual tags sat at 15%. Meanwhile, 92% of photos were captured on mobile, but 40% of retrieval happened on desktop during reporting. Search sessions broke into specific recall and categorical browse—both requiring content-indexed search, surfaced differently in the UI.

Marcus — Site Superintendent. Takes 30–50 photos daily across subcontractor documentation, safety records, and progress updates. Primary device: phone, often in low-connectivity conditions. “By Friday I've taken 200 photos. Finding one from Tuesday? Forget it.”

Sarah — Project Manager. Assembles weekly reports and compliance audits from photos across the whole team. Works on desktop. “I need all concrete pour photos from the east wing, from anyone on the team.”

AWS Rekognition produces generic object labels. SageMaker custom models produce domain labels—rebar types, concrete conditions, equipment models. The taxonomy had to merge both into one vocabulary that matched how field users actually spoke.

SnapSoft's initial label set didn't map to field vocabulary. “Excavating equipment” isn't what Marcus says; “excavator” and “backhoe” are. Building the synonym map required sitting between ML output and user research—forcing an alignment neither party was initially scoping for.