eight years of evaluating speech-AI before it ships.
rolling the dice…
this site is the trophy case for those tools. (the paint is still wet.)
* one company, eight years, five boss fights. the map just likes drama.
the 140-hour problem. watched the team collectively burn 140 hours prepping smart-speaker test data before a single metric could be computed. built the first end-to-end data processing pipeline. the manual clicking retired with full honors.
alexa vs. the world. kept building automation and led evaluation of alexa's speech & speaker recognition against every competitor device that showed up. the lab had more smart speakers than an electronics store.
enter the audio LLMs. took over evaluation of amazon's sonic audio LLM and its competitors. the models learned to talk — somebody had to keep the scorecards honest.
the 3-minute kickoff. built AWS orchestration to start a 100-dataset benchmark in about 3 minutes instead of roughly 300 minutes of sequential starts. the model runs carry on after kickoff — enough time for a less tense coffee.
level locked. the map refuses to say. something is being built behind that curtain.
Test automation
AWS & workflows
Data & SQL
AI evaluation
AI-assisted dev
Lab & misc
140 collective hours → about one.
100-dataset kickoff: 300 minutes → about three.
(or just want to argue about WER methodologies — both welcome.)
want the architecture notes? send me a message.