Speech Model Curricula Design
Designing optimal big data curricula for large speech models
| I worked on this research project at Deepgram, a company building advanced speech-to-text APIs. |
Deepgram uses large datasets to train even larger models. While this allows for powerful end-to-end high-capacity speech transcription systems, it also is computationally expensive and restricts the efficiency of experimental R&D. I find that intentionally restricting the dataset can significantly improve the efficiency of model training between two to five times, and particularly results in substantive absolute improvements to the diminishing-returns regime of large model training.
View a writeup here.