Speech recognition technology and applications

From: aidotengineer

Nvidia Speech AI focuses on enterprise-level speech AI model deployment. Their goal is to enable customers to provide the best possible conversational AI at an enterprise level [00:48:00]. This includes speech translation, text-to-speech development, and speech recognition [00:39:00]. A primary focus is on low-latency, highly efficient models that can be used on embedded devices [00:56:00].

Core Principles of Nvidia Speech AI Model Development

Nvidia’s model development revolves around four key categories [01:08:00]:

• Robustness: Models are designed to work effectively in both noisy and clean environments, accounting for varying sound quality and environmental contamination factors like telephony [01:13:00].
• Coverage: The aim is to meet diverse customer demands across various domains (e.g., medical, entertainment, call center-based) and language demands, including monolingual, multilingual, dialectal variations, and code-switching [01:31:00].
• Personalization: Customers can tailor models to their exact needs, which may involve target speaker AI, word boosting for uncommon vocabulary, or text normalization FST models for specific outputs [01:58:00].
• Deployment Cases: Considerations include the trade-off between speed and accuracy, and whether models should prioritize high variety or efficiency [02:21:00].

Model Architectures

Nvidia utilizes several model types for speech AI applications [02:41:00]:

• CTC (Connectionist Temporal Classification) Models: These models are favored for high-speed inference, especially in streaming environments, due to their non-auto-regressive decoding [02:51:00].
• RNT (Recurrent Neural Transducer) / TDT Models: When higher accuracy is required beyond what non-auto-regression can provide, RNT or the Nvidia-developed TDT variants are used. These feature auto-regressive streaming setups by integrating an audio encoder with an internal language model [03:14:00].
• Attention Encoder-Decoder Setups: For maximum accuracy, where streaming is not a primary concern, these models (like Whisper, ChatGPT, LLM) are employed. They are highly accurate, accommodate multiple tasks within a single model (e.g., speech translation, timestamp prediction, language identification, speech recognition) with simple prompt changes, and require less focus on alignment [03:40:00].
• Fast Conformer: This serves as the fundamental architecture across all decoding platforms. It significantly subsamples the original conformer model, enabling 80-millisecond compression instead of the conventional 40-millisecond step. This reduces memory load during training, makes training more efficient with quicker convergence on less data, and allows for very fast inference due to chunking data into smaller time steps [04:31:00].

Nvidia Reva Offerings

Nvidia’s model offerings are split into two primary options [05:27:00]:

• Reva Parakeet: This focuses on streaming speech recognition cases, utilizing CTC and TDT models for fast and efficient recognition, speech translation, and target speaker ASR [05:31:00].
• Reva Canary: This option employs fast conformer models, prioritizing accuracy and multitask modeling, even if it means slightly less focus on speed [06:00:00].

The overarching philosophy is to provide a variety of models to meet specific needs rather than a “one model fits all” approach [06:36:00].

Enhancements for Accuracy and Customization

Additional models and features are offered to improve accuracy, customization, and readability of transcripts [08:27:00]:

• Speaker Diarization (Softformer): The Parakeet ASR model can be extended for multi-speaker and target speaker scenarios using the Softformer. This end-to-end neural diarizer functions on the “who comes first” principle, bridging speaker timestamps from diarization with speaker tokens recognized by the ASR model [07:00:00]. It can be fine-tuned with a simple objective similar to ASR model training [07:44:00].
• Voice Activity Detection (VAD): Detects speech segments to improve noise robustness. Nvidia offers VAD models based on MarbleNet [08:34:00].
• External Language Models (LMs): Resources like n-gram-based language models are used to enhance ASR transcription for better accuracy and customization [08:47:00].
• Text Normalization (TN) and Inverse Text Normalization (ITN): Converts spoken terms to written forms for improved readability. WFST-based ITN models are supported [09:01:00].
• Punctuation and Capitalization (PNC): Adds punctuation and capitalization to transcriptions for better readability, using BERT-based PNC models [09:14:00].
• Speaker Diarization: Identifies multiple speakers in a conversation. ZooFormer-based speaker diarization models are available in cascade systems, with upcoming end-to-end models [09:28:00].
• Word Boosting: Offered to improve recognition of product names, jargon, and context-specific knowledge [15:16:00].

Nvidia’s approach to customization and variety contributes to its strong performance, with the majority of top models on the Hugging Face Open ASR leaderboard coming from Nvidia [09:52:00].

“Living for the now. Long as time allows. I’mma keep on switching different styles. Keep creative on a cloud. Sweat is on my brow cuz I’m running on these tracks just to keep them running back. You know the drill back and I’ve been practicing my craft. Dedicate this to Kobe. What could be a bigger legacy to” [10:25:00] — A demonstration of accurate transcription even in a noisy setting [10:46:00].

Development Process

Nvidia focuses on fundamental practices for data development and training [11:06:00].

• Data Sourcing:
  • Emphasis on robustness, multilingual coverage, and dialect sensitivity [11:14:00].
  • Incorporation of both open-source data (for variety and domain shift) and proprietary data (for high-quality entity data) [11:30:00].
  • Utilization of pseudo-labeling, where transcripts are generated using top-of-the-line commercial models, benefiting from community advancements [11:44:00].
• Training:
  • The Nemo research toolkit, an open-source library, is used for model training [12:07:00].
  • Tools within Nemo support GPU maximalization, data bucketing, and high-speed data loading via the Latte backend [12:20:00].
  • Data is stored on an object store infrastructure for quick migration between different cluster settings [12:41:00].
• Validation:
  • A mixture of open-source and proprietary data is used [12:52:00].
  • Rigorous bias and domain testing are conducted across all language categories to ensure model robustness before release to end-users [13:04:00].

Deployment with Nvidia Reva and NIM

Trained models are deployed through Nvidia Reva using Nvidia NIM for low-latency and high-throughput inference [13:22:00].

• High Performance Inference: Powered by Nvidia TensorRT optimizations and the Nvidia Triton Inference Server [13:31:00].
• Availability: Offered via gRPC-based microservices for low-latency streaming and high-throughput offline use cases [13:42:00].
• Scalability: Nvidia Reva is fully containerized and can easily scale to hundreds of parallel streams [13:50:00].
• Deployment Environments: Can be run on-premise, in any cloud, at the edge, or on embedded platforms [13:59:00].
• Applications: Supports various applications including contact centers, consumer applications, and video conferencing [14:06:00].
• Nvidia NIM: Provides pre-built containers, industry-standard API support for custom models, and optimized inference engines [14:16:00].

Customization is a key feature in real-world scenarios due to varying domain knowledge requirements (e.g., medical terms, menu names, telephonic conditions, noisy environments in contact centers) [14:27:00]. Nvidia Reva offers customization at every stage, allowing fine-tuning of acoustic models (Parakeet and Canary-based), external language models, punctuation models, and inverse text normalization models [14:55:00].

Getting Started

Nvidia Reva models are available in Nvidia NIM. Users can explore available models at build.envidia.com/explore/speech [15:32:00]. Resources include a quick start guide, developer forum, and a fine-tuning guide for models within the Nemo framework [15:56:00].