GroovyMorning_RLAC

Model and Dataset Hub: Hugging Face Integration

Managing and deploying AI models requires robust infrastructure, which is difficult to achieve on GitHub due to the size of the models. To centralize the models and automate their distribution, the Hugging Face ecosystem has been integrated at the core of the project.

Centralizing Assets

The training module regularly produces new AI models (Audio Segmenters, Transcription Detectors). Rather than storing these large files in Git, they are hosted on the Hugging Face Hub.

This enables:

• Centralization and Versioning: Hosting models (Audio Segmenters, CamemBERT) and datasets on the Hub for strict version tracking.
• Simplified Distribution: The segmenter (which handles chronicle detection) can dynamically download the latest model version via the huggingface_hub library.
• Documentation (Model Cards): Each model is accompanied by a README.md file describing its performance, architecture, and training data.

Pipeline Automation (Push-to-Hub)

One of the strengths of GroovyMorning’s integration is the post-training automation. Several scripts facilitate this transition:

1. Push-to-Hub Pipelines: Automating the transfer of model weights and audio recalibration snapshots as soon as training phases are completed.
2. hf_push_assets.py: Manages the upload of datasets and snapshots needed for audio recalibration.
3. Production Accessibility: Dynamic downloading of the latest versions via the huggingface_hub library, ensuring the detection engine stays up to date.

Available Models

Models are available on the project’s Hugging Face repo:

• rlac-audio-segmenter-chroniques: Native detection via audio signal.
• rlac-audiotranscript-segmenter-chroniques-hybrid: Model combining text and temporal metadata.
• camembert-chronicle-start-detection: Specialized model for spotting introduction phrases.

---

Version française : Hub de Modèles et Datasets : Intégration Hugging Face

This site is open source. Improve this page.