GroovyMorning_RLAC
AI Models Finetuning
Once the dataset is established and the “Ground Truth” set, the next step is to train and refine (finetune) artificial intelligence models capable of segmenting the radio stream with surgical precision. This R&D process explored several architectures, ranging from pure audio signal processing to semantic textual analysis.
Finetuning Strategy
Finetuning involves taking a model pre-trained on vast volumes of data (such as CamemBERT for text or Wav2Vec2 for audio) and specifically adapting it to the characteristics of a radio broadcast (jingles, speech rate, typical transitions).
Two major axes were explored:
- Textual Approach (NLP): Analysis of the transcription generated in real-time.
- Audio Approach (Computer Vision/Signal): Direct analysis of the sound footprint.
Technical Behind-the-Scenes: How to Switch to Finetuning
Switching from a “generic” model (which understands language or sound) to an “expert” model (which recognizes your radio segments) relies on a precise industrial pipeline.
1. The Engine: Frameworks and Libraries
The project relies on the Hugging Face ecosystem via the transformers library. It provides specialized classes such as CamembertForSequenceClassification.
Under the hood, PyTorch, the tensor calculation framework, handles the millions of matrix multiplications required. It transforms text into numerical tokens (unique identifiers for each piece of a word) so that the AI can “calculate” meaning rather than “reading” it.
2. Architecture: The “Head” Graft
Technically, finetuning consists of keeping the body of the pre-trained model (which already possesses encyclopedic knowledge) and replacing its output layer with a blank classification head. This new layer is specifically sized for our task (e.g., 2 outputs for a binary classification: “Chronicle” vs. “Other”).
3. Preparation: Labeling
A golden rule: labeling is always done BEFORE training. This is the creation phase of the Ground Truth. We prepare files where each sequence of tokens is associated with its answer:
- Sequence A (“Hello everyone…”) ➔ Label 1 (Start of chronicle).
- Sequence B (“Advertisement…”) ➔ Label 0 (Other). The model is like a student: it needs both questions and answers to learn.
4. The Training Loop: Iterative Learning
Once the data is ready, the model enters a training loop repeated several times (epochs):
- Forward Pass: The model guesses if the segment is a chronicle.
- Loss Calculation: We measure the gap between its guess and the actual label (our answer key).
- Backward Pass (Backpropagation): The framework identifies which neurons are responsible for the error.
- Optimization: We slightly adjust the model’s settings to reduce the error in the next round.
Semantic Finetuning: The CamemBERT Approach
For text-based detection, we used CamemBERT, a transformer model specialized in the French language.
Semantic Augmentation via Sliding Window
An isolated 2-second segment (“It is 8:12”) is often insufficient to identify a chronicle. We implemented a sliding window technique:
- The model analyzes the target segment surrounded by its immediate context (2 segments before, 2 segments after).
- This “peripheral” vision allows the model to capture the typical discourse structures of chronicle launches.
Hybrid Architecture
To go further, a three-stage architecture was tested:
- CamemBERT for semantic embeddings.
- Bi-LSTM to capture the sequential progression of the broadcast.
- CRF (Conditional Random Field) to ensure the logical consistency of the sequence (e.g., a chronicle cannot end before it has started).
Audio Finetuning: From Wave to Spectrogram
In parallel with the text, audio provides valuable clues, especially musical jingles that transcription ignores.
Wav2Vec2 and AST
The finetuning of state-of-the-art models was experimented with:
- Wav2Vec2: Suitable for binary classification (chronicle vs. silence/music).
- AST (Audio Spectrogram Transformer): This model treats audio as an image via spectrograms. It proved particularly effective in identifying the unique acoustic signatures of Radio France jingles.
Detection via Anchor Points (Jingles)
One of the most robust methods involves using a lightweight model specialized only in detecting intro jingles. Once the jingle is detected with high confidence, the system “anchors” the start of the chronicle and switches to a speech tracking model to determine its end.
Optimization and Prediction Smoothing
AI models can sometimes hesitate (oscillating between “chronicle” and “non-chronicle”). To stabilize detection in real conditions, several post-processing techniques have been developed:
- Hysteresis Thresholding: Use of a high threshold to trigger detection and a lower threshold to maintain it, avoiding untimely cuts.
- Soft Voting: Weighting confidence scores to smooth predictions over time.
- Gap Filling: Automatic filling of micro-silences or hesitations within a coherent detection block.
Monitoring and Iteration
Each training is monitored via Weights & Biases (WandB) to track the evolution of metrics (Loss, F1-Score) and stored on Hugging Face for deployment. Although local finetuning approaches showed promising results, it is the semantic intelligence of LLMs (DeepSeek/Claude) that ultimately offered the best robustness for the current production solution.
Technical details of the training are available in the article dedicated to audio and the one dedicated to text.