I built a multimodal conversational system that processes spoken input, extracts textual, prosodic, and emotional cues, and generates spoken replies, using a pipeline of specialized components.
I use Whisper for speech‑to‑text transcription.
I extract prosodic features with Librosa.
I employ Wav2Vec2‑based emotion detection for emotion recognition.
I use Qwen as the language model to understand context and generate responses.
I generate spoken replies via Kokoro TTS.
The policy component is enabled (true) to enforce tone consistency and safety checks.
I chose a pipeline architecture over a monolithic design for better modularity, debugging ease, and clear responsibility boundaries.