In early October 2026 I posted a series of LinkedIn updates announcing new tiny multimodal models, sharing my upcoming panel at the Science × AI Summit, and summarizing research on RL‑enhanced distillation methods (OPD, ExOPD, RIDE).
I released the model d1‑3B with text and vision capabilities.
I also released d1‑omni‑600M supporting text, vision, audio.
My panel will happen on October 16 from 2:30 PM to 3:20 PM about how industry is building for science.
On‑policy distillation (OPD) is now a standard post‑training technique.
ExOPD suffers from output_layer_compression, as the 512 directions it amplifies least carry 79.8% of the RL change in hidden states, but only 30.0% of it in the logits.
RIDE is the only method whose average beats the RL teacher on every pair (by 0.32 to 1.08 points on AIME and AMC).
Three of the RIDE gains are within one standard deviation across seeds.
ExOPD lands below the teacher on every pair, and -14.1 points below the untouched student on Qwen3‑4B.
I decided to adopt RIDE over ExOPD for RL‑enhanced distillation because it consistently outperforms the RL teacher across model sizes while avoiding the output‑layer compression and high‑variance issues of ExOPD. I also plan to keep the extrapolation factor at 25 % as a good trade‑off between gain and computation.