Qwen3.8-Omni Explained: The First Native Omni-Modal Agent

Last Updated: September 26, 2026

Key takeaways

Qwen3.8-Omni-Flash is Alibaba's first omni model built for agentic work, for editing, translating, and producing audio and video, not just perceiving it.

The model runs on a sparse mixture-of-experts backbone with 1M-token context, and Qwen reports a 25% average improvement over its predecessor across 29 audio, audiovisual, and agent evaluations.

Agentic evidence gathering beats brute-force video processing on the paper's own tests: accuracy up, tokens per query down 45.7%.

Qwen-MM-Plugins bolts audio and video onto agent harnesses like OpenClaw and Claude Code, and Qwen-Live-Harness runs realtime multimodal agents with sub-agent delegation.

Gemini 3.8 Flash still wins most direct video reasoning comparisons, but the gap closes or flips when the Qwen model gets to act like an agent.

What is Qwen3.8-Omni and why does it matter?

Qwen3.8-Omni-Flash is a natively multimodal agentic model from Alibaba's Qwen Team, released in a technical report on September 22, 2026. One model takes in text, images, audio, video, and spatial audio, reasons over all of them, and answers with text or realtime speech. The difference from earlier omni models is the job it is trained for. Previous generations focused on perception and conversation. This one plans multi-step work: it listens to a full meeting and writes minutes with action items, watches hours of footage and builds a commentary video, listens to a song and produces a music video plan shot by shot.

Every multimodal AI team is converging on the same product: an agent that hears and sees. Owning the model layer for that agent, with the plugins to drop it into the tools developers already run, is how Alibaba competes with Google's Gemini and ByteDance's Seed line. The open-source frameworks shipped alongside the model are the real signal. Alibaba is not just publishing a benchmark table. It is shipping the plumbing so any agent harness can suddenly edit video.

Three problems framed the whole research agenda, according to the Qwen Team: video is expensive to store, transmit, and tokenise; most agent harnesses cannot accept streaming audio and video in the main model context; and almost nobody has explored omni models for actual productivity work. The report is the team's answer to all three.

What is inside Qwen3.8-Omni-Flash?

According to the report, Qwen3.8-Omni-Flash keeps the Thinker-Talker architecture of the Qwen-Omni family. The Thinker is a hybrid sparse mixture-of-experts language model inherited from Qwen3.8-Next, combining Gated DeltaNet layers, which compress past context into a fixed-size recurrent state, with interleaved attention layers that upgrade to Qwen Sparse Attention, a learned indexer that selects the relevant context blocks instead of attending to everything. Pretraining used a native 262K context window, extended to 1M tokens after post-training. The Talker turns the Thinker's output into streaming speech, upsampling waveform reconstruction from 24 kHz to 48 kHz.

Three perception encoders feed the Thinker: a vision encoder for images and video frames, a general audio encoder outputting tokens at 6.25 Hz (one token per 160 ms of audio), and, new in this generation, a Spatial AuT encoder that processes multichannel audio in ambisonics format so the model can reason about where a sound is coming from. Explicit timestamps attach to every audio and video representation, so the language model always knows where in the timeline each piece of evidence sits.

Two training choices carry most of the capability. First, native multimodal co-training: 2.5 trillion tokens of pretraining data split across text (1.1T), audio (0.7T, mono and multichannel), image (0.35T), video (0.15T), and paired video-audio (0.3T), so agent skills learned in text transfer to ears and eyes. Second, post-training by multi-teacher distillation: Alibaba trained specialist models for coding, agents, vision, and audio, generated trajectories from each, then distilled them into one student before a unified RL pass that rewards task outcomes rather than self-reported completion.

How much better is it than the last generation?

According to the Qwen Team's own evaluation tables, Qwen3.8-Omni-Flash improves the average score across 29 audio, audiovisual, and agent evaluations by more than 25% compared with Qwen3.5-Omni-Plus, and cuts estimated input costs for an hour of audio and audiovisual content by more than 98% and 93% respectively. The single biggest jump in the whole report is multi-speaker ASR: on the AliMeeting test set, diarisation error rate falls from 88.1 to 3.4, and cross-word error rate from 89.6 to 17.2. The model also matched a perfect 100% success rate on OmniLingua-MultiSpeaker transcription with the lowest error metrics of the models listed.

Text capability, the usual casualty of multimodal scope creep, held. The paper reports 63.3 on SWE-bench Pro and 92.6 on LiveCodeBench v6, the best listed scores in both rows, against the Qwen3.8-Flash text model. Its declared weak spots are honest: FLEURS multilingual transcription regressed slightly (9.3 WER against 7.2 previously), and Gemini 3.8 Flash keeps the crown on straight video reasoning benchmarks, which we unpack below.

What can it actually do with video?

The productivity demos are where the paper earns the word agentic. In Meeting Minutes for Multispeaker, the model takes up to an hour of meeting audiovisual input and performs speaker segmentation, transcription, and identity alignment end to end, then maps participant relationships, extracts action items, and can send the follow-up emails or begin the coding work the meeting asked for.

In Music-to-MV, the model analyses a song's lyrics, structure, and musical events, writes a narrative script aligned to the music's timeline, designs the shots, then reviews its own generated footage for visual quality against the plan. In Drama Translation, it plans the localisation of a short drama end to end: speaker-aware dialogue recognition, translation, character-consistent voice cloning and dubbing, audio remixing, and quality assessment. For a two to three hour movie, Long-Form Movie Commentary interleaves generated narration with original dialogue excerpts and background music, adjusting pacing by scene.

The research-and-learn side converts video into structured knowledge: Omni-Deep Research pulls claims and parameters out of a video, cross-checks them against the web, and produces an interactive report linking every finding back to the moment in the source video. Omni Skill Creator watches a task demonstration, extracts the standard operating procedure, and packages it as a reusable, verified skill other agents can invoke. That last one is the quiet headline: watch once, and the knowledge becomes executable.

One experiment shows the ceiling of this stack. The Qwen Team tasked the model with improving Qwen2.5-Omni-3B speech recognition for the Sichuan dialect within 12 hours, with no human in the loop. The model picked the evaluation set, set the baseline, listened to audio samples to diagnose recognition errors, built 3,413 targeted training examples across four experiment rounds, kept what worked, and rolled back what did not. Character error rate fell from 25.79% to 15.30%, a relative reduction of about 40.7%.

Does agentic video understanding actually beat brute force?

This is the most commercially interesting table in the report. Long-video understanding has traditionally meant shoving every sampled frame plus the audio into one model call and hoping the context window holds. The alternative tested here: let the agent plan, then retrieve and verify only the relevant segments through tool calls, delegating to subagents for parallel audio and visual analysis.

On the three video reasoning benchmarks, agentic execution improved accuracy every time for Qwen3.8-Omni-Flash: OmniVideoBench 63.4 to 67.8, Video-MME-v2 65.0 to 71.3, and LVOmniBench 63.3 to 73.6. Token efficiency improved at the same time: on OmniVideoBench, tokens per query fell from 145,736 to 79,117, a 45.7% reduction, per Table 8 of the report. The LVOmniBench reversal is the story: 7.4 points behind Gemini 3.8 Flash in static mode, 2.9 points ahead in agentic mode.

Honesty note: the same agentic uplift showed up for Gemini 3.8 Flash too, so agents help everyone; the advantage is model-plus-harness, not the harness alone.

How does it compare with Gemini 3.8 Flash on shared benchmarks?

Multi-agent execution is Qwen's stronghold. According to Table 6 of the report, Qwen3.8-Omni-Flash scores 71.0 on WildClawBench-MM, the multimodal subset of the agent benchmark executed through Claude Code, against 58.9 for Gemini 3.8 Flash, and leads UniClawBench, executed through OpenClaw, at 69.6 versus 69.0. Gemini 3.8 Flash leads on AgenticVBench (45.0 versus 36.8) and OmniGAIA web search (78.6 versus 74.0).

Scores from Qwen Team report Tables 5 to 7 and 10, September 2026. Agentic rows reflect each model paired with the stated harness.

The realtime variant holds up under its own constraints: running in non-thinking mode for responsiveness, it still posts 82.3 on OmniVChat-Bench (Gemini: 65.6) and beats Gemini on ODUbench user-intent understanding in both settings. Latency is deployment-grade: time to first audio around one second for audio input, 1.2 to 1.35 seconds with video, and generation at roughly 6.5x realtime speed, per Table 9.

What ships as open source?

Two frameworks land on GitHub under the QwenLM organisation. Qwen-MM-Plugins is the lightweight integration layer: existing agent harnesses (the paper explicitly names harnesses like Claude Code and, in its benchmark setup, OpenClaw) call these plugins for audio and video capability. It bundles Omni-Caption and Omni-Video2Note for turning video into text, Omni-Memory for agentic lazy loading so the model pulls clips on demand instead of hoovering hours of footage into context, and the Omni Skill Creator.

Qwen-Live-Harness is the realtime side: it connects Qwen3.8-Omni-Flash-Realtime to desktop audio and video with three system designs worth stealing. Asynchronous tool use and sub-agent delegation lets the spoken conversation continue while work runs in the background. Delegation targets include Qwen Code, Codex, and Claude Code through one adapter interface, and users can interrupt the talking agent without cancelling the delegated job. Proactive interaction runs perceptual monitors as independent sessions that push text event reports into the conversation when appropriate, with cooldowns to stop nagging. Persistent memory consolidates dialogue, working memory, and user facts into profiles retrieved by lexical and embedding search.

What this means for AI automation work

Three takeaways from a practitioner's seat. First, the plugin architecture matters more than the model: the moment open agent harnesses take video and audio tools natively, client workflows that needed a five-stage pipeline of ASR, diarisation, translation, dubbing, and editing collapse into one planning agent. We still run Demucs-based stem separation and ffmpeg pipelines locally, and a single agentic call replacing that chain is the difference between an automation project and an afternoon.

Second, the token economics change what is affordable. A 93% cut in per-hour video input cost turns previously absurd jobs, like monitoring a worksite timelapse or reviewing hours of meeting footage per department, into line items a client will actually approve.

Third, Skill Creator is the pattern to watch. Instinctively it is how agent ecosystems mature: capture an expert doing the task once, on video, and the system distils a verified, reusable agent skill from it. Deployment knowledge becomes a transferable artifact instead of tribal memory.

The local-AI question is open, as always: the report describes API deployments, and Flash-class MoE models in this family have historically been releasable as open weights, so the real wait is for the day this runs on a cluster like our dual DGX Sparks. Until then it is an API play.

Should you build on it now?

If your workflows are video or audio heavy and your harness is already agentic, yes, pilot it now through Qwen-MM-Plugins, because the integration you build is model-agnostic not waste if a stronger successor lands next quarter. If your workload is plain text coding or document work, nothing here changes your stack; the text numbers are good but that race is not what this release is about. And if you need the highest raw video reasoning scores today, Gemini 3.8 Flash still leads most static benchmarks, though the agentic results show how quickly that lead evaporates once models are allowed to use tools.

Frequently asked questions

What is Qwen3.8-Omni and what does it do?

Qwen3.8-Omni-Flash is Alibaba's natively multimodal agentic model. It reasons over text, images, audio, and video in a single model and executes long-horizon agent tasks such as video editing, drama translation with voice cloning, meeting minutes, and music video production. It uses a sparse mixture-of-experts backbone with a 1M-token context window.

How much context does Qwen3.8-Omni-Flash support?

According to the Qwen Team report, the model pretrains at a native 256K context window and extends to 1M tokens after post-training, enabling long-form multimodal reasoning over hours of audio and video.

Is Qwen3.8-Omni better than Gemini 3.8 Flash?

It depends on the setting. Qwen leads on multimodal agent execution (71.0 vs 58.9 on WildClawBench-MM) and realtime dialogue (82.3 vs 65.6 on OmniVChat-Bench). Gemini 3.8 Flash keeps the lead on direct video reasoning benchmarks (71.0 vs 65.0 on Video-MME-v2) and on agentic web search (78.6 vs 74.0 on OmniGAIA). With an agent harness, Qwen flips the long-video result: 73.6 vs 70.7 on LVOmniBench.

What is Qwen-MM-Plugins?

Qwen-MM-Plugins is an open-source framework that adds native audio and video support to existing agent harnesses such as Qwen Code, OpenClaw, and Claude Code. It includes Omni-Caption, Omni-Video2Note, Omni-Memory for lazy-loading audiovisual content, and an Omni Skill Creator that turns video demonstrations into executable agent skills.

Can Qwen3.8-Omni improve another AI model by itself?

Yes. In the paper's autoresearch experiment, Qwen3.8-Omni-Flash improved Qwen2.5-Omni-3B Sichuan dialect recognition in 12 hours, cutting character error rate from 25.79% to 15.30%, a relative reduction of about 40.7%, with no human in the loop.