Dual-Stream MoE Unifies Multimodal, Garment Video 30x Faster

Featured

01 Unified Multimodal That Doesn't Just Scale Parameters

Two paths dominate unified multimodal today: scale to hundreds of billions, or treat text-image as primary with video and editing bolted on. ByteDance's Lance takes a third route. Train a lightweight native unified model from scratch, with image and video understanding, generation, and editing all in one architecture.

The core move is dual-stream MoE. Understanding and generation each get their own expert paths, but share the same interleaved multimodal context. Position encoding is modality-aware to dampen interference between different visual tokens. The engineering bet: shared context preserves multi-task semantic coherence, while separate experts stop understanding and generation from overwriting each other in a single weight set. Capacity goes into distinguishing capability paths, not stacking depth. Compared with the scaling route, the difference isn't "bigger." It's "more structured."

Training is staged with data scheduled by capability target. Understanding gets locked in first, then generation and editing pile on. Task priority is explicit rather than left to a shared weight set to sort out. The abstract claims clear leadership among open-source unified models on image and video generation, with understanding intact. For teams that want unified multimodal without a giant cluster, open-source plus lightweight means this architecture can be reproduced inside a billion-parameter budget rather than a trillion-parameter one. The experiment threshold drops by an order of magnitude. The abstract gives no comparison against closed-source frontier models, so the real ceiling waits on community reproduction.

Key takeaways: - Joint multi-task training is an alternative to scaling parameters for unified multimodal. Teams that can't afford giant clusters get a new reference point. - Dual-stream MoE plus modality-aware position encoding is the key tradeoff. Shared context, separate paths. - The paper offers no closed-source comparison. Real capability ceiling waits on reproduction.

Source: Lance: Unified Multimodal Modeling by Multi-Task Synergy

02 Train on Single Garment, Swap Many at Runtime

The commercial pull on human video customization has been sitting there for a while. E-commerce and content creation both want it. But garment-level fine-grained control stayed offline. Every swap meant regenerating from scratch, so interactivity was off the table.

FashionChameleon picks a counterintuitive training path. Skip multi-garment video collection entirely. Train a Teacher Model on single-garment video only. By forcing the reference image and the target garment image to mismatch during training, the model implicitly learns to keep motion coherent across swaps. Streaming distillation handles the streaming output side. A training-free KV cache schedule manages mid-inference swaps, so multi-garment scenarios never require their own training pass.

The result: 23.8 FPS on a single GPU, 30 to 180x faster than existing baselines. The number matters less than the shift. Garment video customization just moved from batch rendering to a position where real interaction is possible.

Key takeaways: - "Train on one, control many" via reference-garment mismatch teaches coherent swaps implicitly. The pattern transfers to other object-level video customization tasks. - Swap switching runs through a training-free KV cache schedule. Inference-time engineering solves more than usually assumed. - Human video customization just crossed from offline render to interactive. Content creation and e-commerce tool teams have product-shape room to redesign.

Source: FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customization

03 Fold Multi-Step GRPO Into One, Video Alignment Gets Affordable

The GRPO video alignment bottleneck has never been algorithm choice. A 14B model means hundreds of GPU-days per experiment. That financial wall decides which teams can play and which only watch.

Existing compute-saving routes like sliding-window subsampled timesteps sacrifice stability badly and don't match full-trajectory quality. Flash-GRPO bets the other direction. Compress the entire multi-step trajectory into one-step policy optimization, then patch the obvious failure modes. Two corrections do the work: iso-temporal grouping forces the same prompt to be compared at consistent timesteps, killing the variance from "timestep difficulty." Temporal gradient rectification offsets the magnitude distortion across timesteps.

The paper reports stable convergence from 1.3B to 14B, with low-budget runs beating the full-trajectory baseline. Worth flagging: this is the paper's claim, and the low-compute comparison is also against a low-compute baseline. Independent reproduction is what to watch. If it holds, the bar for video alignment shifts from "scrape together the hardware" to "tune the hyperparameters."

Key takeaways: - Whether a team can attempt video alignment splits on the per-experiment compute bill, not the algorithm choice. - Both corrections — iso-temporal grouping and temporal gradient rectification — need to survive reproduction. Their stability decides whether the path is real. - "Beats full trajectory at low compute" compares against low-compute baselines. Cross-budget conclusions wait on the full paper.

Source: Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization

04 Continual Agent Learning Without LLM Fine-Tuning

The direct path for getting agents better at problems is fine-tuning the base LLM. Application teams mostly can't get that compute budget. Solvita points to another route.

Each of four agents — Planner, Solver, Oracle, Hacker — gets its own trainable graph knowledge network. After every task, pass/fail signals, test coverage quality, and Hacker counterexamples feed back as RL updates to network weights. The base LLM never moves. Experience accumulation shifts out of prompt context and into a separate trainable state. Over time, the system learns which problem class to route to which solving strategy.

The paper reports SOTA across four benchmarks, including CodeContests and APPS. Whether this transfers to your task distribution depends on feedback signal density. Competitive programming pass/fail is a strong, verifiable signal. Most business workflows don't produce anything that clean.

Key takeaways: - Continual agent learning without touching base LLM weights. Encode experience into a separate trainable network. - "External memory plus RL update" is a transferable engineering pattern for teams without fine-tune budget. - The real constraint is feedback signal density. Competition pass/fail is strong. Business scenarios mostly aren't this verifiable.

Source: Solvita: Enhancing Large Language Models for Competitive Programming via Agentic Evolution