Papers

Filtered to mixture-of-experts · clear filter

Browse by term

continual learning 89reinforcement learning 43vision-language models 19benchmarking 12benchmarks 12robotics 12language models 9world models 9large language models 8video generation 7multimodal models 5recursive self-improvement 5video world models 5agents 4diffusion models 4embodied AI 4foundation models 4generative models 4human-computer interaction 4multimodal large language models 4natural language processing 4policy optimization 4reasoning 4self-supervised learning 4spatial reasoning 4agentic models 3conversational AI 3dialogue systems 3evaluation benchmarks 3LLMs 3

Matching papers

IntBMoE: Integrating Block-Level Conditioning into Expert Composition for Full-Participation Mixture-of-Experts

92 upvotes · 18 SEP 2026 · Ran Cheng, Longfei Xu, Zheng Liu et al.

This paper proposes a new architecture for Mixture-of-Experts (MoE) models that balances participation, execution, and materialization costs. Practitioners might care because it can lead to significant performance gains in applications where memory and computational resources are limited.

SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers

86 upvotes · 1 SEP 2026 · Shaowen Wang, Ge Zhang, Kairong Luo et al.

This paper investigates how to make Looped Transformers more efficient by scaling their depth while keeping the same number of parameters and compute usage, and it finds that this approach can lead to significant performance gains. Practitioners might care about this because it could lead to faster and more accurate models in resource-constrained environments.