Mixture-of-Experts (MoE)
ES-MoE blocks with dynamic routing for adaptive computation, allocating experts on a per-instance basis.
YOLO-Master
A powerful MoE-accelerated framework for real-time object detection
Tencent Youtu Lab
Introduction
Existing Real-Time Object Detection (RTOD) methods commonly adopt YOLO-like architectures for their favorable trade-off between accuracy and speed. However, these models rely on static dense computation that applies uniform processing to all inputs regardless of scene complexity.
YOLO-Master introduces instance-conditional adaptive computation via Efficient Sparse Mixture-of-Experts (ES-MoE) blocks. It dynamically allocates computational resources according to scene complexity, outperforming state-of-the-art models on large-scale benchmarks including COCO, VOC, VisDrone, KITTI, and SKU-110K.
Our framework is the first deep integration of Mixture-of-Experts (MoE) into YOLO architecture, enabling dynamic intelligence for vision tasks. By routing each instance to a specialized subset of experts, YOLO-Master achieves superior accuracy-efficiency trade-offs across diverse scenarios ranging from autonomous driving to retail analytics.
Core Architecture
The YOLO-Master architecture consists of three main components: a lightweight backbone with ES-MoE blocks, a feature pyramid neck, and a detection head with dynamic routing.
ES-MoE blocks enable compute-on-demand via dynamic routing. Each block contains multiple expert networks and a lightweight router that determines which experts to activate for each input instance. This sparse activation pattern reduces computational overhead while maintaining expressive power.
The router network analyzes intermediate feature representations and produces a sparse gating distribution over experts. Only the top-k experts are activated per instance, enabling significant speedups on less complex scenes while preserving accuracy on challenging inputs.
Key Features
Six core innovations powering the next generation of real-time detection
LoRA Fine-Tuning
Parameter-efficient fine-tuning with ~10% trainable parameters, enabling rapid domain adaptation.
Sparse SAHI
3–5× speedup on 4K/8K images via content-adaptive slicing, skipping redundant regions.
Cluster-Weighted NMS
Gaussian-weighted fusion for dense scenes, improving mAP in crowded environments.
Mixture-of-Attention (MoA)
CNN-native multi-scale attention fusion, bridging local inductive bias and global context.
Agent Skill System
9 skill runners for AI agent orchestration, enabling autonomous vision pipelines.
Quick Start
Get up and running in under 5 minutes
Installation
Bash
conda create -n yolo_master python=3.11 -y
conda activate yolo_master
git clone https://github.com/Tencent/YOLO-Master.git
cd YOLO-Master
pip install -r requirements.txt
pip install -e .Python Inference
Python
from ultralytics import YOLO
model = YOLO("yolo_master_n.pt")
results = model("path/to/image.jpg")
results[0].show()Citation
If you find YOLO-Master useful in your research, please consider citing our paper
BibTeX
@inproceedings{lin2026yolomaster,
title={{YOLO-Master}: MOE-Accelerated with Specialized Transformers for Enhanced Real-time Detection},
author={Lin, Xu and Peng, Jinlong and Gan, Zhenye and Zhu, Jiawen and Liu, Jun},
booktitle={CVPR},
year={2026}
}FAQ
What is ES-MoE?
ES-MoE (Efficient Sparse Mixture-of-Experts) is our novel building block that replaces standard convolutional layers in YOLO. It routes each instance to a subset of expert networks based on learned gating, enabling adaptive computation.
Which datasets are supported?
YOLO-Master supports COCO, VOC, VisDrone, KITTI, SKU-110K, and any custom dataset compatible with the YOLO format. Pre-trained weights are available for all model sizes (N/S/M/L/X).
Can I use MoE with my existing YOLO models?
Yes! YOLO-Master is designed as a drop-in replacement. You can use our ES-MoE blocks in your existing YOLO pipelines with minimal code changes. See the Docs for integration guides.