AFD Plugin Explained

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

afd-plugin is a vLLM external plugin for AFD. The attention part and the feed-forward (FFN/MoE) part of a model run in two separate server processes, and a connector moves hidden states between them. The plugin adds its own worker classes, model runners, model wrappers, connectors, config validation and compatibility patches, and it changes no vLLM source file (README.md:9-13, README.md:21-24). This site explains the plugin code at commit 7452d52, and the vLLM code it builds on at commit ced6857. You need to know Python and the basics of LLM serving.

At a glance#

The system map#

One decode step runs left to right, then back. The attention server computes attention and hands its hidden states to the FFN server through the connector. The FFN server runs the experts and hands the result back. The colour stripe shows where the code runs. Each box links to its page and to the file.

Start here#

Pick the path that fits what you know.

• Path A — new to AFD. Start with What is AFD?, then How vLLM plugins work and The vLLM v1 map. Then read Attention and FFN roles: what each server owns and the tour Tour: Life of a request.
• Path B — you know vLLM internals. Go straight to The plugin entry point: register_afd, Worker selection: choosing the role worker and The connector contract: the Attention/FFN transport. Then follow Tour: One decode step.
• Path C — you want to deploy. Read Deployment topologies: how to place the roles and Tuning: the knobs the repo exposes, then Tour: Run a recipe and the Recipes reference table.

How to use this site#

• Levels. Each page shows a level dot: Beginner, Intermediate or Advanced.
• Two depths. Each page starts simple, then has a Deep dive with shapes, numbers and a line-by-line code reading. Use "Open all deep dives" to expand them.
• Check yourself. Most pages end with a short quiz.
• Search. Press / to search. Hover a glossary term for its tooltip.
• Code links. Every file:line link opens the exact code at the pinned commit.
• Grey "soon" entries in the sidebar are pages that are not written yet.

What is not in this release#

This site has limits of scope, and the plugin itself states limits.

Out of the site's scope.

• The C/C++ kernel sources in csrc/. The site explains only how the Python code selects, builds, loads and calls them.
• The Ascend NPU code, except the one overview page Ascend NPU overview: the same contract, different transports. Other pages name the NPU variant in one line only.
• The tests. They are a source of facts, not a page topic.

What the repository calls experimental or limited.

• The project is experimental and needs more large-scale testing (README.md:17-19).
• The GPU target is exactly vLLM 0.30.0; other versions are not supported on GPU (README.md:69-70).
• GPU CUDA graphs are limited to FULL_DECODE_ONLY (README.md:86).
• Native DBO is limited to exactly two ubatches (README.md:87).
• Qwen3-MoE rejects attention-side gate placement, sequence-parallel MoE, EPLB, pipeline parallelism, speculative decoding, LoRA and NPU (README.md:91-92).
• The GPU and NPU end-to-end tests are opt-in and need real hardware and model weights (README.md:85).

The code version this site explains#

This site explains one exact commit of each repository. Use these versions when you study the code with the site. The plugin pins vLLM to one release: vllm==0.30.0 (pyproject.toml:37), and the README names the exact commit (README.md:21). So every line number on this site is stable only at these two commits.

To get exactly this version of the code, run these commands:

git clone https://github.com/vllm-project/afd-plugin.git
cd afd-plugin
git checkout 7452d52748fec2a858b63e4cb22941dadd7a3ee3

The site also cites vLLM. Citations with the prefix vllm: point to this commit:

git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout ced6857afa0ea7b2e3f0846a62e1394e90f15607