Glossary

Referenceabout 12 min read

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

This page lists every term that has a tooltip on this site. Each entry gives a short definition. Many entries link to a page with the full explanation.

A B C D E F G H L M P R S U V W X

A#

ACL graph also: ACL graph mode, Ascend graph

The Ascend NPU form of a captured graph, the NPU counterpart of a CUDA graph. The NPU validator accepts the FULL and FULL_DECODE_ONLY modes.

The NPU runtime is covered on Ascend NPU overview: the same contract, different transports. The mode check is afd_plugin/validation.py:233-239.

additional_config also: --additional-config

A vLLM config field. The plugin reads its "afd" key for the role, the connector, the topology sizes and the host/port.

AFD also: Attention-FFN Disaggregation, attention-ffn disaggregation

Attention-FFN Disaggregation. The attention part and the feed-forward (FFN/MoE) part of a model run in separate server processes and exchange hidden states.

B#

BATCH_SIZE_FACTOR also: CAM receive capacity

An Ascend NPU environment variable that multiplies the CAM receive capacity (base 262144 tokens), with native float rounding. Default 1.

Read in afd_plugin/validation.py:45-54 and afd_plugin/compat/npu/feature_validation.py:245-249. It has no effect on GPU.

C#

CAM

The Ascend NPU communication operators used by the NPU connectors. One connector is synchronous, the other asynchronous.

citation

A link to a file and line in the code. Every claim about the code on this site has one.

D#

data plane

The path that carries the hidden-state tensors between the two roles. It is the connector's send and receive calls.

DBO also: dual-batch overlap

Dual-batch overlap. The step is split into two microbatches so compute and communication overlap. AFD uses exactly two.

DBO yield also: manual_dbo_yield, dbo yield

A custom op that hands control to the other ubatch thread. The AFD proxy calls it between the send and the receive, so the second stage runs while the first waits.

Registered as torch.ops.vllm.manual_dbo_yield (afd_plugin/v1/worker/dbo.py:27-49). It yields only when vLLM DBO is active (afd_plugin/v1/worker/dbo.py:52-54).

DP metadata also: data-parallel metadata

The per-step data-parallel metadata the attention side sends to the FFN side on the control plane. It carries the per-stage token counts that set the tensor shapes.

E#

EngineCore

The vLLM v1 engine core process. On the FFN role it returns early after the executor is built: no scheduler, no loop.

entry point

A Python packaging hook. vLLM reads the vllm.general_plugins entry points and calls each one at start-up.

afd-plugin declares afd = "afd_plugin:register_afd" under vllm.general_plugins (pyproject.toml:45-46).

F#

FFN graph key also: make_ffn_graph_key, graph key

A hashable key built from the per-stage DP metadata. The FFN runner uses it to store and look up one captured CUDA graph per token shape.

Built by make_ffn_graph_key (afd_plugin/v1/worker/cuda_graph.py:104-133).

FFN role also: FFN side

The side that runs the feed-forward and MoE experts. It receives hidden states, computes, and sends the result back.

FFN server

The vllm serve process that plays the FFN role. It has no API server, no scheduler and no KV cache; a connector loop drives it.

FFN server loop also: FFN loop, server loop

The connector-driven daemon thread on the FFN worker. It waits for DP metadata, then runs the FFN compute for one step. No scheduler drives it.

Started in AFDFFNWorker.start_ffn_server_loop (afd_plugin/v1/worker/ffn_worker.py:133); the loop body is _run_ffn_server_loop (afd_plugin/v1/worker/ffn_worker.py:160).

G#

general plugin

A vLLM plugin in the vllm.general_plugins group. vLLM loads it with load_general_plugins in several processes.

H#

HCCL also: Huawei Collective Communication Library

Huawei Collective Communication Library, the Ascend NPU collective transport. The CAMP2p connector carries hidden states over HCCL and DP metadata over a smaller Gloo group.

L#

load_general_plugins also: load_general_plugins()

The vLLM function that loads every plugin in the vllm.general_plugins group, then calls each one. It runs once per process.

Defined in vLLM (vllm:vllm/plugins/__init__.py:77-90); a guard stops a second run in the same process.

M#

Model Runner V2

The vLLM v1 model runner, version 2. Both AFD roles need it on; the README sets VLLM_USE_V2_MODEL_RUNNER=1. AFD adds a seam over it.

model wrapper

An AFD<arch> model class that makes a model role-aware. On the attention side the FFN stage becomes a remote proxy.

ModelRegistry also: ModelRegistry.register_model

vLLM's registry of model architectures. A plugin adds a model with register_model, using a lazy "module:Class" string.

The singleton is defined in vLLM (vllm:vllm/model_executor/models/registry.py:1523-1531); register_afd calls it for each AFD model (afd_plugin/__init__.py:211-214).

multipod also: multi-pod, multipod deployment

A Kubernetes deployment that puts one AFD role per pod. Each vllm serve is gated on a POD environment variable, so with POD unset nothing runs.

P#

P2P subgroup

In the GPU P2P connector, one FFN rank plus the attention ranks mapped to it. The FFN rank is subgroup rank 0.

patch also: compat patch

A compat module that replaces a vLLM function at load time. The plugin applies a small set of patches.

prefill/decode disaggregation also: PD disaggregation, prefill-decode disaggregation

A serving layout that runs prefill and decode in separate processes. Each process still runs the whole model. It splits by phase, not by model part.

R#

register_afd() also: register_afd

The plugin entry function. It applies the compat patches and registers the routing strategy, the DBO op and the model wrappers.

Defined in the package init (afd_plugin/__init__.py:142); it guards against a second run.

remote experts also: remote experts shell, AFDDeepseekV2RemoteExpertsMoE

A parameter-free MoE shell on the attention role. It keeps the native MoE forward but sends the expert compute to the FFN role.

On CUDA, every MoE layer on the attention side builds AFDDeepseekV2RemoteExpertsMoE, a native DeepseekV2MoE with its experts replaced by a proxy (afd_plugin/model_executor/models/deepseek_v2.py:277-335).

RemoteFFNProxy

A parameter-free module on the attention side. It replaces the FFN stage of a decoder layer and uses the connector instead.

Defined for DeepSeek-V2 at afd_plugin/model_executor/models/deepseek_v2.py:117.

rendezvous

The host and port where the two roles meet to set up the connector process groups. Both servers use the same host and port.

The connector opens a TCP store at host:port (afd_plugin/connectors/gpu/p2p.py:271).

role rank

The rank of a process inside its own role group. It comes from the DP, PCP and TP placement.

Resolved by resolve_role_rank (afd_plugin/distributed/topology.py:61).

S#

scoped patch also: scoped monkey-patch, context-manager patch

A patch that is active only inside a context manager, for the time of one AFD call. The code restores the original function on exit.

Example: use_gpu_mrv2_ubatch_factory and use_gpu_mrv2_dbo_dispatch scope the DP1 DBO seams (afd_plugin/compat/patches/gpu_mrv2_ubatching.py:263-282).

stage

The DBO ubatch index of one transfer on the wire. A connector sends one transfer per layer and stage.

The transfer metadata carries stage_idx next to layer_idx (afd_plugin/connectors/metadata.py:188-190).

step

One engine iteration on the attention server: the scheduler picks a batch, the model runs all layers once, and vLLM samples. The FFN server follows each step.

The attention-side EngineCore runs one step in EngineCore.step (vllm:vllm/v1/engine/core.py:589-601). The control plane runs once per step; the data plane runs once per layer and stage.

U#

ubatch also: microbatch

A microbatch: one of the two token groups that DBO runs at the same time. On the wire its index is the stage.

V#

version gate also: is_target_vllm_compatible, install gate

A check that decides whether a patch installs. is_target_vllm_compatible allows the target vLLM release and any dev build; on a mismatch, no patch installs.

Defined in is_target_vllm_compatible (afd_plugin/compat/vllm.py:53-68). Different from the strict start-up check assert_vllm_version_supported (afd_plugin/compat/vllm.py:38-50).

W#

worker_cls

The vLLM parallel-config field that names the worker class. The plugin rewrites it, but only when it is "auto".

X#

xAyF also: 2A2F, 4A4F

A topology name: x attention ranks and y FFN ranks, for example 2A2F. Use it only in topology names, not in prose.