AFD Plugin Explained

The code map: folders, sizes and dependencies

Code mapabout 5 min read

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

This page maps the repository. It shows the folder tree with counts, which folder depends on which, and how much of the Python code is GPU, NPU or shared. The module pages then read the most important files.

Purpose#

afd-plugin is one Python package, afd_plugin, plus kernels, recipes, docs and tests. The package is what vLLM loads. This map helps you find a file and see where it sits before you open a module page.

The folder tree#

The tree below is generated from the repository at the pinned commit. The counts are tracked files and text lines per folder. Out-of-scope folders are marked.

595 tracked files, 139,046 lines (text files only; counted with git ls-tree at the pinned commit).

  • .agents/ 95 files · 10,124 lines
  • .buildkite/ 9 files · 1,437 lines
  • .claude/ 1 files · 1 lines
  • .github/ 9 files · 396 lines
  • afd_plugin/ 82 files · 20,214 lines Shared. The plugin Python package that vLLM loads. Holds the GPU and NPU code.
    • afd_plugin/compat/ 20 files · 3,971 lines Shared. Compat patches that replace vLLM functions at load time. The npu/ subfolder is NPU.
    • afd_plugin/connectors/ 12 files · 4,333 lines Mixed. The connector contract and factory (shared); gpu/ is GPU and npu/ is NPU. page
    • afd_plugin/distributed/ 3 files · 359 lines Shared. The AFD process group, the topology and the role-rank helpers.
    • afd_plugin/model_executor/ 20 files · 5,257 lines Mixed. The AFD model wrappers (shared/GPU); the npu/ subfolder is NPU.
    • afd_plugin/v1/ 20 files · 5,036 lines Mixed. The role workers, the model runners, DBO and the CUDA graph code; worker/npu is NPU.
  • csrc/ 179 files · 47,057 lines Out of scope. C/C++ kernel sources. gpu/ is a GPU stub; npu/ is the Ascend operator sources.
    • csrc/gpu/ 1 files · 3 lines GPU. A stub (one file).
    • csrc/npu/ 177 files · 47,046 lines NPU (Ascend). The prebuilt operator sources. Out of scope: only how the Python code loads them.
  • docker/ 1 files · 59 lines
  • docs/ 21 files · 4,975 lines Design notes and user guides. Written against an older commit: use as hints, not as a source.
    • docs/assets/ 3 files · 242 lines
    • docs/design/ 9 files · 2,898 lines
    • docs/gpu/ 1 files · 217 lines GPU. The P2P connector user guide.
    • docs/npu/ 8 files · 1,618 lines NPU (Ascend). The CAM connector user guides and notes.
  • recipe/ 24 files · 2,441 lines Launch scripts per connector and model. gpu/ is GPU and npu/ is NPU.
    • recipe/gpu/ 12 files · 1,003 lines GPU. Launch scripts for the P2P connector.
    • recipe/npu/ 11 files · 1,387 lines NPU (Ascend). Launch scripts for the CAM connectors.
  • tests/ 149 files · 42,186 lines Source of facts only. e2e/ and unit/ are shared; npu/ is NPU.
  • tools/ 11 files · 2,332 lines Shared. Repository tooling: benchmarks, datasets and pre-commit checks.

How the folders depend on each other#

The Python imports run in one direction, from the entry point down to the distributed helpers. The main edges are:

  1. Entry point. afd_plugin/__init__.py imports the config (afd_plugin/__init__.py:16), then the compat patches, the routing simulator and the DBO op (afd_plugin/__init__.py:172-192).
  2. Config and compat. The config_validation patch reads the AFD config and the validation helpers (afd_plugin/compat/patches/config_validation.py:20-24).
  3. Workers and model wrappers. The runners in v1/worker and the model wrappers in model_executor import the connectors (afd_plugin/v1/worker/attention_model_runner_v2.py:37, afd_plugin/model_executor/models/deepseek_v2.py:25).
  4. Connectors. The connectors read the config and resolve the role rank from the distributed helpers (afd_plugin/connectors/factory.py:11-17).
  5. Distributed. afd_plugin/distributed holds the process group, the topology and the role-rank helpers; it is at the bottom of the chain.

So the order is: entry point → config and compat → workers and model wrappers → connectors → distributed.

The GPU / NPU / shared split#

About half of the Python is NPU-specific, and only a small part is GPU-specific. The rest is shared.

GroupFiles (.py)Lines
Shared (not in a gpu/ or npu/ folder)429,734
NPU (Ascend)349,329
GPU (CUDA)21,132
Total afd_plugin/7820,195

How the numbers were measured#

The line counts above and in the tree come from the repository at the pinned commit, not from a tool report. The method:

  1. List tracked files with git ls-tree -r --name-only at the pinned commit.
  2. Keep the .py files under afd_plugin/ for the split; count the newlines in each one (a file with no trailing newline still counts its last line).
  3. Classify each file: NPU when a path part is npu, GPU when a path part is gpu, else shared.

The folder tree uses the same git ls-tree file list and newline count, over all text files, not only Python. The site tool tools/gen_folder_tree.py does this and writes the tree between the markers. Binary files count as 0 lines.

The module pages#

These eight pages read the most important modules: