AFD Plugin Explained

afd_plugin/__init__.py: the plugin entry module

IntermediateCode mapabout 5 min read + 1 min deep dives

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

In one sentence

This module is the plugin entry point. register_afd applies the compat patches and registers the routing strategy, the DBO op and the AFD models.

Purpose#

vLLM calls register_afd once per process through the vllm.general_plugins entry point. The function installs AFD and then stops; it runs no model and keeps no connection.

Fileafd_plugin/__init__.py (234 lines, afd-plugin).
Main functionsregister_afd (afd_plugin/__init__.py:142), _force_spawn_multiprocessing_if_requested (afd_plugin/__init__.py:51), __getattr__ (afd_plugin/__init__.py:19).
Runs inEvery process that loads general plugins: process 0, the engine core, each worker, and the model-inspection subprocess. See How vLLM plugins work.
Called byvLLM's load_general_plugins, through the entry point afd = "afd_plugin:register_afd" (pyproject.toml:45-46).
CallsThe four compat patch modules, the routing simulator, the DBO custom op, the NPU Ascend config patch (NPU only), and vLLM's ModelRegistry (afd_plugin/__init__.py:172-214).
Not in this fileThe patch bodies, the connector code, and the worker classes. The worker classes are reached lazily through __getattr__ (afd_plugin/__init__.py:19-33).

Where it sits#

Figure: register_afd runs safety checks first (any can return early), then the registration steps in order. Step 8 runs only on NPU.
Open full size ↗

Who calls it#

Only vLLM calls it, through the entry point (pyproject.toml:45-46). The call sites and processes are on the How vLLM plugins work page.

What it calls#

It imports the four patch modules (afd_plugin/__init__.py:172-176), the routing simulator (afd_plugin/__init__.py:183-185), the DBO op (afd_plugin/__init__.py:190), and, on NPU, the Ascend config patch (afd_plugin/__init__.py:205-207). It ends by calling vLLM's ModelRegistry.register_model (afd_plugin/__init__.py:211-214).

Key classes and functions#

NameLineRole
register_afdafd_plugin/__init__.py:142 The entry point. Applies the patches and registers the models.
_force_spawn_multiprocessing_if_requestedafd_plugin/__init__.py:51 Opt-in: force the spawn start method and redirect forkserver.
__getattr__afd_plugin/__init__.py:19 Lazy access to the worker classes and assert_compatible_afd_stack.
_MODEL_REGISTRATIONSafd_plugin/__init__.py:133-139 The read-only map from a vLLM architecture name to an AFD model class.
__version__afd_plugin/__init__.py:36-44 The package version, with a fallback that never raises.

Walk through register_afd#

The function runs in two parts: safety checks that can return early, then the registration steps.

Part 1: Guard, import safety and version check#

afd_plugin/__init__.py:150-170GitHub ↗
    global _registered
    if _registered:
        _logger.debug("AFD plugin: register_afd() already completed")
        return

    _force_spawn_multiprocessing_if_requested()
    _logger.debug("AFD plugin: register_afd() called")
    if importlib.util.find_spec("vllm") is None:
        _logger.debug("AFD plugin: vLLM not found, skipping runtime registration")
        _registered = True
        return

    try:
        from afd_plugin.compat.vllm import assert_vllm_version_supported

        assert_vllm_version_supported(strict=False)
    except Exception:
        _logger.debug(
            "AFD plugin: vLLM version check could not be completed",
            exc_info=True,
        )
The guard, the spawn override, the vLLM check and the version check.
  1. Line 151. The guard returns if _registered is already set, so the registration body runs once per process (afd_plugin/__init__.py:150-153).
  2. Line 155. It applies the opt-in spawn override again (it also ran at import time).
  3. Line 157. If vLLM is not importable, it sets _registered and returns. So importing the package stays safe on a machine with no vLLM (afd_plugin/__init__.py:157-160).
  4. Line 165. It checks the vLLM version with strict=False, so a mismatch gives a warning, not an error (afd_plugin/compat/vllm.py:38-50). The whole check sits in a try/except (afd_plugin/__init__.py:162-170).

Part 2: Patches, routing, DBO, the NPU patch and the models#

afd_plugin/__init__.py:172-197GitHub ↗
    try:
        import afd_plugin.compat.patches.async_dp_engine  # noqa: F401
        import afd_plugin.compat.patches.async_dp_forward_context  # noqa: F401
        import afd_plugin.compat.patches.config_validation  # noqa: F401
        import afd_plugin.compat.patches.engine_core  # noqa: F401
    except Exception:
        _logger.debug(
            "AFD plugin: compatibility patches could not be applied",
            exc_info=True,
        )

    from afd_plugin.model_executor.routing_simulator import (
        register_afd_balanced_routing_strategy,
    )

    register_afd_balanced_routing_strategy()

    try:
        from afd_plugin.v1.worker.dbo import register_dbo_yield_custom_op

        register_dbo_yield_custom_op()
    except Exception:
        _logger.debug(
            "AFD plugin: DBO yield custom op could not be registered",
            exc_info=True,
        )
Import the patches, register the routing strategy, then the DBO custom op.
  1. Lines 172-176. It imports the four patch modules. Importing a patch module applies it. The import sits in a try/except (afd_plugin/__init__.py:172-181).
  2. Line 187. It registers the AFD routing strategy. This call is not wrapped (afd_plugin/__init__.py:183-187).
  3. Line 192. It registers the DBO yield custom op, inside a try/except (afd_plugin/__init__.py:189-197).
afd_plugin/__init__.py:199-216GitHub ↗
    # EngineCore constructs Ascend schedulers before handshake revalidation or
    # AFD worker startup. Install the namespace factory at plugin registration
    # so direct scheduler calls and already imported aliases accept AFD settings.
    from vllm.platforms import current_platform

    if current_platform.device_type == "npu":
        from afd_plugin.compat.patches.npu.ascend_config import (
            apply_afd_ascend_config_patch,
        )

        apply_afd_ascend_config_patch()

    from vllm.model_executor.models import ModelRegistry

    for model_arch, model_cls in _MODEL_REGISTRATIONS.items():
        ModelRegistry.register_model(f"AFD{model_arch}", model_cls)

    _registered = True
The NPU-only Ascend patch, the model registration loop, and the final flag.
  1. Line 204. It applies the Ascend config patch only when the platform device type is "npu" (afd_plugin/__init__.py:202-209). This step is not wrapped.
  2. Lines 211-214. It registers each model under its AFD-prefixed name (afd_plugin/__init__.py:211-214).
  3. Line 216. It sets _registered = True, so a later call in the same process returns early.

The model registration table#

The map keys are vLLM architecture names; the loop registers each one under the name "AFD" + key (afd_plugin/__init__.py:211-214). So DeepseekV2ForCausalLM becomes AFDDeepseekV2ForCausalLM.

afd_plugin/__init__.py:96-117GitHub ↗
_DEEPSEEK_MODEL_REGISTRATIONS = {
    "DeepseekForCausalLM": (
        "afd_plugin.model_executor.models.deepseek_v2:AFDDeepseekForCausalLM"
    ),
    "DeepseekV2ForCausalLM": (
        "afd_plugin.model_executor.models.deepseek_v2:AFDDeepseekV2ForCausalLM"
    ),
    "DeepseekV3ForCausalLM": (
        "afd_plugin.model_executor.models.deepseek_v2:AFDDeepseekV3ForCausalLM"
    ),
    "DeepseekV32ForCausalLM": (
        "afd_plugin.model_executor.models.deepseek_v2:AFDDeepseekV3ForCausalLM"
    ),
    "DeepseekV4ForCausalLM": (
        "afd_plugin.model_executor.models.npu.deepseek_v4:AFDDeepseekV4ForCausalLM"
        if importlib.util.find_spec("torch_npu") is not None
        else "afd_plugin.model_executor.models.deepseek_v4:AFDDeepseekV4ForCausalLM"
    ),
    "GlmMoeDsaForCausalLM": (
        "afd_plugin.model_executor.models.deepseek_v2:AFDGlmMoeDsaForCausalLM"
    ),
}
The DeepSeek registrations. DeepseekV4 picks the NPU or GPU class by torch_npu.
Registered nameResolves toNote
AFDDeepseekForCausalLMdeepseek_v2:AFDDeepseekForCausalLM
AFDDeepseekV2ForCausalLMdeepseek_v2:AFDDeepseekV2ForCausalLM
AFDDeepseekV3ForCausalLMdeepseek_v2:AFDDeepseekV3ForCausalLM
AFDDeepseekV32ForCausalLMdeepseek_v2:AFDDeepseekV3ForCausalLM V3.2 reuses the V3 class (afd_plugin/__init__.py:106-108).
AFDDeepseekV4ForCausalLM npu.deepseek_v4:... or deepseek_v4:... NPU class when torch_npu is importable, else the GPU class (afd_plugin/__init__.py:109-113).
AFDGlmMoeDsaForCausalLMdeepseek_v2:AFDGlmMoeDsaForCausalLM
AFDQwen3MoeForCausalLMqwen3_moe:AFDQwen3MoeForCausalLM
AFDQwen3_5MoeForConditionalGeneration qwen3_5:AFDQwen3_5MoeForConditionalGeneration

The spawn override#

The override is opt-in through one environment variable. It changes Python's default multiprocessing context, for an Ascend async recipe that cannot use forkserver.

afd_plugin/__init__.py:51-62GitHub ↗
def _force_spawn_multiprocessing_if_requested() -> None:
    """Pin Python's default context for the A3 Python 3.12 runtime.

    ``VLLM_WORKER_MULTIPROC_METHOD`` controls vLLM's explicit context only.
    Some NPU runtime helpers use Python's default context and otherwise retain
    the task's inherited ``forkserver`` setting, which cannot restore the
    signal-handler state in this environment.  Keep the override opt-in so it
    remains isolated to the affected CAM async recipe.
    """
    if os.environ.get("AFD_FORCE_SPAWN_MULTIPROCESSING") != "1":
        return
    multiprocessing.set_start_method("spawn", force=True)
The guard and the start-method change.
  1. Line 60. It returns at once unless AFD_FORCE_SPAWN_MULTIPROCESSING is "1" (afd_plugin/__init__.py:60).
  2. Line 62. It forces the default start method to spawn.
  3. It then redirects the forkserver context and get_context("forkserver") to the spawn context (afd_plugin/__init__.py:70-87).

It runs once at import time (afd_plugin/__init__.py:94) and again inside register_afd (afd_plugin/__init__.py:155). The import-time call reaches the Ascend workers, which import the worker class directly and do not call the entry point (afd_plugin/__init__.py:90-94).

Deep dive#

Deep dive

Error paths: what is caught, and what is not

Four steps swallow their own exceptions, and three do not. The difference decides which failures are quiet and which stop the process.

StepLinesOn failure
Version checkafd_plugin/__init__.py:162-170Caught: logged at debug.
Import the 4 patchesafd_plugin/__init__.py:172-181Caught: logged at debug.
DBO custom opafd_plugin/__init__.py:189-197Caught: logged at debug.
Routing strategyafd_plugin/__init__.py:183-187Not caught: the error propagates.
NPU Ascend patchafd_plugin/__init__.py:199-209Not caught: the error propagates.
Model registrationafd_plugin/__init__.py:211-214Not caught: the error propagates.

The version-check exception is caught twice: the helper warns instead of raising with strict=False, and the call also sits in a try/except (afd_plugin/compat/vllm.py:48-50).

Deep dive

Import safety and environment variables

Import safety. Importing afd_plugin must not need vLLM, so local CPU tests work. Two things make it safe: register_afd returns early when find_spec("vllm") is None (afd_plugin/__init__.py:157-160), and __version__ falls back to a setuptools_scm value or "0.0.0+unknown" (afd_plugin/__init__.py:36-44).

VariableRead atEffect
AFD_FORCE_SPAWN_MULTIPROCESSINGafd_plugin/__init__.py:60 "1" forces the spawn start method and redirects forkserver. Any other value does nothing.

Check yourself#

  1. 1
    Under what name is DeepseekV2ForCausalLM registered?
    easy
    Show answer

    AFDDeepseekV2ForCausalLM. The loop registers each key under "AFD" + key (afd_plugin/__init__.py:211-214).

  2. 2
    Which step runs only on NPU?
    easy
    Show answer

    The Ascend config patch. It runs only when the platform device type is "npu" (afd_plugin/__init__.py:202-209).

  3. 3
    What happens when register_afd runs on a machine with no vLLM installed?
    medium
    Show answer

    It sets _registered = True and returns before any registration, so the import stays safe (afd_plugin/__init__.py:157-160).

  4. 4
    If registering a model raises, does the process keep running? And if the version check raises?
    hard
    Show answer

    Model registration is not wrapped, so its error propagates (afd_plugin/__init__.py:211-214). The version check is wrapped in a try/except, and with strict=False it only warns, so it does not stop the process (afd_plugin/__init__.py:162-170).

  • The plugin entry point: register_afdregister_afd is the one function vLLM calls to install AFD. This page explains why it is safe to call many times, how it stays importable without vLLM, its version gate, and the order and error behaviour of its registration steps.
  • Compat patches: how AFD changes vLLM without editing itAFD edits no vLLM source. It imports patch modules that replace vLLM functions in place, behind a version gate. This page covers the discipline, the gates, and what each GPU patch changes.
  • Worker selection: choosing the role workerAFD chooses a role-specific worker class by rewriting parallel_config.worker_cls, but only when the user left it as "auto". This page shows the rewrite, the platform-to-worker table, the native resolution, the runner swap and the compatibility check.
  • Model registration: the AFD<arch> wrappersAFD registers each supported architecture under an AFD<arch> name, as a lazy string. At build time it swaps the model to its AFD identity with a deepcopy. This page shows the registration, the identity swap, and which architectures map to which wrapper.
  • afd_plugin/config.py: the AFD config schema and parserconfig.py defines AFDConfig and the functions that parse and validate additional_config["afd"]. config_utils.py holds the type coercers; envs.py holds two diagnostic environment flags. This page reads the three files.