Reference: compat patches

AdvancedReferenceabout 5 min read

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

In one sentence

This table lists every patch in afd_plugin/compat/patches/, its vLLM target at the pinned commit, its install mechanism, when it is active, and what it changes.

How to read this table#

Each row is one replaced function. The vLLM target is the function the patch stands in for, at the pinned vLLM commit (ced6857). The install column says where and how AFD installs the patch and what gates it. The active when column says which configs the patch actually changes; the others see native behavior. Every GPU patch is behind the version gate is_target_vllm_compatible (afd_plugin/compat/vllm.py:53-68). The concept page is Compat patches: how AFD changes vLLM without editing it.

GPU patches#

config_validation (config_validation.py)#

vLLM target	Install	Active when	Effect	Page
`EngineArgs.create_engine_config` (`vllm:vllm/engine/arg_utils.py:2040`)	import time, gated; original saved once (`afd_plugin/compat/patches/config_validation.py:235-261`)	an AFD config is present	rewrites `worker_cls` to the AFD role worker when it was `"auto"` (`afd_plugin/compat/patches/config_validation.py:92-93`, `afd_plugin/compat/patches/config_validation.py:179-197`); relaxes one ubatch backend check (`afd_plugin/compat/patches/config_validation.py:200-214`)	Worker selection: choosing the role worker
`VllmConfig.__post_init__` (`vllm:vllm/config/vllm.py:1254`)	import time, gated (`afd_plugin/compat/patches/config_validation.py:243-260`)	AFD config with ubatching, on config revalidation	presents a validation-safe all2all backend during revalidation, then restores it (`afd_plugin/compat/patches/config_validation.py:128-157`)	Worker selection: choosing the role worker

engine_core (engine_core.py)#

vLLM target	Install	Active when	Effect	Page
`EngineCore.__init__` (`vllm:vllm/v1/engine/core.py:108`)	import time, gated (`afd_plugin/compat/patches/engine_core.py:722-727`)	role is `ffn`	returns after the executor is built; no real scheduler (a no-op stand-in), no KV cache (`afd_plugin/compat/patches/engine_core.py:56-68`, `afd_plugin/compat/patches/engine_core.py:599-623`)	Attention and FFN roles: what each server owns
`EngineCore._initialize_kv_caches` (`vllm:vllm/v1/engine/core.py:254`)	import time, gated (`afd_plugin/compat/patches/engine_core.py:724`)	role is `ffn`, late-loaded path	returns a minimal KV cache config and sets zero blocks for FFN daemons (`afd_plugin/compat/patches/engine_core.py:257-264`, `afd_plugin/compat/patches/engine_core.py:635-647`)	Attention and FFN roles: what each server owns
`EngineCore.shutdown` (`vllm:vllm/v1/engine/core.py:756`)	import time, gated (`afd_plugin/compat/patches/engine_core.py:725`)	role is `ffn`	stops the FFN worker loop before tearing down the executor (`afd_plugin/compat/patches/engine_core.py:216-228`)	Attention and FFN roles: what each server owns
`EngineCoreProc.run_busy_loop` (`vllm:vllm/v1/engine/core.py:1403`)	import time, gated (`afd_plugin/compat/patches/engine_core.py:726`)	FFN engine	runs the connector loop: `collective_rpc("start_ffn_server_loop")`, then polls worker errors (`afd_plugin/compat/patches/engine_core.py:411-419`, `afd_plugin/compat/patches/engine_core.py:650-662`)	The FFN runtime: a connector-driven server loop
`DPEngineCoreProc.run_busy_loop` (`vllm:vllm/v1/engine/core.py:2181`)	import time, gated (`afd_plugin/compat/patches/engine_core.py:727`)	FFN engine (DP path)	same connector loop for the DP engine process (`afd_plugin/compat/patches/engine_core.py:416-419`)	The FFN runtime: a connector-driven server loop

gpu_mrv2_ubatching (gpu_mrv2_ubatching.py)#

vLLM target	Install	Active when	Effect	Page
`maybe_build_ubatch_runner` (`vllm:vllm/v1/worker/gpu/ubatch_utils.py:538`)	scoped: `use_gpu_mrv2_ubatch_factory` (`afd_plugin/compat/patches/gpu_mrv2_ubatching.py:263-271`)	inside AFD `initialize_kv_cache`, DP size 1 (`afd_plugin/v1/worker/attention_model_runner_v2.py:479-486`)	builds the ubatch runner at DP1 and binds DP1 child metadata (`afd_plugin/compat/patches/gpu_mrv2_ubatching.py:47-73`)	Dual-batch overlap (DBO): two stages per step
`dispatch_cg_and_sync_dp` (`vllm:vllm/v1/worker/gpu/dp_utils.py:214`)	scoped: `use_gpu_mrv2_dbo_dispatch` (`afd_plugin/compat/patches/gpu_mrv2_ubatching.py:274-282`)	inside one AFD `execute_model` call (`afd_plugin/v1/worker/attention_model_runner_v2.py:340-349`)	applies the DBO threshold and ubatch descriptor at DP1 with no DP handshake (`afd_plugin/compat/patches/gpu_mrv2_ubatching.py:194-226`)	Dual-batch overlap (DBO): two stages per step

NPU-gated patches#

These two modules install at import time but only change behavior for async-DP configs, which are NPU only. On GPU, async_dp must be false, so they run as transparent re-implementations (afd_plugin/config.py:303-308).

async_dp_engine (async_dp_engine.py)#

vLLM target	Install	Active when	Effect	Page
`EngineCoreProc.run_engine_core` (`vllm:vllm/v1/engine/core.py:1283`)	import time, gated (`afd_plugin/compat/patches/async_dp_engine.py:424-432`)	NPU async attention config	selects `EngineCoreProc` instead of `DPEngineCoreProc` (`afd_plugin/compat/patches/async_dp_engine.py:117-120`)	Ascend NPU overview: the same contract, different transports
`launch_core_engines` (`vllm:vllm/v1/engine/utils.py:1104`)	import time, gated (`afd_plugin/compat/patches/async_dp_engine.py:428-429`)	NPU async config	disables DP wave coordination on the coordinator (`afd_plugin/compat/patches/async_dp_engine.py:248-252`)	Ascend NPU overview: the same contract, different transports
`DPAsyncMPClient.add_request_async` (`vllm:vllm/v1/engine/core_client.py:1486`)	import time, gated (`afd_plugin/compat/patches/async_dp_engine.py:430`)	NPU async config	skips the `FIRST_REQ` wakeup for async configs (`afd_plugin/compat/patches/async_dp_engine.py:389-401`)	Ascend NPU overview: the same contract, different transports

async_dp_forward_context (async_dp_forward_context.py)#

vLLM target	Install	Active when	Effect	Page
`set_forward_context` (`vllm:vllm/forward_context.py:269`)	import time, gated; also rebinds the symbol in importing modules (`afd_plugin/compat/patches/async_dp_forward_context.py:207-212`)	NPU async-DP config	skips native DP metadata construction only for async configs (`afd_plugin/compat/patches/async_dp_forward_context.py:88-127`)	Ascend NPU overview: the same contract, different transports

NPU device patches#

These install only when the device is NPU (afd_plugin/__init__.py:204-209). Their upstream targets are in vLLM-Ascend, which is not one of the two pinned repositories, so the row cites the AFD module. They are covered on Ascend NPU overview: the same contract, different transports.

AFD module	Upstream (external)	Effect
npu/ascend_config.py	vLLM-Ascend `ascend_config`	keeps AFD settings outside the strict Ascend config namespace (`afd_plugin/compat/patches/npu/ascend_config.py:3`)
npu/ascend_platform.py	vLLM-Ascend `platform.py` (`afd_plugin/compat/patches/npu/ascend_platform.py:5`)	patches Ascend platform config normalization for AFD DBO (`afd_plugin/compat/patches/npu/ascend_platform.py:3`)
npu/mla_graph.py	vLLM-Ascend `attention/mla_v1.py` (`afd_plugin/compat/patches/npu/mla_graph.py:5`)	routes Ascend MLA ACL graph parameters through AFD forward contexts (`afd_plugin/compat/patches/npu/mla_graph.py:3`)
npu/mrv2_ubatching.py	vLLM native ubatch runner (Ascend baseline) (`afd_plugin/compat/patches/npu/mrv2_ubatching.py:3-4`)	scopes Ascend seams for the native ubatch runner (`afd_plugin/compat/patches/npu/mrv2_ubatching.py:3-8`)

What is not patched#

AFD does not patch the API server, the scheduler, the sampler or the attention and KV kernels. The attention runner wraps native methods instead of replacing them (afd_plugin/v1/worker/attention_model_runner_v2.py:569-595), and sampling stays native (vllm:vllm/v1/worker/gpu/model_runner.py:2048-2050).