AFD Plugin Explained

model_executor/models/deepseek_v2.py: the DeepSeek wrapper

AdvancedCode mapabout 5 min read + 1 min deep dives

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

In one sentence

This file is the DeepSeek model wrapper. It builds a role-aware decoder layer and routes the FFN stage through the connector.

Purpose#

This file implements the model split for DeepSeek V2/V3/V3.2 and GLM-MoE-DSA. The concept is on Model wrappers: the role-aware decoder layer and the gate handoff on The MoE handoff: where the gate runs; this page reads the file class by class.

ItemValue
File (lines, origin)model_executor/models/deepseek_v2.py (994); AFD-owned
Main classesAFDDeepseekV2DecoderLayer (afd_plugin/model_executor/models/deepseek_v2.py:375), AFDDeepseekV2Model (afd_plugin/model_executor/models/deepseek_v2.py:725), AFDDeepseekV2ForCausalLM (afd_plugin/model_executor/models/deepseek_v2.py:925)
Runs inboth roles; the class branches on afd_config.role and the device
Called byvLLM model loading, through the registered AFDDeepseekV2ForCausalLM (afd_plugin/__init__.py:100-102)
Callsnative vLLM DeepSeek modules (vllm:vllm/model_executor/models/deepseek_v2.py:1), the connector, the DBO yield op
Not in this filethe runners (v1/worker: the attention worker and runner, v1/worker: the FFN worker and runner), the NPU DeepSeek files

Where it sits#

Who calls it#

The plugin registers AFDDeepseekV2ForCausalLM under the model name AFDDeepseekV2ForCausalLM (afd_plugin/__init__.py:100-102, afd_plugin/__init__.py:213-214). The worker selects the AFD model identity before loading (afd_plugin/v1/worker/attention_worker.py:75-80), so vLLM loads this class.

What it calls#

The classes subclass native vLLM DeepSeek modules imported as native (afd_plugin/model_executor/models/deepseek_v2.py:22). The proxy calls the connector through the AFD forward metadata, and the DBO yield op between send and receive (afd_plugin/model_executor/models/deepseek_v2.py:146-158).

Key classes and functions#

NameLineRole
_checkpoint_weight_rolesafd_plugin/model_executor/models/deepseek_v2.py:69-95classify a weight path by its owning role
RemoteFFNProxyafd_plugin/model_executor/models/deepseek_v2.py:117parameter-free FFN stage over the connector
AFDAttentionFusedMoEafd_plugin/model_executor/models/deepseek_v2.py:161experts proxy; sends router logits on external routing
GateOnlyRemoteMoEafd_plugin/model_executor/models/deepseek_v2.py:197NPU attention-side gate, remote experts
AFDDeepseekV2RemoteExpertsMoEafd_plugin/model_executor/models/deepseek_v2.py:277native MoE shell with a remote-experts proxy
AFDDeepseekV2DecoderLayerafd_plugin/model_executor/models/deepseek_v2.py:375role-aware decoder layer
AFDDeepseekV2Modelafd_plugin/model_executor/models/deepseek_v2.py:725role-aware model; builds the layers
AFDDeepseekV2ForCausalLMafd_plugin/model_executor/models/deepseek_v2.py:925causal-LM wrapper; filters weights by role
family subclassesafd_plugin/model_executor/models/deepseek_v2.py:977-986Deepseek, V3, GLM-MoE-DSA pass-throughs

Walk through the main methods#

Construction branches: what each role builds#

AFDDeepseekV2DecoderLayer.__init__ builds only the modules its role owns (afd_plugin/model_executor/models/deepseek_v2.py:383-561). The table names the branch and the lines.

RoleLayer kindModule builtLines
attentionanyself_attn: native attentionafd_plugin/model_executor/models/deepseek_v2.py:474-491
attentionCUDA MoEAFDDeepseekV2RemoteExpertsMoEafd_plugin/model_executor/models/deepseek_v2.py:493-500
attentionNPU MoE + gateGateOnlyRemoteMoEafd_plugin/model_executor/models/deepseek_v2.py:501-507
attentiondense + gatenative DeepseekV2MLPafd_plugin/model_executor/models/deepseek_v2.py:508-515
attentionotherwiseRemoteFFNProxyafd_plugin/model_executor/models/deepseek_v2.py:516-517
FFNanyself_attn: PPMissingLayerafd_plugin/model_executor/models/deepseek_v2.py:520
FFNMoEnative DeepseekV2MoE (gate cleared on CUDA + gate)afd_plugin/model_executor/models/deepseek_v2.py:523-553
FFNdensenative DeepseekV2MLPafd_plugin/model_executor/models/deepseek_v2.py:554-561

The layer's compute methods, by role#

The layer has no single forward: the runner calls role-specific compute methods. On the attention role, compute_attn_output runs the input norm, attention and post-attention norm, and (on NPU) the gate (afd_plugin/model_executor/models/deepseek_v2.py:577-647). The mlp module then sends the hidden states through the connector.

    def _send_and_receive(
        self,
        hidden_states: torch.Tensor,
        **send_kwargs: torch.Tensor,
    ) -> torch.Tensor:
        afd_metadata = get_afd_metadata_from_forward_context()
        if afd_metadata is None:
            raise RuntimeError("RemoteFFNProxy requires AFD forward metadata")
        forward_context = get_forward_context()
        stage_idx = int(
            getattr(forward_context, "ubatch_idx", afd_metadata.stage_idx),
        )
        afd_metadata.stage_idx = stage_idx
        metadata = AFDTransferMetadata.create_attention_metadata(
            layer_idx=self.layer_idx,
            stage_idx=stage_idx,
            seq_len=int(hidden_states.shape[0]),
        )
        context = AFDTransferContext(metadata=metadata)
        afd_metadata.connector.send_attn_output(
            hidden_states,
            context,
            **send_kwargs,
        )
        hidden_states = maybe_apply_dbo_yield(
            hidden_states,
            role="attention",
        )
        return afd_metadata.connector.recv_ffn_output(
            ref_tensor=hidden_states,
            ubatch_idx=stage_idx,
        )
RemoteFFNProxy._send_and_receive: send, DBO yield, receive.

On the FFN role, the runner calls compute_ffn_output for a dense or gate-on-FFN layer, or compute_experts_output for an attention-side-gate MoE layer (afd_plugin/model_executor/models/deepseek_v2.py:649-721).

    def compute_experts_output(
        self,
        hidden_states: torch.Tensor,
        router_logits: torch.Tensor,
    ) -> torch.Tensor:
        """Execute the native external-router runner on the FFN role."""
        if not self.compute_gate_on_attention or not self.is_moe_layer:
            raise RuntimeError(
                "compute_experts_output requires an Attention-side-gate MoE layer",
            )
        if not isinstance(self.mlp, native.DeepseekV2MoE):
            raise RuntimeError("FFN role does not own a native DeepSeek MoE")
        # v0.30 runner contract: gate is None means the runner consumes the
        # router logits transported from Attention (external routing).
        if self.mlp.experts.gate is not None:
            raise RuntimeError("FFN native runner must use external routing")
        return self.mlp.experts(
            hidden_states=hidden_states,
            router_logits=router_logits,
        )
compute_experts_output: run the native experts with the transported router logits.

The model and causal-LM wrappers#

AFDDeepseekV2Model is decorated with support_torch_compile and builds the layers with the role-aware constructor (afd_plugin/model_executor/models/deepseek_v2.py:724-725, afd_plugin/model_executor/models/deepseek_v2.py:791-802). It forwards to the native model, except for the async NPU connector, which uses its own forward (afd_plugin/model_executor/models/deepseek_v2.py:839-863).

AFDDeepseekV2ForCausalLM filters the weight stream by role before the native loader (afd_plugin/model_executor/models/deepseek_v2.py:964-974). The DeepSeek, V3 and GLM-MoE-DSA classes are thin subclasses (afd_plugin/model_executor/models/deepseek_v2.py:977-986).

get_experts_layer_indices and get_experts_routing_spec#

The FFN runner needs to know which layers hold experts, and the routing spec for graph capture. get_experts_layer_indices returns the MoE-layer indices (afd_plugin/model_executor/models/deepseek_v2.py:887-895).

    def get_experts_layer_indices(self) -> tuple[int, ...]:
        # Async NPU FFN and CUDA remote-expert execution share this inventory.
        # uses_remote_experts selects a CUDA-specific construction path; it
        # does not describe whether an NPU decoder contains routed experts.
        return tuple(
            layer_idx
            for layer_idx, layer in enumerate(self.layers)
            if isinstance(layer, AFDDeepseekV2DecoderLayer) and layer.is_moe_layer
        )

    def get_experts_routing_spec(
        self,
        layer_idx: int,
    ) -> AFDExpertRoutingSpec:
        """Return the static native-router contract for graph capture."""
        layer = self.layers[layer_idx]
        gate = layer.mlp.gate
        return AFDExpertRoutingSpec(
            router_logits_width=int(layer.mlp.n_routed_experts),
            router_logits_dtype=gate.out_dtype or gate.weight.dtype,
        )
The expert-layer inventory and the static routing spec.
  1. Lines 887-895. Collect every AFDDeepseekV2DecoderLayer whose is_moe_layer is true.
  2. Lines 897-907. Return the router-logits width and dtype, so graph capture can pre-allocate the wire buffers.

The causal-LM class forwards both to the model (afd_plugin/model_executor/models/deepseek_v2.py:955-962).

Deep dive#

Deep dive

The remote proxies, compared

ClassBaseExtra payloadLine
RemoteFFNProxynn.Modulenone (hidden states only)afd_plugin/model_executor/models/deepseek_v2.py:117-158
AFDAttentionFusedMoERemoteFFNProxyrouter_logits on external routingafd_plugin/model_executor/models/deepseek_v2.py:161-191
GateOnlyRemoteMoERemoteFFNProxygate outputs (NPU)afd_plugin/model_executor/models/deepseek_v2.py:197-274
AFDDeepseekV2RemoteExpertsMoEnative DeepseekV2MoEwraps AFDAttentionFusedMoE as its expertsafd_plugin/model_executor/models/deepseek_v2.py:277-335

The shell computes the gate before transport only when it owns one; a gate-less shell keeps the native delegation (afd_plugin/model_executor/models/deepseek_v2.py:347-371).

Deep dive

The weight filter in detail

_checkpoint_weight_roles decides each weight's owner (afd_plugin/model_executor/models/deepseek_v2.py:69-95). A non-layer path (embeddings, final norm) returns both roles; a self_attn path returns attention; a dense mlp path follows the gate flag; a MoE gate path returns both roles when the gate runs on the attention side, and the other expert weights return the FFN role. The causal-LM class passes the connector choice so the async NPU connector can also keep the shared-expert weights on the attention role (afd_plugin/model_executor/models/deepseek_v2.py:969-973).

Check yourself#

  1. 1
    What base class does AFDDeepseekV2RemoteExpertsMoE extend?
    easy
    Show answer

    The native DeepseekV2MoE; it keeps the native MoE forward but replaces the experts with a proxy (afd_plugin/model_executor/models/deepseek_v2.py:277).

  2. 2
    The decoder layer has no single forward. Which methods run its two halves, and on which role?
    medium
    Show answer

    compute_attn_output on the attention role (afd_plugin/model_executor/models/deepseek_v2.py:577-647); compute_ffn_output or compute_experts_output on the FFN role (afd_plugin/model_executor/models/deepseek_v2.py:649-721).

  3. 3
    Why does get_experts_routing_spec exist, and what does it return?
    hard
    Show answer

    Graph capture must pre-allocate the router-logits buffer, so it needs a static shape. It returns the router-logits width (the routed-expert count) and dtype (afd_plugin/model_executor/models/deepseek_v2.py:897-907).