model_executor/models/deepseek_v2.py: the DeepSeek wrapper
Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857
In one sentence
This file is the DeepSeek model wrapper. It builds a role-aware decoder layer and routes the FFN stage through the connector.
Purpose#
This file implements the model split for DeepSeek V2/V3/V3.2 and GLM-MoE-DSA. The concept is on Model wrappers: the role-aware decoder layer and the gate handoff on The MoE handoff: where the gate runs; this page reads the file class by class.
| Item | Value |
|---|---|
| File (lines, origin) | model_executor/models/deepseek_v2.py (994); AFD-owned |
| Main classes | AFDDeepseekV2DecoderLayer (afd_plugin/model_executor/models/deepseek_v2.py:375), AFDDeepseekV2Model (afd_plugin/model_executor/models/deepseek_v2.py:725), AFDDeepseekV2ForCausalLM (afd_plugin/model_executor/models/deepseek_v2.py:925) |
| Runs in | both roles; the class branches on afd_config.role and the device |
| Called by | vLLM model loading, through the registered AFDDeepseekV2ForCausalLM (afd_plugin/__init__.py:100-102) |
| Calls | native vLLM DeepSeek modules (vllm:vllm/model_executor/models/deepseek_v2.py:1), the connector, the DBO yield op |
| Not in this file | the runners (v1/worker: the attention worker and runner, v1/worker: the FFN worker and runner), the NPU DeepSeek files |
Where it sits#
Who calls it#
The plugin registers AFDDeepseekV2ForCausalLM under the model name
AFDDeepseekV2ForCausalLM (afd_plugin/__init__.py:100-102,
afd_plugin/__init__.py:213-214). The worker selects the AFD model identity before loading
(afd_plugin/v1/worker/attention_worker.py:75-80), so vLLM loads this class.
What it calls#
The classes subclass native vLLM DeepSeek modules imported as native
(afd_plugin/model_executor/models/deepseek_v2.py:22). The proxy calls the connector through the AFD
forward metadata, and the DBO yield op between send and receive
(afd_plugin/model_executor/models/deepseek_v2.py:146-158).
Key classes and functions#
| Name | Line | Role |
|---|---|---|
_checkpoint_weight_roles | afd_plugin/model_executor/models/deepseek_v2.py:69-95 | classify a weight path by its owning role |
RemoteFFNProxy | afd_plugin/model_executor/models/deepseek_v2.py:117 | parameter-free FFN stage over the connector |
AFDAttentionFusedMoE | afd_plugin/model_executor/models/deepseek_v2.py:161 | experts proxy; sends router logits on external routing |
GateOnlyRemoteMoE | afd_plugin/model_executor/models/deepseek_v2.py:197 | NPU attention-side gate, remote experts |
AFDDeepseekV2RemoteExpertsMoE | afd_plugin/model_executor/models/deepseek_v2.py:277 | native MoE shell with a remote-experts proxy |
AFDDeepseekV2DecoderLayer | afd_plugin/model_executor/models/deepseek_v2.py:375 | role-aware decoder layer |
AFDDeepseekV2Model | afd_plugin/model_executor/models/deepseek_v2.py:725 | role-aware model; builds the layers |
AFDDeepseekV2ForCausalLM | afd_plugin/model_executor/models/deepseek_v2.py:925 | causal-LM wrapper; filters weights by role |
| family subclasses | afd_plugin/model_executor/models/deepseek_v2.py:977-986 | Deepseek, V3, GLM-MoE-DSA pass-throughs |
Walk through the main methods#
Construction branches: what each role builds#
AFDDeepseekV2DecoderLayer.__init__ builds only the modules its role owns
(afd_plugin/model_executor/models/deepseek_v2.py:383-561). The table names the branch and the lines.
| Role | Layer kind | Module built | Lines |
|---|---|---|---|
| attention | any | self_attn: native attention | afd_plugin/model_executor/models/deepseek_v2.py:474-491 |
| attention | CUDA MoE | AFDDeepseekV2RemoteExpertsMoE | afd_plugin/model_executor/models/deepseek_v2.py:493-500 |
| attention | NPU MoE + gate | GateOnlyRemoteMoE | afd_plugin/model_executor/models/deepseek_v2.py:501-507 |
| attention | dense + gate | native DeepseekV2MLP | afd_plugin/model_executor/models/deepseek_v2.py:508-515 |
| attention | otherwise | RemoteFFNProxy | afd_plugin/model_executor/models/deepseek_v2.py:516-517 |
| FFN | any | self_attn: PPMissingLayer | afd_plugin/model_executor/models/deepseek_v2.py:520 |
| FFN | MoE | native DeepseekV2MoE (gate cleared on CUDA + gate) | afd_plugin/model_executor/models/deepseek_v2.py:523-553 |
| FFN | dense | native DeepseekV2MLP | afd_plugin/model_executor/models/deepseek_v2.py:554-561 |
The layer's compute methods, by role#
The layer has no single forward: the runner calls role-specific compute methods. On the attention role,
compute_attn_output runs the input norm, attention and post-attention norm, and (on NPU) the gate
(afd_plugin/model_executor/models/deepseek_v2.py:577-647). The mlp module then sends the
hidden states through the connector.
def _send_and_receive(
self,
hidden_states: torch.Tensor,
**send_kwargs: torch.Tensor,
) -> torch.Tensor:
afd_metadata = get_afd_metadata_from_forward_context()
if afd_metadata is None:
raise RuntimeError("RemoteFFNProxy requires AFD forward metadata")
forward_context = get_forward_context()
stage_idx = int(
getattr(forward_context, "ubatch_idx", afd_metadata.stage_idx),
)
afd_metadata.stage_idx = stage_idx
metadata = AFDTransferMetadata.create_attention_metadata(
layer_idx=self.layer_idx,
stage_idx=stage_idx,
seq_len=int(hidden_states.shape[0]),
)
context = AFDTransferContext(metadata=metadata)
afd_metadata.connector.send_attn_output(
hidden_states,
context,
**send_kwargs,
)
hidden_states = maybe_apply_dbo_yield(
hidden_states,
role="attention",
)
return afd_metadata.connector.recv_ffn_output(
ref_tensor=hidden_states,
ubatch_idx=stage_idx,
)On the FFN role, the runner calls compute_ffn_output for a dense or gate-on-FFN layer, or
compute_experts_output for an attention-side-gate MoE layer
(afd_plugin/model_executor/models/deepseek_v2.py:649-721).
def compute_experts_output(
self,
hidden_states: torch.Tensor,
router_logits: torch.Tensor,
) -> torch.Tensor:
"""Execute the native external-router runner on the FFN role."""
if not self.compute_gate_on_attention or not self.is_moe_layer:
raise RuntimeError(
"compute_experts_output requires an Attention-side-gate MoE layer",
)
if not isinstance(self.mlp, native.DeepseekV2MoE):
raise RuntimeError("FFN role does not own a native DeepSeek MoE")
# v0.30 runner contract: gate is None means the runner consumes the
# router logits transported from Attention (external routing).
if self.mlp.experts.gate is not None:
raise RuntimeError("FFN native runner must use external routing")
return self.mlp.experts(
hidden_states=hidden_states,
router_logits=router_logits,
)The model and causal-LM wrappers#
AFDDeepseekV2Model is decorated with support_torch_compile and builds the layers with
the role-aware constructor (afd_plugin/model_executor/models/deepseek_v2.py:724-725,
afd_plugin/model_executor/models/deepseek_v2.py:791-802). It forwards to the native model, except for
the async NPU connector, which uses its own forward
(afd_plugin/model_executor/models/deepseek_v2.py:839-863).
AFDDeepseekV2ForCausalLM filters the weight stream by role before the native loader
(afd_plugin/model_executor/models/deepseek_v2.py:964-974). The DeepSeek, V3 and GLM-MoE-DSA classes are
thin subclasses (afd_plugin/model_executor/models/deepseek_v2.py:977-986).
get_experts_layer_indices and get_experts_routing_spec#
The FFN runner needs to know which layers hold experts, and the routing spec for graph capture.
get_experts_layer_indices returns the MoE-layer indices
(afd_plugin/model_executor/models/deepseek_v2.py:887-895).
def get_experts_layer_indices(self) -> tuple[int, ...]:
# Async NPU FFN and CUDA remote-expert execution share this inventory.
# uses_remote_experts selects a CUDA-specific construction path; it
# does not describe whether an NPU decoder contains routed experts.
return tuple(
layer_idx
for layer_idx, layer in enumerate(self.layers)
if isinstance(layer, AFDDeepseekV2DecoderLayer) and layer.is_moe_layer
)
def get_experts_routing_spec(
self,
layer_idx: int,
) -> AFDExpertRoutingSpec:
"""Return the static native-router contract for graph capture."""
layer = self.layers[layer_idx]
gate = layer.mlp.gate
return AFDExpertRoutingSpec(
router_logits_width=int(layer.mlp.n_routed_experts),
router_logits_dtype=gate.out_dtype or gate.weight.dtype,
)- Lines 887-895. Collect every
AFDDeepseekV2DecoderLayerwhoseis_moe_layeris true. - Lines 897-907. Return the router-logits width and dtype, so graph capture can pre-allocate the wire buffers.
The causal-LM class forwards both to the model
(afd_plugin/model_executor/models/deepseek_v2.py:955-962).
Deep dive#
Deep diveThe remote proxies, compared
| Class | Base | Extra payload | Line |
|---|---|---|---|
RemoteFFNProxy | nn.Module | none (hidden states only) | afd_plugin/model_executor/models/deepseek_v2.py:117-158 |
AFDAttentionFusedMoE | RemoteFFNProxy | router_logits on external routing | afd_plugin/model_executor/models/deepseek_v2.py:161-191 |
GateOnlyRemoteMoE | RemoteFFNProxy | gate outputs (NPU) | afd_plugin/model_executor/models/deepseek_v2.py:197-274 |
AFDDeepseekV2RemoteExpertsMoE | native DeepseekV2MoE | wraps AFDAttentionFusedMoE as its experts | afd_plugin/model_executor/models/deepseek_v2.py:277-335 |
The shell computes the gate before transport only when it owns one; a gate-less shell keeps the native
delegation (afd_plugin/model_executor/models/deepseek_v2.py:347-371).
Deep diveThe weight filter in detail
_checkpoint_weight_roles decides each weight's owner
(afd_plugin/model_executor/models/deepseek_v2.py:69-95). A non-layer path (embeddings, final norm)
returns both roles; a self_attn path returns attention; a dense mlp path follows the
gate flag; a MoE gate path returns both roles when the gate runs on the attention side, and the
other expert weights return the FFN role. The causal-LM class passes the connector choice so the async NPU
connector can also keep the shared-expert weights on the attention role
(afd_plugin/model_executor/models/deepseek_v2.py:969-973).
Related concepts#
- Model wrappers: the role-aware decoder layer — the construction branches in prose.
- The MoE handoff: where the gate runs — the gate placement and the wire payloads.
- v1/worker: the attention worker and runner — the runner that drives the attention side.
- v1/worker: the FFN worker and runner — the runner that calls the FFN compute methods.
Check yourself#
- 1What base class doeseasy
AFDDeepseekV2RemoteExpertsMoEextend?Show answer
The native
DeepseekV2MoE; it keeps the native MoE forward but replaces the experts with a proxy (afd_plugin/model_executor/models/deepseek_v2.py:277). - 2The decoder layer has no single forward. Which methods run its two halves, and on which role?medium
Show answer
compute_attn_outputon the attention role (afd_plugin/model_executor/models/deepseek_v2.py:577-647);compute_ffn_outputorcompute_experts_outputon the FFN role (afd_plugin/model_executor/models/deepseek_v2.py:649-721). - 3Why doeshard
get_experts_routing_specexist, and what does it return?Show answer
Graph capture must pre-allocate the router-logits buffer, so it needs a static shape. It returns the router-logits width (the routed-expert count) and dtype (
afd_plugin/model_executor/models/deepseek_v2.py:897-907).