Ascend NPU overview: the same contract, different transports

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

The idea#

The NPU path is the one place this site covers Ascend code. The attention/FFN contract is the same as on GPU (see The connector contract: the Attention/FFN transport): the attention side sends hidden states, the FFN side runs the experts and sends the result back. What changes is the transport. The NPU has two connectors, both backed by Ascend CAM operators.

• CAMP2pAFDConnector — synchronous, with a control plane. HCCL carries the hidden states; a smaller Gloo group carries the DP metadata (afd_plugin/connectors/npu/camp2p.py:3-8).
• CAMAsyncAFDConnector — asynchronous, with control_plane = None. The CAM dispatch/combine operators carry the routing metadata, so a separate control plane is not needed (afd_plugin/connectors/npu/async_cam.py:277-283).

The picture#

Read the panels top to bottom. Panel 1 is the shared contract. Panel 2 is the synchronous connector, which keeps a control plane. Panel 3 is the asynchronous connector, where the receive loop itself drives each FFN layer.

In the code#

Step 1: CAMP2p, the synchronous connector with a control plane#

The synchronous connector builds its own HCCL groups for the hidden states (afd_plugin/connectors/npu/camp2p.py:418-431) and a smaller Gloo group for the DP metadata (afd_plugin/connectors/npu/camp2p.py:460-468). It sets a control plane, CAMP2pAFDControlPlane (afd_plugin/connectors/npu/camp2p.py:385), so the DP metadata travels once per step, the same timing as the GPU connector. See The control plane: DP metadata once per step.

Step 2: CAMAsync has no control plane; its receive loop drives the FFN side#

The asynchronous connector sets control_plane = None (afd_plugin/connectors/npu/async_cam.py:283). The CAM dispatch/combine operators own both the data motion and its routing metadata, so there is no separate DP metadata path (afd_plugin/connectors/npu/async_cam.py:277-282).

    second. CAM dispatch/combine operators own both the collective data motion
    and its routing metadata, so this connector has no DP metadata control
    plane (``control_plane`` stays ``None``) and FFN work is triggered directly
    by the connector receive loop.
    """

    control_plane = None

So what drives the FFN side? The NPU async FFN runner loops over the layers and, for each one, calls recv_ffn_work_item (afd_plugin/v1/worker/npu/async_ffn_model_runner.py:442-446). Each work item carries its own layer index, read from the CAM metadata of the dispatch-receive, not from a control-plane payload (afd_plugin/connectors/npu/async_cam.py:497-541).

        for _ in self.layer_ids[:1] if single else self.layer_ids:
            work_item = connector.recv_ffn_work_item(
                stage_idx=stage_idx,
                max_num_tokens=self.max_num_tokens,
            )
            hidden_states = work_item.hidden_states
            metadata = work_item.context.metadata
            legacy_states = work_item.context.states
            if not isinstance(legacy_states, AFDAsyncTransferState):
                raise RuntimeError(
                    "CAM async FFN work item requires AFDAsyncTransferState",
                )
            layer_idx = work_item.layer_idx
            num_tokens = work_item.num_tokens
            afd_metadata = AFDForwardContextMetadata(
                tokens_start_loc=[0],
                requests_start_loc=[0],
                stage_idx=stage_idx,
                connector=self.connector,
                tokens_lens=[num_tokens],
                num_stages=1,
                tokens_unpadded_lens=[num_tokens],
            )

            with ascend_forward_context(
                vllm_config=self.vllm_config,
                afd_metadata=afd_metadata,
                model_instance=self.model,
                num_tokens=num_tokens,
                skip_mc2_mask=self.afd_config.compute_gate_on_attention,
            ) as forward_context:
                forward_context.dp_metadata = None
                forward_context.additional_kwargs["afd_metadata"] = metadata
                _set_moe_layer_index(forward_context, layer_idx)

The forward context sets dp_metadata = None for each work item (afd_plugin/v1/worker/npu/async_ffn_model_runner.py:473). The connector receive is the clock: when a dispatch arrives, the runner computes that layer and sends the result back. This is the mechanism the The control plane: DP metadata once per step page names in one line.

Step 3: The operator loader#

Both connectors need the plugin's Ascend operators. The loader ensure_cam_p2p_ops_available sets the custom op-API environment, imports afd_plugin._C_ascend, and asserts the torch.ops.afd_ascend namespace (afd_plugin/compat/npu/ops.py:73-91). ensure_afd_ascend_ops_loaded is a backward-compatible alias for it (afd_plugin/compat/npu/ops.py:94-97).

@lru_cache(maxsize=1)
def ensure_cam_p2p_ops_available() -> None:
    """Import the custom operators used by ``CAMP2pAFDConnector``.

    The extension is optional at package import time.  It is built by default
    in an Ascend environment unless ``AFD_BUILD_ASCEND_OPS=0`` is set.
    """

    _ensure_afd_custom_opp_env()
    try:
        import torch

        import afd_plugin._C_ascend  # noqa: F401
    except ImportError as exc:
        raise RuntimeError(
            "CAMP2P Ascend custom ops are not available. Build the package with "
            "Ascend ops enabled in a torch-npu/CANN environment.",
        ) from exc
    _assert_afd_namespace_registered(torch)

The async connector needs four more operators, so it calls ensure_cam_async_ops_available, which checks the four afd_async_* ops (afd_plugin/compat/npu/ops.py:107-119). The extension afd_plugin._C_ascend is built from csrc/npu/pybind (setup.py:116-120), gated on AFD_BUILD_ASCEND_OPS or an Ascend platform (setup.py:43-54) and the SOC_VERSION (setup.py:67, setup.py:88). See Environment variables reference.

Step 4: Async data-parallel patches, gated to NPU#

The async path needs two extra patches. They install only when the role is attention and async_dp is on (afd_plugin/compat/npu/runtime.py:44-53), and the whole NPU patch set installs only when the device is npu (afd_plugin/__init__.py:199-209). On GPU, async_dp must be false. See Compat patches: how AFD changes vLLM without editing it.

• async_dp_engine — swaps the attention engine entry point to a plain EngineCoreProc for async DP (afd_plugin/compat/patches/async_dp_engine.py:117-120; installed and gated at afd_plugin/compat/patches/async_dp_engine.py:414-432).
• async_dp_forward_context — skips the DP metadata only for the async path, and rebinds the symbol in the importing modules (afd_plugin/compat/patches/async_dp_forward_context.py:88-127, afd_plugin/compat/patches/async_dp_forward_context.py:207-212).

Deep dive#

What can go wrong#

• Operators not built. The loader raises when afd_plugin._C_ascend is missing; build with Ascend ops enabled (afd_plugin/compat/npu/ops.py:86-90).
• Async ops missing. The async connector raises when any of the four afd_async_* operators is absent (afd_plugin/compat/npu/ops.py:115-119).
• async_dp on GPU. The async-DP patches are NPU-only and gated at run time; on GPU async_dp must be false (afd_plugin/compat/npu/runtime.py:44-53).
• Expecting a control plane with the async connector. The async connector has none; the receive loop drives the FFN side (afd_plugin/connectors/npu/async_cam.py:283).

Why it matters. The NPU keeps the AFD contract but changes the transport and the driving mechanism. The async connector folds the control plane into the data operators, so the FFN side follows the receive loop, not a metadata step.