Attention and FFN roles: what each server owns

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

The idea#

A normal vLLM server owns the whole request path: the API server, the scheduler, the KV cache, the model and the sampler. AFD keeps all of that on the attention server. The FFN server is different. It has no API server, no KV cache and no real scheduler (only a no-op stand-in). A connector loop drives it: it waits for work, runs the feed-forward and MoE experts, and sends the result back.

Each side also owns a different part of every decoder layer. The attention server builds the real attention module and sends its output to the FFN side. The FFN server builds the real experts and drops the attention module.

An everyday comparison. Think of a kitchen with a front counter. The counter takes the order, tracks the bill and hands finished plates to the customer. The cooks only cook what the counter passes through. The comparison stops here: the FFN server is not a separate service. It runs part of the same model's forward pass, once per layer, for every step.

The picture#

Read panel 1 as a comparison of what each server owns. Read panel 2 as the split of one decoder layer: the same layer is built with different modules on each side.

In the code#

Step 1: Two workers, two runners#

AFD does not register its role workers through a vLLM API. A patch rewrites parallel_config.worker_cls when it is "auto"; see Worker selection: choosing the role worker. Each role worker then swaps in its own model runner inside init_device (afd_plugin/v1/worker/attention_worker.py:82-87, afd_plugin/v1/worker/ffn_worker.py:93-98).

Step 2: The FFN EngineCore returns early#

A patch on EngineCore.__init__ checks the role. For an FFN config, it builds the model executor and then returns, before the KV cache and the scheduler are set up (afd_plugin/compat/patches/engine_core.py:59-68).

    # ### PATCH START: AFD FFN EngineCore daemon initialization
    # FFN ranks are connector daemons, so stop EngineCore initialization after
    # executor construction instead of setting up KV cache and scheduler state.
    if _is_afd_ffn_config(vllm_config):
        _initialize_ffn_engine_core(
            self,
            core_module,
            vllm_config,
            executor_class,
            log_stats,
            executor_fail_callback,
        )
        return
    # ### PATCH END: AFD FFN EngineCore daemon initialization

The FFN EngineCore sets the KV block counts to zero and installs a no-op scheduler. The no-op scheduler is not None, because a native readiness handshake calls a method on it (afd_plugin/compat/patches/engine_core.py:605-606, afd_plugin/compat/patches/engine_core.py:608-614).

Step 3: The FFN worker has no KV cache and does not sample#

The FFN worker reports an empty KV cache spec, so vLLM allocates no KV blocks for it (afd_plugin/v1/worker/ffn_worker.py:102-105). The FFN runner repeats this (afd_plugin/v1/worker/ffn_model_runner.py:147-148).

    def get_kv_cache_spec(self) -> dict[str, KVCacheSpec]:
        """FFN workers do not allocate KV cache."""

        return {}

Two methods guard against the native scheduler path. The scheduler-driven execute_model raises, and the FFN runner's sample_tokens raises (afd_plugin/v1/worker/ffn_worker.py:122-131, afd_plugin/v1/worker/ffn_model_runner.py:405-406).

    def execute_model(
        self,
        scheduler_output: SchedulerOutput,
    ) -> ModelRunnerOutput | AsyncModelRunnerOutput | None:
        """Fail fast if the default scheduler tries to execute FFN work."""

        raise RuntimeError(
            "AFD FFN workers are connector-driven; scheduler-driven "
            "execute_model() is not supported.",
        )

Step 4: Each role builds half of a decoder layer#

The model wrapper is role-aware. AFDDeepseekV2DecoderLayer reads the role, then builds only the modules that role owns (afd_plugin/model_executor/models/deepseek_v2.py:383-396). On the attention role it builds the native attention module (afd_plugin/model_executor/models/deepseek_v2.py:474-491) and replaces the feed-forward:

• a dense layer becomes a RemoteFFNProxy (afd_plugin/model_executor/models/deepseek_v2.py:516-517);
• an MoE layer becomes an AFDDeepseekV2RemoteExpertsMoE shell (afd_plugin/model_executor/models/deepseek_v2.py:493-500).

On the FFN role it drops attention (a PPMissingLayer) and keeps the native experts (afd_plugin/model_executor/models/deepseek_v2.py:520, afd_plugin/model_executor/models/deepseek_v2.py:538-559). The Model wrappers: the role-aware decoder layer page reads this split in full.

Step 5: Where requests go, and the start order#

Requests go only to the attention API server. The two servers may start in either order (README.md:258-260).

Attention and FFN may be started in either order. Send requests only to the
Attention API server. FFN workers are connector-driven; scheduler-driven FFN
`execute_model()` calls fail fast.

Deep dive#

What can go wrong#

• A request sent to the FFN server. The FFN server has no API server, so there is nothing to answer it. Send requests only to the attention API server (README.md:258-260).
• The scheduler reaches the FFN worker. A scheduler-driven execute_model on the FFN worker raises at once (afd_plugin/v1/worker/ffn_worker.py:122-131).
• A connector with no control plane on GPU. The GPU FFN loop needs a control-plane-driven connector and raises for one that sets control_plane = None (afd_plugin/v1/worker/ffn_worker.py:169-174).

Why it matters. The two servers do very different work. If you treat the FFN server like a normal vLLM server, nothing schedules its work and no token comes back.