connectors/base.py, factory.py, metadata.py: the connector core

Code version: afd-plugin @ 7452d52 · 2026-10-27 · vLLM @ ced6857

Purpose#

These files hold the parts every connector shares. The contract and the factory are explained on the The connector contract: the Attention/FFN transport page; this page maps the three files and reads the metadata objects and the factory loader.

Where it sits#

Who calls it#

The factory builds a connector for a runner. The FFN runner calls create_connector at construction (afd_plugin/v1/worker/ffn_model_runner.py:80-85); the attention runner does the same. The model wrappers build the metadata objects for each transfer (afd_plugin/model_executor/models/deepseek_v2.py:140-145).

What it calls#

base.py reads the AFD config helper connector_extra_config_from_source (afd_plugin/connectors/base.py:105-107). factory.py imports the connector module lazily with importlib (afd_plugin/connectors/factory.py:38-45). metadata.py encodes the control payload with json (afd_plugin/connectors/metadata.py:344-346).

Key classes and functions#

Walk through the main methods#

AFDTransferMetadata: the per-transfer record#

This dataclass describes one layer-and-stage transfer. It validates its lengths and derives the total token count (afd_plugin/connectors/metadata.py:171-200).

    layer_idx: int
    stage_idx: int
    seq_lens: list[int]

    def __post_init__(self) -> None:
        if not self.seq_lens:
            raise ValueError("seq_lens cannot be empty")
        if any(length <= 0 for length in self.seq_lens):
            raise ValueError("all sequence lengths must be positive")

    @property
    def total_tokens(self) -> int:
        return sum(self.seq_lens)

1. Line 188-190. The fields: layer_idx, stage_idx, and the per-peer seq_lens list.
2. Line 192-196. __post_init__ rejects an empty list and any non-positive length.
3. Line 198-200. total_tokens is the sum of the lengths; it is the expected leading dimension.

Two class methods build the attention form (one length) and the FFN form (a list of lengths) (afd_plugin/connectors/metadata.py:202-228). The shape check is validate_tensor_shape (afd_plugin/connectors/metadata.py:230-231).

encode_control_payload: the wire format#

The control payload is serialized to compact JSON, with only the fields the FFN side reads back (afd_plugin/connectors/metadata.py:320-346).

    metadata_payload: dict[str, dict[str, int | list[int]]] = {}
    for stage_idx, dp_metadata in payload.dp_metadata_list.items():
        stage_payload: dict[str, int | list[int]] = {
            "num_tokens_across_dp_cpu": dp_metadata.num_tokens_across_dp_cpu.tolist(),
            "max_tokens_across_dp_cpu": _to_int(dp_metadata.max_tokens_across_dp_cpu),
        }
        metadata_payload[str(int(stage_idx))] = stage_payload

    wire_payload = {
        "dp_metadata_list": metadata_payload,
        "is_graph_capturing": bool(payload.is_graph_capturing),
        "is_warmup": bool(payload.is_warmup),
        "is_graph_replaying": bool(payload.is_graph_replaying),
        "is_profile": bool(payload.is_profile),
    }
    return json.dumps(wire_payload, separators=(",", ":"), sort_keys=True).encode(
        "utf-8",
    )

1. Line 329-335. One entry per stage: the per-DP-rank token list and its max.
2. Line 337-343. The graph-capture, warm-up, replay and profile flags.
3. Line 344-346. json.dumps with sorted keys and no spaces. The The control plane: DP metadata once per step page reads the send and receive.

AFDConnectorFactory: name to class#

The factory keeps a registry of loaders. register_connector stores a loader that imports the module lazily (afd_plugin/connectors/factory.py:26-47); create_connector looks the name up, resolves the role rank, and builds the instance (afd_plugin/connectors/factory.py:49-68).

        def loader() -> type[AFDConnectorBase]:
            module = importlib.import_module(module_path)
            connector_cls = vars(module)[class_name]
            if not issubclass(connector_cls, AFDConnectorBase):
                raise TypeError(
                    f"{module_path}.{class_name} is not an AFDConnectorBase",
                )
            return connector_cls

        cls._registry[name] = loader

Three connectors are registered at import time; an unknown name raises (afd_plugin/connectors/factory.py:58-59, afd_plugin/connectors/factory.py:88-102).

Deep dive#

Related concepts#

• The connector contract: the Attention/FFN transport — the ABC, the lifecycle and the factory in full.
• The control plane: DP metadata once per step — how the control payload is published and received.
• Data on the wire: tensors per layer and stage — how the transfer metadata drives the fan-in and split.
• connectors/gpu/p2p.py: the GPU P2P connector — the GPU connector that implements the contract.