# Kimi-K2

A 1T MoE served prefill/decode disaggregated across two H200 nodes.

Source: /recipes/kimi-k2/

<!-- vale write-good.Passive = NO -->
A 1T MoE (1 trillion parameters) served prefill/decode disaggregated across two
H200 nodes, one engine for each phase. The llm-d endpoint picker and sidecar
route between them. This recipe serves an INT4 quantization of the model; the
native FP8 weights need four such nodes.

The engines ran end to end with the `InferenceClass` below. The
`ModelDeployment` now writes its own routing, as
[Configure Routing]({{< ref "/models/routing" >}}) describes, and hasn't run
live in that form yet. Apply the platform side first, then the ML side. Edit the EC2 capacity reservation placeholder in the `InferenceCluster`
before applying it.

## Validated deployments

{{< validated-deployments >}}

## Platform

{{< manifests "recipes/kimi-k2/inference-class.yaml" >}}

{{< manifests path="recipes/kimi-k2/inference-cluster.yaml" apply="false" >}}

{{< editCode >}}
```bash
curl -fsSL {{< manifest-url "recipes/kimi-k2/inference-cluster.yaml" >}} \
  | sed 's/cr-0123456789abcdef0/$@<your-reservation-id>$@/' \
  | kubectl apply -f -
```
{{< /editCode >}}

## Deployment

{{< manifests "recipes/kimi-k2/model-cache.yaml" >}}

{{< manifests "recipes/kimi-k2/model-deployment.yaml" >}}

{{< manifests "recipes/kimi-k2/model-service.yaml" >}}
<!-- vale write-good.Passive = YES -->