> ## Documentation Index
> Fetch the complete documentation index at: https://docs.metaphi.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Batches

> Create up to 10,000 twins with one request.

A `batch` creates one [twin](https://docs.metaphi.ai/kennet/api/twins.md) per uploaded [file](https://docs.metaphi.ai/kennet/api/files.md), up to 10,000 per batch. Requests in a batch run in parallel. Use a batch for more than a handful of traces.

## The batch object

```json theme={"dark"}
{
  "id": "batch_5e0b...",
  "object": "kennet.batch",
  "status": "in_progress",
  "created": 1760054400,
  "ended": null,
  "cancelled": null,
  "paused": null,
  "expires": 1760659200,
  "expired": null,
  "error": null,
  "estimated_cost_usd": 412.0,
  "balance_needed_usd": null,
  "livemode": true,
  "request_counts": {
    "total": 2000,
    "queued": 1460,
    "running": 120,
    "paused": 0,
    "completed": 412,
    "failed": 8,
    "cancelled": 0,
    "expired": 0
  },
  "results_url": null
}
```

| Field | Type | Description |
| - | - | - |
| `id` | string | Unique identifier, prefixed `batch_` |
| `object` | string | `kennet.batch` |
| `status` | enum | `in_progress`, `paused`, `cancelling`, `completed`, `cancelled` or `expired`. See [statuses](#statuses) |
| `created` | integer | Time the batch was created, in Unix seconds |
| `ended` | integer or null | Time the batch ended, in Unix seconds |
| `cancelled` | integer or null | Time a cancel was requested, in Unix seconds |
| `paused` | integer or null | Start of the current pause, in Unix seconds |
| `expires` | integer or null | Time the batch expires if still open, in Unix seconds. `null` while paused and once ended |
| `expired` | integer or null | Time the batch expired, in Unix seconds |
| `error` | object or null | Set while `paused`: `out_of_credit`, with `action_url`, the page to add credit |
| `estimated_cost_usd` | number or null | Estimated price of every request in the batch. `null` in test mode, and if the price could not be read when the batch was created |
| `balance_needed_usd` | number or null | Credit to add to run every request not yet started, against what the organization can spend (balance plus credit limit, less twins in progress). `null` if that covers them, and once ended |
| `livemode` | boolean | `false` for a test-mode batch |
| `request_counts` | object | Number of requests in each item status, and their `total` |
| `results_url` | string or null | Path of the [results file](#download-a-batchs-results). Set once the batch has ended |

## Statuses

| Status | Meaning |
| - | - |
| `in_progress` | At least one request is queued or running |
| `paused` | The organization ran out of credit. Queued requests wait; running twins finish. Resumes where it stopped when credit is added or other twins finish |
| `cancelling` | Cancel requested; running twins are finishing |
| `completed` | Every request has ended. Check `request_counts.failed`. Final |
| `cancelled` | Cancelled, and every running twin has ended. Final |
| `expired` | Open for 7 days, not counting time paused. Final |

A batch expires 7 days after creation, not counting time paused. At expiry, open requests are set to `expired` with error `batch_expired`, and running twins are cancelled. A paused batch does not expire.

## The batch item object

A `kennet.batch_item` is one request in a batch.

```json theme={"dark"}
{
  "object": "kennet.batch_item",
  "batch_id": "batch_5e0b...",
  "custom_id": "trace-0001",
  "file_id": "file_3a9e...",
  "end_user": "cust_8812",
  "twin_id": "twin_4FFTGxX_b40dXo8F-ObejckZj1DsFKPhqvJFo25CNH0",
  "status": "completed",
  "error": null
}
```

| Field | Type | Description |
| - | - | - |
| `object` | string | `kennet.batch_item` |
| `batch_id` | string | The batch |
| `custom_id` | string | Your ID for the request |
| `file_id` | string | The file to twin |
| `end_user` | string or null | The request's `end_user` |
| `twin_id` | string or null | The request's twin, set once it starts |
| `status` | enum | `queued`, `running`, `paused`, `completed`, `failed`, `cancelled` or `expired` |
| `error` | object or null | `{type, code, message, retryable, action_url}`. Set when `failed` or `expired`, and `out_of_credit` when `paused`. See [item errors](https://docs.metaphi.ai/kennet/api/errors.md#item-errors) |

## Create a batch

`POST /v1/kennet/batches`

Creates a batch of twins. Send JSON with an [`Idempotency-Key`](https://docs.metaphi.ai/kennet/api/idempotency) header. Max body 4 MiB.

| Parameter | Type | Required | Description |
| - | - | - | - |
| `requests` | array | yes | 1 to 10,000 requests |
| `requests[].custom_id` | string | yes | Your ID for the request, unique in the batch. 1 to 64 letters, digits, `_` and `-` |
| `requests[].file_id` | string | yes | A file not used in any batch. Each file once |
| `requests[].end_user` | string | no | Your ID for the trace's customer, as on [create a twin](https://docs.metaphi.ai/kennet/api/twins.md#create-a-twin) |
| `confidential` | array of strings | no | Public organisations to treat as confidential, for every request |

```sh theme={"dark"}
curl https://api.metaphi.ai/v1/kennet/batches \
  -H "Authorization: Bearer $API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {"custom_id": "trace-0001", "file_id": "file_3a9e...", "end_user": "cust_8812"},
      {"custom_id": "trace-0002", "file_id": "file_71c4..."}
    ]
  }'
```

**Returns** `202` with a `batch` object and a `Location` header. `status` is `in_progress`, or `paused` if the organization is out of credit. Each request creates a `twin`; list them with [`GET /v1/kennet/batches/{id}/items`](#list-a-batchs-items).

The whole batch is validated first. If any request is invalid, nothing is created.

| Status | Code | Cause |
| - | - | - |
| 400 | `parameter_invalid` | An invalid request; the message names it, such as `requests.3.file_id`. Includes an invalid `custom_id`, a file that is not yours or is already used, and a file named twice |
| 400 | `parameter_unknown` | An unknown field |
| 400 | `json_invalid` | The body is not valid JSON |
| 400 | `invalid_end_user` | Invalid `end_user` |
| 409 | `idempotency_key_reused`, `idempotency_key_in_use` | See [Idempotency](https://docs.metaphi.ai/kennet/api/idempotency.md) |
| 413 | `body_too_large` | Body over 4 MiB |

## Retrieve a batch

`GET /v1/kennet/batches/{id}`

Retrieves a batch with its current `request_counts`. Poll no more than every 30 seconds, or use [webhooks](https://docs.metaphi.ai/kennet/api/webhooks.md).

**Returns** a `batch` object, or `404 resource_missing`.

## List batches

`GET /v1/kennet/batches`

Lists batches, newest first.

| Parameter | Type | Required | Description |
| - | - | - | - |
| `limit` | integer | no | 1 to 100. Default 10 |
| `starting_after` | string | no | A batch ID. Returns the page after it |

**Returns** `{object: "list", data, has_more, url}`, with `batch` objects in `data`.

## List a batch's items

`GET /v1/kennet/batches/{id}/items`

Lists a batch's requests, in the order sent.

| Parameter | Type | Required | Description |
| - | - | - | - |
| `limit` | integer | no | 1 to 100. Default 100 |
| `starting_after` | string | no | A `custom_id`. Returns the page after it |
| `status` | enum | no | Only requests in this status: `queued`, `running`, `paused`, `completed`, `failed`, `cancelled` or `expired` |

**Returns** `{object: "list", data, has_more, url}`, with `kennet.batch_item` objects in `data`. If `has_more` is `true`, pass the last `custom_id` as `starting_after`.

## Download a batch's results

`GET /v1/kennet/batches/{id}/results`

Downloads a batch's items as JSON Lines, one `kennet.batch_item` per line, in the order sent. Available once the batch has ended; `results_url` gives the path.

```sh theme={"dark"}
curl https://api.metaphi.ai/v1/kennet/batches/batch_5e0b.../results \
  -H "Authorization: Bearer $API_KEY" -o results.jsonl
```

**Returns** `200`, `application/x-ndjson`. Download each completed item's twin with [`GET /v1/kennet/twins/{twin_id}/content`](https://docs.metaphi.ai/kennet/api/twins#download-a-twins-content).

| Status | Code | Cause |
| - | - | - |
| 409 | `batch_not_ended` | The batch has not ended. Retry once `results_url` is set |

To retry failed or expired requests, upload their traces again and create a new batch.

## Cancel a batch

`POST /v1/kennet/batches/{id}/cancel`

Cancels a batch. Requests not yet started are set to `cancelled`; running twins finish.

**Returns** a `batch` object, `status: cancelling`. The batch moves to `cancelled` and sends `kennet.batch.cancelled` when its running twins end.

| Status | Code | Cause |
| - | - | - |
| 409 | `batch_already_ended` | The batch has ended |

## Example: a directory of traces

Uploads every trace in `traces/`, creates one batch per 10,000 files, and downloads each completed twin into `twins/`.

```python theme={"dark"}
import json, os, pathlib, time, uuid
from concurrent.futures import ThreadPoolExecutor
import requests

API = "https://api.metaphi.ai/v1/kennet"
S = requests.Session()
S.headers["Authorization"] = f"Bearer {os.environ['API_KEY']}"

def upload(path):
    with path.open("rb") as f:
        r = S.post(f"{API}/files", files={"file": (path.name, f)})
    r.raise_for_status()
    return path.stem, r.json()["id"]

paths = sorted(pathlib.Path("traces").glob("*.jsonl"))
with ThreadPoolExecutor(16) as pool:
    files = list(pool.map(upload, paths))

batches = []
for i in range(0, len(files), 10_000):
    reqs = [{"custom_id": cid, "file_id": fid} for cid, fid in files[i:i + 10_000]]
    r = S.post(f"{API}/batches", json={"requests": reqs},
               headers={"Idempotency-Key": str(uuid.uuid4())})
    r.raise_for_status()
    batches.append(r.json()["id"])

out = pathlib.Path("twins")
out.mkdir(exist_ok=True)
for batch_id in batches:
    # Or wait for kennet.batch.completed / .cancelled / .expired
    while (batch := S.get(f"{API}/batches/{batch_id}").json())["results_url"] is None:
        time.sleep(60)
    results = S.get(f"{API}/batches/{batch_id}/results")
    results.raise_for_status()
    for line in results.text.splitlines():
        item = json.loads(line)
        if item["status"] != "completed":
            continue
        with S.get(f"{API}/twins/{item['twin_id']}/content", stream=True) as r:
            r.raise_for_status()
            with (out / f"{item['custom_id']}.jsonl").open("wb") as f:
                for chunk in r.iter_content(1 << 20):
                    f.write(chunk)
```

Store each batch's `Idempotency-Key` with the batch, so a retry after a timeout returns that batch.

## Test mode

Test mode allows 2 MiB files and 5 twins per organization per 24 hours. Requests beyond that fail with `test_quota_exhausted`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.