---
title: "Fetch a page"
summary: "Give Kerf an address and get the page as Markdown, in the order a person reads it, as a job you can wait for or collect."
category: "developer"
tags: ["kerf", "reference", "extract", "url", "markdown"]
updated: "2026-09-14T04:25:17.550Z"
url: "https://support.quorumtech.ch/articles/kerf-extract-url"
markdown: "https://support.quorumtech.ch/articles/kerf-extract-url.md"
---

# Fetch a page

`POST https://api.quorumtech.ch/kerf/v1/extract/url`

Fetch a page and convert it to Markdown.

Fetches the address you give it and returns Markdown whose reading order follows the page's
layout rather than the order of the markup: a two-column article is read column by column,
a table arrives as rows, and navigation, cookie notices and footers are dropped.

The answer is a job. Most pages are ready in a second or two, so `wait` holds the request
open for up to that many milliseconds and returns the finished extraction directly; past
the wait you get a `202` with an `id` to collect from `GET /kerf/v1/jobs/{jobId}`. Where the
service holds a short cache of pages it fetched recently, a hit answers `200` with
`cached: true` and no job to collect.

A page that only renders in a browser is fetched in one, which is slower. You do not choose
and you are not charged differently: a page costs a tenth of a credit whichever way it is
fetched.

The tenth of a credit is held when the job is accepted and charged when you collect the
page (`X-Credits-Reserved` on the `202`, `X-Credits-Charged` on the collection). A job that
comes back with no page costs nothing. A page answered inline, because `wait` covered it or
it came from cache, is charged on the spot.

The call to start with. Send an address, read Markdown.

Set `wait` to the milliseconds you are willing to hold the request open. Most pages finish inside two seconds, so a `wait` of a few thousand usually means one call and no job to follow. Past the wait you get a `202` carrying an `id`; collect it from `GET /kerf/v1/jobs/{jobId}` when it is ready, or watch `.../events`. Collecting is free.

The tenth of a credit is held when the job is accepted and charged when you collect the page, so a fetch that comes back with nothing costs nothing. A page the request itself carries, because `wait` covered it or the cache did, is charged there and then.

Leave `mode` alone unless you have a reason. `auto` fetches the page directly and opens a browser only when what came back is a shell rather than an article, and the price is the same either way, so the default is both the fastest common case and the most reliable one.

Two options exist for particular work. `location` fetches the page from a country, for prices and terms as a visitor there sees them; `optimiseForRAG` shapes the Markdown for a chunk-and-retrieve reader. The guides on [fetching from a country](/docs/kerf-location) and [Markdown for retrieval](/docs/kerf-for-rag) say when each is worth turning on.

The service may answer from a short cache of pages it fetched recently, as a `200` carrying `cached: true` and no job id. It is charged like any other page: the cache is there to spare the site, not to change your bill. It is off on this deployment today, so every call is a fresh fetch.

```json
{
  "url": "https://example.com/",
  "wait": 4000
}
```

## Request

The address, and the few choices that change what comes back. The defaults suit reading a page: the fetch picks its own path, links are kept, images are not, and the site's robots.txt is not consulted.

A JSON body with these fields:

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `url` | string | yes | The page to fetch. `http` or `https`, and a public address. |
| `wait` | integer | no | Milliseconds to hold the request open for a page that finishes quickly, so a short fetch needs no second call. Past it you get a `202` with a job to collect. Ignored by `/kerf/v1/render`, which always waits. |
| `mode` | `auto` or `fast` or `browser` | no | `auto` fetches the page directly and falls back to a browser when what came back is a shell rather than the article. `fast` never opens a browser and fails instead. `browser` always opens one. The price is the same for all three. Default `"auto"`. |
| `furniture` | `drop` or `end` | no | Navigation, cookie notices, share bars and footers: dropped, or kept and moved to the end of the Markdown where they do not interrupt the reading. Default `"drop"`. |
| `hidden` | `include` or `drop` | no | Text the page holds but does not show yet: inactive tabs, closed accordions, collapsed panels. Included, it is written after the visible text under a `<!-- folded -->` comment, which is where a pricing page keeps its yearly prices. `drop` leaves it out. Default `"include"`. |
| `linksEnabled` | boolean | no | Keep links as Markdown links. Off, the text stays and the addresses go. Default `true`. |
| `imagesEnabled` | boolean | no | Include every image with an address. Off, only images carrying a description are kept, as alt text. Default `false`. |
| `followRobots` | boolean | no | Consult the site's robots.txt before fetching, and refuse the page with `robots_disallowed` when it disallows it. Off by default because this fetches one page you named rather than crawling a site. Default `false`. |
| `location` | `us` or `gb` or `de` or `ch` or `in` or `sg` | no | Fetch the page from a country, so prices, currency and terms are the ones a visitor there sees. A region the fleet does not offer answers `400 location_unavailable`. When the country cannot be reached the page is fetched from our own address and `report.locationFallback` says so; with `strictLocation` it answers `424 location_blocked` instead, and nothing is charged. |
| `strictLocation` | boolean | no | With `location`: fail with `location_blocked` rather than fall back to our own address. Nothing is charged for a page that fails this way. Default `false`. |
| `optimiseForRAG` | boolean | no | Shape the Markdown for a reader that cuts it into chunks and retrieves by similarity, so a chunk holds whole facts: a name stays with its figures and a table row keeps its column names. Fewer headings and no Markdown tables come back, so leave it off when people or a table renderer read the result. How the shaping is done changes as the extractor does; depend on the outcome, not on the shape of a line. Default `false`. |
| `humanizeEnabled` | boolean | no | Render in a browser that moves its cursor the way a person does. Slower, rarely needed, and it implies `mode: browser`. Cannot be combined with `location`. Default `false`. |

## Example request

```bash
curl -X POST "https://api.quorumtech.ch/kerf/v1/extract/url" \
  -H "Authorization: Bearer $QUORUM_API_KEY" \
  -H "content-type: application/json" \
  -d '{
  "url": "https://example.com/",
  "wait": 4000
}'
```

## Responses

### 200

The finished extraction, either because `wait` covered it or because the page was in cache. A cached answer carries `cached: true` and costs the same as a fresh one.

Every shape below carries these fields:

| Field | Type | Description |
| --- | --- | --- |
| `state` | `done` |  |
| `result` | Extraction | One page, converted. |
| `result.url` | string | The address asked for. |
| `result.finalUrl` | string | The address actually read, after any redirects. |
| `result.status` | integer | The HTTP status the site answered with. |
| `result.title` | string | May be null. |
| `result.markdown` | string | The page, in reading order. |
| `result.report` | ExtractionReport | What the fetch did, for logging and for deciding whether to trust a thin result. The fields below are stable; the report also carries diagnostics that change as the extractor does, so read it by name rather than by shape. |
| `result.report.path` | `fast` or `browser` | Whether the page was fetched directly or rendered in a browser. |
| `result.report.words` | integer | Words in the Markdown. A page that should be an article and is not is usually a wall rather than an empty page. |
| `result.report.items` | integer | How many blocks the page was read as, counting paragraphs, headings, list items and table rows. |
| `result.report.tables` | integer | Tables recovered, including ones drawn as boxes rather than marked up as tables. |
| `result.report.images` | integer |  |
| `result.report.links` | integer |  |
| `result.report.uncertain` | boolean | The layout did not settle into one reading order, so the Markdown may not read the way the page does. Browser path only. |
| `result.report.interstitial` | string | A wall the page put up, when it did: what kind, and whether waiting cleared it. Present only when one was met. |
| `result.report.fastFallback` | string | Why the direct read was not served, when the page went to a browser after one: `client_rendered`, a `script_filled:` reason, or the direct read's failure. |
| `result.report.location` | string | The region the page was asked from, when one was. |
| `result.report.locationFallback` | object | Present when a `location` could not be served and the page came from our own address instead. `to` is `direct`; `reason` is `refused` or `thin`. |
| `result.report.locationFallback.to` | string |  |
| `result.report.locationFallback.reason` | string |  |
| `result.report.optimiseForRAG` | boolean | Whether the Markdown was shaped for retrieval. |
| `result.renderMs` | integer | Milliseconds spent fetching and converting. |
| `result.cached` | boolean | Present and `true` when this came from a recent fetch of the same page rather than a new one. |

#### Extraction job done

A finished job, with the page.

| Field | Type | Description |
| --- | --- | --- |
| `id` | string |  |
| `url` | string |  |
| `queuedMs` | integer |  |
| `renderMs` | integer |  |
| `edge` | string |  |
| `attempts` | integer |  |

#### Cached extraction

A page fetched recently enough to be answered from cache. There is no job to collect, so there is no id.

| Field | Type | Description |
| --- | --- | --- |
| `cached` | `true` |  |

### 202

The job was accepted. Collect it at the `Location` header's address.

| Field | Type | Description |
| --- | --- | --- |
| `id` | string |  |
| `state` | `queued` or `running` |  |
| `url` | string |  |
| `queuedMs` | integer | Milliseconds the job waited before an edge took it. |
| `renderMs` | integer | Null until the job finishes. May be null. |
| `edge` | string | Which edge is fetching it. May be null. |
| `attempts` | integer |  |

## Errors

The body is `{ "error": { "code", "message" } }`; the full catalogue with what to do about each code is on the Errors page.

| Status | Codes | When |
| --- | --- | --- |
| 400 |  | The request does not match the schema. `error.issues` names each failing field. |
| 401 |  | The key is missing, not recognised, or revoked. |
| 402 | `payment_required`, `credits_exhausted`, `remaining`, `needed` | The workspace has no active API subscription (`payment_required`), or is prepaid and has no credits left for this request (`credits_exhausted`, with `remaining` and `needed`). Manage billing and add credits at https://developer.quorumtech.ch/billing. |
| 403 | `insufficient_scope`, `url_not_allowed`, `robots_disallowed` | The key is not scoped to Kerf (`insufficient_scope`), the address is one this service does not fetch (`url_not_allowed`: a private or loopback address, or a redirect to one), or the site's robots.txt disallows the page and you asked for it to be followed (`robots_disallowed`). |
| 424 | `site_refused`, `location_blocked`, `load_failed`, `timeout` | The page could not be fetched, and the site is the reason rather than us: it refused to serve the page (`site_refused`, which is the site answering 401, 403, 407, 429 or 451, or a wall it would not lower), the country asked for with `strictLocation` could not serve it (`location_blocked`), or it would not load (`load_failed`, `timeout`). Nothing is charged for a request that answers this, and sending it again unchanged answers the same way. |
| 429 | `rate_limited`, `busy` | Over the key's requests-per-minute limit (`rate_limited`), or the fetching fleet's queue is full (`busy`). Both carry `Retry-After`; a `busy` clears in seconds. |
| 503 | `no_edge_available` | No part of the fetching fleet is answering (`no_edge_available`). Retry shortly. |

## Cost

A tenth of a credit ($0.001) a page, whichever way the page is fetched. Held when the job is accepted and charged when you collect the page; a page answered inline or from cache is charged on the spot. A page that never arrives charges nothing.