# Download exports (/en/api/exports)

The export files contain all the data of a project. When a scrape or a matching completes, the API writes the files again with the new data. The job stays in the `EXPORTING` status until the files are ready, and then changes to `COMPLETED`.

## Datasets [#datasets]

A self-service project has these datasets:

| Dataset | Content | Files |
| --- | --- | --- |
| [`all_products_with_details`](/en/datasets/products.md) | The products of one website, with all fields. | One file for each website. There is no file for the whole project. |
| [`matches`](/en/datasets/matches.md) | The product pairs from the matching jobs. | One file for the project. |

The datasets `all_products_with_products` and `prices_and_availability_by_website` are only for managed customers. To compare your prices with your competitors, join `matches` with the product files of the websites. Use `website_id` and `product_id` on both sides of a pair.

The files are JSON Lines: one JSON object for each row, one row on each line. They are compressed with gzip (`.jsonl.gz`).

## List the files [#list-the-files]

Send [`GET /exports`](/en/api/reference/exports/list-exports.md) with the `project_id`:

```bash tab="cURL" tab-group="lang"
curl "https://api.deepscout.ai/v1/exports?project_id=$PROJECT_ID" \
  -H "Authorization: Bearer $SCOUT_API_KEY"
```

```python tab="Python"
exports = api.get(f"{BASE}/exports", params={"project_id": project_id}).json()
for file in exports["exports"]:
    if file["exists"]:
        print(file["dataset"], file.get("website_id"), file["size_bytes"], file["download_url"])
```

```json title="Response"
{
  "enabled": true,
  "generating": false,
  "exports": [
    {
      "dataset": "all_products_with_details",
      "website_id": "example-shop.cz",
      "file_name": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz_all_products_with_details_example-shop.cz.jsonl.gz",
      "download_url": "https://api.deepscout.ai/v1/exports/all_products_with_details?project_id=proj_01k7c2r9v8e3s0b1x6m4q5n2tz&website_id=example-shop.cz",
      "exists": true,
      "size_bytes": 11500000,
      "updated_at": "2026-10-13T15:45:00.000Z"
    },
    {
      "dataset": "matches",
      "file_name": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz_matches.jsonl.gz",
      "download_url": "https://api.deepscout.ai/v1/exports/matches?project_id=proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
      "exists": false
    }
  ]
}
```

| Field | Description |
| --- | --- |
| `enabled` | `false` if the project has no exports. Then the list is empty. |
| `generating` | `true` while the API writes new files. Until it is done, the listed files are from the previous export. |
| `exists` | `false` if the file is not written yet, for example the `matches` file before the first matching. |
| `website_id` | The website of the file. The `matches` file is for the whole project, so it has no `website_id`. |
| `download_url` | The URL to download the file with your API key. |

## Download a file [#download-a-file]

Send [`GET /exports/{dataset}`](/en/api/reference/exports/get-export.md), or use the `download_url` from the list. The API answers with a redirect (`302`) to a temporary storage URL. Let your HTTP client follow the redirect. The `all_products_with_details` files also need `website_id`.

```bash tab="cURL" tab-group="lang"
curl -L "https://api.deepscout.ai/v1/exports/all_products_with_details?project_id=$PROJECT_ID&website_id=$WEBSITE_ID" \
  -H "Authorization: Bearer $SCOUT_API_KEY" \
  -o products.jsonl.gz
```

```python tab="Python"
import shutil

with api.get(
    f"{BASE}/exports/all_products_with_details",
    params={"project_id": project_id, "website_id": website_id},
    stream=True,
) as response:
    response.raise_for_status()
    with open("products.jsonl.gz", "wb") as file:
        shutil.copyfileobj(response.raw, file)
```

```js tab="Node.js"
import { writeFile } from 'node:fs/promises';

const url = `https://api.deepscout.ai/v1/exports/all_products_with_details?project_id=${process.env.PROJECT_ID}&website_id=${process.env.WEBSITE_ID}`;
const response = await fetch(url, {
  headers: { Authorization: `Bearer ${process.env.SCOUT_API_KEY}` },
});
await writeFile('products.jsonl.gz', Buffer.from(await response.arrayBuffer()));
```

The temporary URL is valid for 1 hour. If you only need the URL, for example to give it to another system, send [`GET /exports/{dataset}/url`](/en/api/reference/exports/get-export-url.md). It returns `{"url": "..."}` instead of a redirect.

<Callout title="Keys for one project">
With an API key for one project (`PROJECT` scope), this endpoint always uses the project of the key. You can leave out `project_id`. Such a key is good for a data warehouse or a scheduled import. See [Key scopes](/en/api/authentication.md#key-scopes).
</Callout>

## Read a file [#read-a-file]

Each line of the file is one JSON object.

```bash tab="Shell" tab-group="read"
gunzip -c products.jsonl.gz | head -n 1 | jq .
```

```python tab="Python"
import gzip
import json

with gzip.open("products.jsonl.gz", "rt", encoding="utf-8") as file:
    for line in file:
        product = json.loads(line)
        print(product["website_id"], product["name"], product["price"])
```

The rows are not in random order. Thus the first rows of a file are not a sample of the data.

The [Datasets](/en/datasets.md) pages describe the fields of each dataset.

## Know when new files are ready [#know-when-new-files-are-ready]

- A scrape or a matching job is `COMPLETED` only after the export files have its data. Wait for `COMPLETED`, then download.
- `generating` in [`GET /exports`](/en/api/reference/exports/list-exports.md) is `true` while an export runs. When it changes to `false`, the files are new.
- `updated_at` of each file shows when the API wrote it.

The API writes the files only after a job. There is no scheduled export.

## Errors [#errors]

| Status | Code | Cause |
| --- | --- | --- |
| `400` | `PROJECT_ID_REQUIRED` | A `USER` key needs the `project_id` parameter. |
| `403` | `EXPORTS_NOT_ENABLED` | The project has no exports. |
| `404` | `DATASET_NOT_FOUND` | The project has no such dataset, or the dataset has no files for one website. |
| `404` | `EXPORT_FILE_NOT_FOUND` | The API did not write the file yet. Wait for the first completed job. |