# Aktivace exportů (/activation)
Tato stránka platí pro projekty, které spravuje DeepScout. V self-service projektu jsou exporty zapnuté vždy. Viz [Stažení exportů](/api/exports.md).
## Aktivace [#aktivace]
Exporty se zapínají v aplikaci v sekci [Nastavení → Exporty](https://reports.deepscout.cz/nastaveni). Stačí kliknout na **Zapnout exporty** a vybrat formát (JSON nebo XML). Po aktivaci se automaticky vygenerují všechny výchozí datasety — může to trvat přibližně 15 minut.
{/* TODO: přidat screenshoty z Reports UI */}
## API klíč [#api-klíč]
Pro přístup k exportům potřebujete API klíč, který najdete v sekci [Nastavení → API](https://reports.deepscout.cz/nastaveni).
## Přístup k datům [#přístup-k-datům]
Po aktivaci jsou data dostupná na adrese níže. API klíč posílejte v hlavičce `Authorization`. Odpověď přesměruje na dočasnou adresu souboru, proto v `curl` použijte `-L`.
```
GET https://api.deepscout.ai/v1/exports/{dataset}
Authorization: Bearer VÁŠ_API_KLÍČ
```
```bash
curl -L https://api.deepscout.ai/v1/exports/all_products_with_details \
-H "Authorization: Bearer VÁŠ_API_KLÍČ" \
-o all_products_with_details.jsonl.gz
```
Starší parametr `?token=VÁŠ_API_KLÍČ` stále funguje, ale je zastaralý.
## Aktualizace dat [#aktualizace-dat]
Exporty aktualizujeme po každém zpracování dat, obvykle každých 8 hodin. API vždy vrátí poslední soubor.
## Chyby [#chyby]
| Kód | Význam |
|---|---|
| `401` | Neplatný API klíč. |
| `403` | Exporty nejsou zapnuté, nebo je projekt zastavený. |
| `404` | Dataset není v projektu zapnutý, nebo soubor ještě není připravený. |
| `503` | Údržba. Zkuste to později. |
## Formáty [#formáty]
| Formát | Popis |
|---|---|
| **JSON** | JSONL — jeden JSON objekt na řádek. Výchozí formát. |
| **XML** | XML feed. |
# Add a website (/api/add-website)
A setup job adds a website to your project. The job does three things:
1. **Analysis.** It finds the protections of the website, the language, the type of products and the size of the catalog. This usually takes a few minutes.
2. **Training.** It scrapes a small part of the website to learn how to read the product pages. This can take up to about two hours.
3. **Sample.** It extracts up to 1,000 sample products, so that you can examine the data before you pay.
## Before you start [#before-you-start]
Each new setup job uses one website analysis of your customer. The customer gets 10 free analyses each month. After them, an analysis costs €0.50 from your [credit](/api/credits.md). To see the free analyses that remain and the price, read the customer with a `USER` key:
```bash
curl https://api.deepscout.ai/v1/customers/$CUSTOMER_ID \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```json title="Response"
{
"customer_id": "cust_01k7c2r9t2f8a5d3h7j9k1m6pw",
"name": "Example Ltd",
"role": "OWNER",
"quotas": {
"analyses_remaining": 9,
"analyses_per_month": 10,
"currency": "EUR",
"analysis_price": "0.50"
}
}
```
The `customer_id` of a project is in [`GET /projects`](/api/reference/projects/list-projects.md).
When `analyses_remaining` is `0`, the next setup job is paid:
1. When the API creates the job, it holds `analysis_price` from your credit. If the spendable credit is too low, the API answers `402 INSUFFICIENT_FUNDS`. If the price does not fit in the [monthly spend limit](/api/credits.md#monthly-spend-limit), it answers `403 MONTHLY_LIMIT_REACHED`.
2. When the analysis adds the website to your project, the API charges the price.
3. If the analysis fails, the API releases the hold and charges nothing. The exception is an analysis that finds that the website does not offer your currency. See [The website does not offer the currency](#the-website-does-not-offer-the-currency).
A failed analysis also gives a free analysis back, with the same exception. At the start of each month (UTC), the free analyses go back to 10. Free analyses that you do not use do not carry over.
## Start the setup job [#start-the-setup-job]
Send [`POST /jobs/setup`](/api/reference/setup-jobs/create-setup-job.md):
| Field | Description |
| --- | --- |
| `project_id` | The project that gets the website. |
| `url` | The URL of the store, for example `https://www.example-shop.cz`. The API uses only the domain. It ignores the path and the query. |
| `country` | The country of the shoppers, as a two-letter country code (ISO 3166-1 alpha-2), for example `CZ`. `UK` is also accepted for `GB`. The scraper browses the store as a shopper from this country. |
| `currency` | The currency of the prices, as an ISO 4217 code, for example `CZK`. The website must offer this currency. To compare stores that sell in different currencies, use the currency of each store. The [localization](/api/localization.md) of the project converts the prices. |
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/setup \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"url": "https://www.example-shop.cz",
"country": "CZ",
"currency": "CZK"
}'
```
```python tab="Python"
job = api.post(f"{BASE}/jobs/setup", json={
"project_id": project_id,
"url": "https://www.example-shop.cz",
"country": "CZ",
"currency": "CZK",
}).json()
```
The API answers `201 Created` with the new job. If the project already has an open setup job for the same website, the API returns this job with `200 OK`. It does not use an analysis.
The API accepts the country and the currency in any letter case. It returns the country in lower case (`cz`) and the currency in upper case (`CZK`).
If the project has no localization, the setup job gives it this country and this currency when it adds the website. See [Localization](/api/localization.md#how-a-project-gets-its-localization).
The setup is always for the whole store. To scrape only some products, add the website first. Then [scrape a URL list](/api/url-scrape.md).
## Follow the job [#follow-the-job]
Read the job with [`GET /jobs/setup/{job_id}`](/api/reference/setup-jobs/get-setup-job.md). The job goes through these statuses:
| Status | What happens |
| --- | --- |
| `CREATED` | The job is in the queue. |
| `ANALYZING` | The analysis runs. `analysis_progress` shows each step. |
| `ESTIMATE_READY` | The analysis is complete. The website is now in the project with its `full_scrape_estimate`, and the job has its `website_id`. |
| `TRAINING` | The training scrape runs. `train_progress` shows the phase and the percent. |
| `COMPLETED` | The website is ready, with its sample data. |
If DeepScout already knows the website, the job skips the analysis. If the scraper of the website is already trained, the job also skips the training and completes in a few seconds.
### Analysis [#analysis]
`analysis_progress` has four steps. Each step has a `state`: `RUNNING`, `DONE` or `FAILED`. A step that has not started is not in the response.
| Step | What it finds | Result fields |
| --- | --- | --- |
| `config` | The protections of the website and how to get past them. | `challenge` on the website |
| `homepage` | If the website is a store, its language and its type of products. | `is_ecommerce`, `language`, `product_type` |
| `sitemap` | The URLs in the sitemap. | `sitemap_size` |
| `size` | The number of products in the catalog. | `products_estimate` |
If the home page shows another currency than the currency of the job, the analysis tries to switch the website to your currency. This can make the analysis some minutes longer.
```json title="A setup job during the analysis"
{
"job_id": "jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3",
"job_type": "WEBSITE_SETUP",
"status": "ANALYZING",
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"url": "https://www.example-shop.cz",
"country": "cz",
"currency": "CZK",
"analysis_progress": {
"config": { "state": "DONE" },
"homepage": { "state": "DONE" },
"sitemap": { "state": "RUNNING" },
"is_ecommerce": true,
"product_type": "health and household products",
"language": "cs"
},
"created_at": "2026-10-13T09:00:00.000Z",
"updated_at": "2026-10-13T09:02:10.000Z"
}
```
### Training [#training]
When the analysis is complete, the job gets its `website_id` and the website shows in [`GET /websites`](/api/reference/websites/list-websites.md). Then the training scrape starts. `train_progress` shows how far it is:
| `phase` | Console label | Typical time left |
| --- | --- | --- |
| `COLLECTING_PAGES` | Collecting pages | About 2 hours |
| `ANALYZING_PAGES` | Analyzing pages | Less than 1 hour |
| `UNDERSTANDING_PRODUCTS` | Understanding products | Less than 30 minutes |
| `EXTRACTING_SAMPLE` | Extracting sample products | A few minutes |
| `PREPARING_SAMPLE` | Preparing your sample | A few minutes |
| `QUALITY_CHECK` | Final check | A few minutes |
```json title="train_progress during the training"
{
"phase": "COLLECTING_PAGES",
"percent": 10,
"pages": 800,
"pages_estimate": 2600,
"pages_limit": 5000
}
```
`percent` is the progress of the whole training. While the phase is `COLLECTING_PAGES`, `pages` is the number of pages that the scraper downloaded.
### Wait for the result [#wait-for-the-result]
A setup job takes minutes to hours, so read it every 30 to 60 seconds. You also get an email when the training completes or fails.
```python
import time
TERMINAL = {"COMPLETED", "FAILED", "CANCELLED"}
while job["status"] not in TERMINAL:
time.sleep(30)
job = api.get(f"{BASE}/jobs/setup/{job['job_id']}").json()
print(job["status"], job.get("train_progress", {}).get("percent"))
website_id = job.get("website_id")
```
When the status is `COMPLETED`:
- [Check the sample data](/api/sample-data.md).
- Read the full-scrape estimate in `full_scrape_estimate` of [`GET /websites/{website_id}`](/api/reference/websites/get-website.md).
- [Scrape the full catalog](/api/full-scrape.md) or [scrape a URL list](/api/url-scrape.md).
## When the setup fails [#when-the-setup-fails]
A failed job has the status `FAILED`, an `error` text and an `error_code`. If a team can help, the job also has a `contact_reason`, which tells you which team.
| `error_code` | Cause | What to do |
| --- | --- | --- |
| `WEBSITE_TOO_LARGE` | The catalog has more than 2,000,000 products, or the website is a large marketplace. | Ask the sales team for a custom offer. You get the analysis back. |
| `CURRENCY_NOT_AVAILABLE` | The website does not offer the currency of the job. | Start a new setup job with a currency that the website offers. The analysis is used. See [The website does not offer the currency](#the-website-does-not-offer-the-currency). |
| `ANALYSIS_FAILED` | The analysis was not able to read the website, for example because the website blocked it. | Make sure that the URL is correct, then start a new setup job. You get the analysis back. |
| `TRAINING_FAILED` | The training was not able to read the product pages. | Ask the support team for help. |
To ask for help, send [`POST /contact-requests`](/api/reference/contact/create-contact-request.md) with the `contact_reason` and the `job_id` of the failed job:
```bash
curl https://api.deepscout.ai/v1/contact-requests \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"reason": "WEBSITE_TOO_LARGE",
"job_id": "jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3",
"phone": "+420 123 456 789",
"message": "We need only the pharmacy category."
}'
```
The team contacts you by email, or by phone if you give a number. `phone` and `message` are optional. See [Failed jobs](/api/jobs.md#failed-jobs).
### The website does not offer the currency [#the-website-does-not-offer-the-currency]
The analysis checks if the website can show its prices in the currency of the job. For example, it tries the currency switcher of the website. If no switcher offers the currency, or the switch does not change the prices, the job fails with the error code `CURRENCY_NOT_AVAILABLE`. The job fails early, so you do not wait for a training that cannot work.
```json title="The failure fields of the job"
{
"status": "FAILED",
"currency": "EUR",
"error": "www.example-shop.cz does not offer prices in EUR, it shows CZK. Run the analysis again with a currency the website offers.",
"error_code": "CURRENCY_NOT_AVAILABLE"
}
```
- The `error` names the currency that the website shows, if the analysis found it.
- The job has no `website_id` and no `contact_reason`. You can fix the cause yourself, so a contact request about the job gets `409 CONTACT_REASON_MISMATCH`.
- The job uses its analysis. A free analysis stays used, and the API charges a paid analysis.
- You get an email.
- DeepScout keeps the result of the analysis. A new setup job with the same URL, country and currency fails at once, and uses another analysis.
To add the website, start a new setup job with a currency that the website offers, usually the currency in the `error`. The exports still show the prices in the currency of your project, because the [localization](/api/localization.md#currency-conversion) converts them.
In the console, click **Retry analysis** on the failed website. The dialog shows the error, and you can select another currency.
If the analysis cannot check the currency, for example because the website did not load, the setup continues.
## Remove a website [#remove-a-website]
To remove a website from the project, send [`DELETE /websites/{website_id}`](/api/reference/websites/remove-website.md) with the `project_id`. You need the `ADMIN` role in the project.
```bash
curl -X DELETE "https://api.deepscout.ai/v1/websites/example-shop.cz?project_id=$PROJECT_ID" \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
- The API refuses the request with `409 WEBSITE_BUSY` while a job of the website runs.
- The API cancels the quotes of the website that wait for approval.
- The current export files keep the products of the website until the next export of the project.
- To add the website again, start a new setup job. It uses one analysis.
A setup job that failed before it found its website has no `website_id`. These jobs show in [`GET /jobs/setup`](/api/reference/setup-jobs/list-setup-jobs.md). To remove one, send [`DELETE /jobs/setup/{job_id}`](/api/reference/setup-jobs/remove-setup-job.md).
To stop a setup job that runs, send [`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md). You do not get the analysis back.
# Authentication (/api/authentication)
## Send the API key [#send-the-api-key]
Send the API key in the `Authorization` header of each request, with the `Bearer` scheme.
```bash
curl https://api.deepscout.ai/v1/projects \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
A request without a key gets `401` with the code `NOT_AUTHENTICATED`. A request with an unknown or revoked key gets `401` with the code `INVALID_API_KEY`.
Older integrations send the key in a `token` header or in a `?token=` query parameter. Both still work, but they are deprecated. Use the `Authorization` header in new code.
## Key scopes [#key-scopes]
An API key always belongs to you, the user who created it. Its scope sets where the key works.
| Scope | Console label | Where the key works |
| --- | --- | --- |
| `USER` | All my projects | All endpoints, for all projects that you can access. |
| `PROJECT` | The name of one project | The project, website, job, sample, export and contact request endpoints, only for this project. |
Use a `PROJECT` key to keep an integration or an agent inside one project. With this key:
- [`GET /projects`](/api/reference/projects/list-projects.md) shows only the project of the key.
- A request for another project gets the same error as a project that you cannot access, for example `404 PROJECT_NOT_FOUND`.
- The account, customer and billing endpoints answer `403 INSUFFICIENT_SCOPE`. The key cannot create keys or projects, and it cannot read or top up the credit.
- The key has your role in the project. With the `ADMIN` or `EDITOR` role, it can approve jobs, and the jobs use the credit of the customer.
Use a `USER` key for a script that also manages keys, projects or the credit.
A request that needs `project_id` with a `USER` key also needs it with a `PROJECT` key. The export download, [`GET /exports/{dataset}`](/api/reference/exports/get-export.md), is the only exception. With a `PROJECT` key, it always uses the project of the key, and `project_id` is optional.
## Manage keys [#manage-keys]
You can manage keys in the console, under **Settings → API keys**, or with the API.
| Action | Request |
| --- | --- |
| List your active keys | [`GET /account/api-keys`](/api/reference/account/list-api-keys.md) |
| Create a key | [`POST /account/api-keys`](/api/reference/account/create-api-key.md) |
| Revoke a key | [`DELETE /account/api-keys/{key_id}`](/api/reference/account/revoke-api-key.md) |
The API shows the full key only one time, in the response to the create request. The list shows only the last four characters, in `key_suffix`.
```bash
curl https://api.deepscout.ai/v1/account/api-keys \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "Data warehouse", "project_id": null}'
```
```json title="Response (201 Created)"
{
"api_key": "Xb3v...full-key...9QaL",
"key_id": "key_01k7c4m2n3p4q5r6s7t8v9w0x1",
"scope": "USER"
}
```
Send `project_id: null` for a `USER` key, or the ID of a project for a `PROJECT` key.
After you revoke a key, the API can accept it for up to 5 more minutes. If a key leaks, revoke it and replace it immediately.
You can have more than one active key. To rotate a key without downtime, create the new key, change your integration to use it, and then revoke the old key.
## Roles [#roles]
What a key can do also depends on your roles. A key has the roles of the user who created it.
### Customer roles [#customer-roles]
A customer is the account that pays. Each user has a role in each customer: `OWNER`, `ADMIN` or `MEMBER`. [`GET /customers`](/api/reference/customers/list-customers.md) shows your role.
| Action | Needed role |
| --- | --- |
| Read the customer, its quotas and its billing | `OWNER` or `ADMIN` |
| Create a project | `OWNER` or `ADMIN` |
| Top up the credit | `OWNER` or `ADMIN` |
### Project roles [#project-roles]
Each user also has a role in each project: `ADMIN`, `EDITOR` or `VIEWER`. When you create a project, you become its `ADMIN`.
| Action | Needed role |
| --- | --- |
| Read websites, jobs, samples and exports | Any role |
| Create, approve and cancel jobs | `ADMIN` or `EDITOR` |
| Ask support about a failed job | `ADMIN` or `EDITOR` |
| Rename the project, change its localization, remove a website, remove a setup job | `ADMIN` |
A request without the needed role gets `403` with the code `INSUFFICIENT_ROLE` or `PROJECT_ACCESS_DENIED`. A project that you cannot access gets `404 PROJECT_NOT_FOUND`. Updating a project, removing a website and creating a project key answer `403 PROJECT_ACCESS_DENIED` instead.
The self-service API shows only the projects of self-service customers. If DeepScout manages your account, use the [export files](/activation.md) of your projects.
# Core concepts (/api/concepts)
## Customer [#customer]
A customer is the account that pays. It holds the prepaid credit, the invoices and the quota of free analyses. Your first sign-in creates a personal customer, and you are its `OWNER`. A customer can have more users, each with a role.
Get the customers that you belong to with [`GET /customers`](/api/reference/customers/list-customers.md).
## Project [#project]
A project keeps websites, jobs and exports separate from other projects. For example, use one project for each market or for each of your clients. Each project belongs to one customer, and the jobs of the project use the credit of this customer.
Every account has a default project. You can create more projects with [`POST /customers/{customer_id}/projects`](/api/reference/customers/create-project.md), up to 50 for each customer. Most requests need a `project_id`.
Each project has a localization: a country and a currency. The exports show the prices of all websites in the currency of the project. See [Localization](/api/localization.md).
## Website [#website]
A website is one online store for one country and one currency. For example, the Czech and the Slovak versions of the same store are two websites. The setup job identifies the store by its domain, so a path or a query in the URL has no effect. The analysis finds the home page of the store for the country and the currency. This home page can be on another domain than the URL that you send. If the store does not offer the currency, the setup fails.
The website ID is a string such as `example-shop.cz`. Use it as an opaque value. Do not build it from the URL yourself.
A website is in your project after its setup job. [`GET /websites`](/api/reference/websites/list-websites.md) lists the websites of a project. Each website shows the job that sets its current state, and the estimate for a full scrape.
## Job [#job]
All work runs as jobs. A job runs in the background, and you read its status to follow it. There are four types:
| Job type | What it does | Cost |
| --- | --- | --- |
| `WEBSITE_SETUP` | Analyzes a website, trains the scraper and makes sample data. | Paid, 10 free each month |
| `FULL_SCRAPE` | Scrapes the full product catalog of a website. | Paid |
| `URL_SCRAPE` | Scrapes a list of product URLs. | Paid |
| `MATCHING` | Finds the same products on different websites. | Paid, one free sample for each project |
Paid jobs do not start immediately. The API first gives a quote, and the job waits in the `AWAITING_APPROVAL` status. When you approve the quote, the job starts. See [Job lifecycle](/api/jobs.md).
## Estimate and charge [#estimate-and-charge]
The quote of a paid job is its `estimate`:
- `quantity`: the number of products in the estimate.
- `unit_price_per_1000`: the price for 1,000 products.
- `total`: the estimated price of the job.
- `cap`: the maximum charge, 1.5 times the `total`.
When you approve the job, the API holds the `cap` from your credit. When the job ends, the API charges the products that the job actually processed, never more than the `cap`. Then it releases the rest of the hold. The charge is in the `settlement` of the job. See [Pricing](/api/pricing.md).
## Credit [#credit]
Paid jobs use prepaid credit in EUR. You buy credit with a top-up by card, and you get an invoice for each top-up. The balance has two parts: the credit that running jobs hold, and the credit that you can spend. See [Credits and billing](/api/credits.md).
## Exports [#exports]
Each project has export files. When a scrape or a matching completes, the API writes the files again with the new data. You download them as gzipped JSON Lines. See [Download exports](/api/exports.md).
## How they connect [#how-they-connect]
```txt
Customer ─── credit, invoices, analyses quota
└── Project ─── exports
├── Website ─── sample data
│ ├── setup job
│ ├── full scrape jobs
│ └── URL scrape jobs
└── matching jobs (between websites of the project)
```
# API conventions (/api/conventions)
## Base URL and versions [#base-url-and-versions]
```txt
https://api.deepscout.ai/v1
```
The `v1` in the path is the version of the API. The OpenAPI specification of the version is at [`/v1/openapi.json`](https://api.deepscout.ai/v1/openapi.json).
## Requests and responses [#requests-and-responses]
- Send request bodies as JSON, with the header `Content-Type: application/json`.
- All responses are JSON, except file downloads.
- Field names are in `snake_case`. Enum values are in `UPPER_CASE`, except the sign-in `provider` of the account (`google` or `github`).
### Fields without a value [#fields-without-a-value]
The API leaves out the fields that have no value. It does not send `null`. For example, a job without a `settlement` has no `settlement` field. In your code, read optional fields with a default:
```python
percent = job.get("scrape_progress", {}).get("percent", 0)
```
Fields with a default value, for example `"sample": false`, are always in the response.
## IDs [#ids]
All IDs are strings. Use them as opaque values: do not build them, and do not parse them.
| ID | Example |
| --- | --- |
| `project_id` | `proj_01k7c2r9v8e3s0b1x6m4q5n2tz` |
| `customer_id` | `cust_01k7c2r9t2f8a5d3h7j9k1m6pw` |
| `job_id` | `jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3` |
| `key_id` | `key_01k7c4m2n3p4q5r6s7t8v9w0x1` |
| `website_id` | `example-shop.cz` |
A website ID can contain a slash (`/`). You can put it in the path as it is.
## Timestamps [#timestamps]
Timestamps are strings in RFC 3339 format, in UTC, with milliseconds:
```txt
2026-10-13T09:00:00.000Z
```
## Money [#money]
Amounts of money are decimal strings, not numbers, with the currency in a separate `currency` field. This keeps the exact value of each amount.
```json
{ "currency": "EUR", "total": "12.88", "cap": "19.32" }
```
Amounts have two decimal places. Convert the strings to a decimal type, not to a float:
```python
from decimal import Decimal
cap = Decimal(job["estimate"]["cap"])
```
## Countries, currencies and languages [#countries-currencies-and-languages]
| Value | Format | Example |
| --- | --- | --- |
| Country | Two-letter country code (ISO 3166-1 alpha-2). The API accepts any letter case and returns lower case. | `cz` |
| Currency | ISO 4217, in upper case. | `CZK` |
| Language | ISO 639-1. | `cs` |
## Lists [#lists]
- The job lists ([setup](/api/reference/setup-jobs/list-setup-jobs.md), [scraping](/api/reference/scraping-jobs/list-scraping-jobs.md), [matching](/api/reference/matching-jobs/list-matching-jobs.md)) show the newest jobs first. `limit` sets the number of jobs, from 1 to 100. The default is 20.
- The other lists return all items.
- Lists are JSON arrays, except the invoices and the usage, which have an `items` field, and the exports, which have an `exports` field.
## Repeated requests [#repeated-requests]
Some create requests are safe to send again:
| Request | When you send it again |
| --- | --- |
| `POST /jobs/setup` | While a setup job for the same store, country and currency runs in the project, you get this job with `200 OK`. After it ends, a new request creates a new job and uses another analysis. |
| `POST /jobs/scraping` without `urls` | While the full scrape of the website is not finished, you get the same job with `200 OK`. |
| `POST /contact-requests` | For the same job, you get the first request with `200 OK`. |
| `POST /jobs/scraping` with `urls` | Each request creates a new job. |
| `POST /jobs/matching` | The API answers `409 MATCHING_BUSY` while the first matching is not finished. |
Approvals are also safe. A second approval of the same job answers `409 JOB_NOT_AWAITING_APPROVAL`, and the API holds the credit only one time.
## Long-running work [#long-running-work]
Jobs run in the background. The API has no webhooks yet. Read the job at an interval until it is in an end status. See [Polling](/api/jobs.md#polling).
# Credits and billing (/api/credits)
Paid jobs use prepaid credit in EUR. The credit belongs to the customer, and all projects of the customer use it. The billing endpoints need a `USER` key and the `OWNER` or the `ADMIN` role in the customer.
## How a job uses credit [#how-a-job-uses-credit]
1. **Approval.** The API holds the `cap` of the job estimate from your credit. If the spendable credit is less than the `cap`, the approval answers `402 INSUFFICIENT_FUNDS`.
2. **Run.** The hold stays while the job runs. You cannot spend the held credit on other jobs.
3. **Settlement.** The API charges the actual cost of the job, never more than the `cap`. It releases the rest of the hold at the same time.
A failed job costs nothing. Its hold goes back to your spendable credit.
A paid website analysis has no quote and no approval. The API holds its price when it creates the setup job. If the analysis finds that the website does not offer the currency of the job, the API charges the price, although the job fails. See [Add a website](/api/add-website.md#before-you-start).
## Read the balance [#read-the-balance]
Send [`GET /customers/{customer_id}/billing/balance`](/api/reference/billing/get-balance.md):
```bash
curl https://api.deepscout.ai/v1/customers/$CUSTOMER_ID/billing/balance \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```json title="Response"
{ "currency": "EUR", "balance": "25.00", "held": "7.20", "spendable": "17.80" }
```
| Field | Description |
| --- | --- |
| `balance` | All your credit. |
| `held` | The credit that approved jobs hold until their settlement. |
| `spendable` | The credit that you can use for a new approval: `balance` − `held`. |
## Top up the credit [#top-up-the-credit]
A top-up is a card payment through Stripe Checkout. A person must complete the payment in a browser.
### Create the checkout [#create-the-checkout]
Send [`POST /customers/{customer_id}/billing/topup`](/api/reference/billing/create-topup.md) with the amount in EUR. The amount must be from 5 to 5,000 EUR, with not more than two decimal places.
```bash
curl https://api.deepscout.ai/v1/customers/$CUSTOMER_ID/billing/topup \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"amount": "50.00", "currency": "EUR"}'
```
```json title="Response (201 Created)"
{
"checkout_url": "https://checkout.stripe.com/c/pay/cs_live_...",
"session_id": "cs_live_a1B2c3D4e5F6"
}
```
If the approval of a job failed with `INSUFFICIENT_FUNDS`, you can send its `job_id` with the top-up. This only connects the payment to the job. It does not approve the job.
### Pay [#pay]
Open the `checkout_url` in a browser and pay by card. Stripe adds the VAT to the amount. Your credit is the amount without VAT. If you are a business in the EU, add your VAT ID in the checkout. Then the reverse charge applies.
### Wait for the credit [#wait-for-the-credit]
Read [`GET /customers/{customer_id}/billing/topups/{session_id}`](/api/reference/billing/get-topup-status.md) every few seconds. When `fulfilled` is `true`, the credit is in your balance.
```json title="Response"
{ "session_id": "cs_live_a1B2c3D4e5F6", "fulfilled": true }
```
### Approve the job again [#approve-the-job-again]
If you topped up for a job, approve it again. See [Approve the quote](/api/full-scrape.md#3-approve-the-quote).
## Invoices [#invoices]
You get one invoice for each top-up, after the payment.
- [`GET /customers/{customer_id}/billing/invoices`](/api/reference/billing/list-topup-invoices.md) lists the invoices, newest first.
- [`GET /customers/{customer_id}/billing/invoices/{invoice_id}/pdf`](/api/reference/billing/download-topup-invoice.md) downloads the PDF.
```json title="GET /customers/{customer_id}/billing/invoices"
{
"items": [
{
"invoice_id": "4821",
"invoice_code": "FV-2026-0001",
"created_at": "2026-10-13T09:30:00.000Z",
"currency": "EUR",
"total": "60.50"
}
]
}
```
The job charges have no invoices. They use credit that you already paid.
## Usage [#usage]
[`GET /customers/{customer_id}/billing/usage`](/api/reference/billing/get-usage.md) lists the charges of one calendar month, newest first. Set the month with `month=YYYY-MM`. Without it, you get the current month. The months use the Europe/Prague time zone.
```json title="Response"
{
"month": "2026-10",
"currency": "EUR",
"total": "257.57",
"items": [
{
"job_id": "jobreq_01k7d0f1g2h3j4k5m6n7p8q9r0",
"job_type": "FULL_SCRAPE",
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"website_id": "example-shop.cz",
"quantity": 17950,
"amount": "12.57",
"capped": false,
"settled_at": "2026-10-13T15:40:00.000Z"
}
]
}
```
A failed paid job shows with the amount `0.00`, except a paid analysis that failed with `CURRENCY_NOT_AVAILABLE`. A free matching sample does not show.
## Monthly spend limit [#monthly-spend-limit]
Each customer has a monthly spend limit. The default is 5,000 EUR. The API adds the charges of the month, the current holds and the `cap` of the new job. If the sum is more than the limit, the approval answers `403 MONTHLY_LIMIT_REACHED`:
```json title="403 MONTHLY_LIMIT_REACHED"
{
"code": "MONTHLY_LIMIT_REACHED",
"detail": "Monthly spend limit is reached",
"currency": "EUR",
"monthly_limit": "5000.00"
}
```
A top-up does not help. Wait until running jobs settle and release their holds, or until the next month starts. You can also write to [support@deepscout.ai](mailto:support@deepscout.ai) to change the limit.
If DeepScout manages your account, the billing endpoints answer `403 MANAGED_CUSTOMER`. DeepScout sends you invoices separately.
# Errors (/api/errors)
## Error format [#error-format]
Every error has a JSON body with a `code` and a `detail`:
```json
{ "code": "INVALID_API_KEY", "detail": "Invalid API key" }
```
- `code` is a stable value in upper case. Use it in your code to select an action.
- `detail` is a text for people. It can change. Do not parse it.
Some errors add more fields:
| Code | More fields |
| --- | --- |
| `INSUFFICIENT_FUNDS` | `currency`, `missing_topup`: the top-up that permits the approval. |
| `MONTHLY_LIMIT_REACHED` | `currency`, `monthly_limit`: the monthly spend limit of the customer. |
| `ESTIMATE_CHANGED` | `estimate`: the new quote of the job. |
### Validation errors [#validation-errors]
If the request has a wrong field type, or a required field is missing, the API answers `422` with the code `VALIDATION_ERROR`. Then `detail` is a list, with one item for each error:
```json
{
"code": "VALIDATION_ERROR",
"detail": [
{
"type": "missing",
"loc": ["query", "project_id"],
"msg": "Field required",
"input": null
}
]
}
```
`loc` shows where the error is: the part of the request (`body`, `query` or `path`), then the field.
If the body is not valid JSON, the error has the type `json_invalid`. If the API cannot read the body at all, for example because it is not UTF-8, the API answers `400` with the code `INVALID_BODY`.
## HTTP status codes [#http-status-codes]
| Status | Meaning |
| --- | --- |
| `200 OK` | The request is complete. A create request also answers `200` when it returns an existing job instead of a new one. |
| `201 Created` | The API created a new resource. |
| `204 No Content` | The API removed the resource. There is no body. |
| `302 Found` | A redirect to a download URL. |
| `400 Bad Request` | The values in the request are not correct. |
| `401 Unauthorized` | The API key is missing, unknown or revoked. |
| `402 Payment Required` | The credit is too low. |
| `403 Forbidden` | Your role does not permit the action, or a limit stops it. |
| `404 Not Found` | The resource does not exist, or you cannot access it. |
| `409 Conflict` | The current state of a job or a website does not permit the action. |
| `422 Unprocessable Content` | The request does not agree with the schema. |
| `502 Bad Gateway` | The payment or the invoicing provider has a problem. Try again later. |
| `503 Service Unavailable` | Billing or file downloads are temporarily not available. Try again later. |
## Error codes [#error-codes]
### Authentication and access [#authentication-and-access]
| Status | Code | Cause | What to do |
| --- | --- | --- | --- |
| `401` | `NOT_AUTHENTICATED` | The request has no API key. | Send the key in the `Authorization` header. |
| `401` | `INVALID_API_KEY` | The key is unknown or revoked. | Use a valid key. |
| `401` | `USER_NOT_FOUND` | The user of the key does not exist any more. | Create a new key. |
| `403` | `INSUFFICIENT_SCOPE` | A key for one project (`PROJECT` scope) is used on an account, customer or billing endpoint. | Use a key with the `USER` scope. See [Key scopes](/api/authentication.md#key-scopes). |
| `403` | `INSUFFICIENT_ROLE` | Your role in the customer or in the project does not permit the action. | Ask an administrator for a role. See [Roles](/api/authentication.md#roles). |
| `403` | `PROJECT_ACCESS_DENIED` | You have no role in the project. | Ask an administrator of the project. |
| `403` | `PROJECT_DISABLED` | The project of a `PROJECT` key is stopped. | Contact DeepScout. |
| `403` | `MANAGED_CUSTOMER` | DeepScout manages this customer, so self-service actions are not available. | Contact your account manager. |
| `403` | `EXPORTS_NOT_ENABLED` | The project has no exports. | Contact DeepScout. |
### Not found [#not-found]
| Status | Code | Cause |
| --- | --- | --- |
| `404` | `PROJECT_NOT_FOUND` | The project does not exist, or you cannot access it. |
| `404` | `WEBSITE_NOT_FOUND` | The website is not in the project. For a matching: the website has no completed scrape in the project. |
| `404` | `JOB_NOT_FOUND` | The job does not exist, or it is in a project that you cannot access. |
| `404` | `CUSTOMER_NOT_FOUND` | The customer does not exist. For most customer endpoints, an unknown customer gets `403 INSUFFICIENT_ROLE` instead. |
| `404` | `API_KEY_NOT_FOUND` | The key does not exist, or it is not your key. |
| `404` | `INVOICE_NOT_FOUND` | The invoice does not exist. |
| `404` | `TOPUP_NOT_FOUND` | The top-up session does not exist. |
| `404` | `DATASET_NOT_FOUND` | The project has no such dataset, or the dataset has no files for one website. |
| `404` | `EXPORT_FILE_NOT_FOUND` | The export file is not written yet. |
| `404` | `SAMPLE_NOT_FOUND` | The sample of the website is not ready yet. |
| `404` | `URL_RESULTS_NOT_FOUND` | The URL scrape has no URL results yet. They are available after the settlement of the job. See [Scrape a URL list](/api/url-scrape.md#when-the-results-are-available). |
| `404` | `NOT_FOUND` | The URL does not exist. |
### Request values [#request-values]
| Status | Code | Cause |
| --- | --- | --- |
| `400` | `INVALID_WEBSITE_URL` | The website URL is not valid. |
| `400` | `INVALID_LOCALE` | The country is not a two-letter country code, or the currency is not an ISO 4217 code. |
| `400` | `INVALID_URL_LIST` | The API accepted no URL of the list, or the list is too large. Split a large list into more jobs. |
| `400` | `INVALID_MATCHING_SCOPE` | The matching websites are not correct, for example the reference website is also a competitor. |
| `400` | `INVALID_TOPUP_AMOUNT` | The top-up amount is not from 5 to 5,000 EUR. |
| `400` | `PROJECT_ID_REQUIRED` | The export download needs `project_id` with a `USER` key. |
| `400` | `PROJECT_LIMIT_REACHED` | The customer has 50 projects. |
| `400` | `JOB_NOT_APPROVABLE` | The job cannot be approved, for example it has no quote. |
| `422` | `VALIDATION_ERROR` | The request does not agree with the schema. |
| `400` | `INVALID_BODY` | The API cannot read the body, for example because it is not UTF-8. Send JSON in UTF-8. |
| `405` | `METHOD_NOT_ALLOWED` | The endpoint does not accept this HTTP method. |
### Credit and limits [#credit-and-limits]
| Status | Code | Cause | What to do |
| --- | --- | --- | --- |
| `402` | `INSUFFICIENT_FUNDS` | The spendable credit is less than the `cap` of the job, or less than the price of a paid [website analysis](/api/add-website.md#before-you-start). | Top up at least `missing_topup` (and at least 5 EUR), then send the request again. |
| `403` | `MONTHLY_LIMIT_REACHED` | The job does not fit in the monthly spend limit. | Wait until running jobs settle or the next month starts, or contact DeepScout. A top-up does not help. |
### Job and website state [#job-and-website-state]
| Status | Code | Cause | What to do |
| --- | --- | --- | --- |
| `409` | `ESTIMATE_CHANGED` | The price went up since the quote. | Read the new `estimate`. If you accept it, approve again. |
| `409` | `CATALOG_EXPIRED` | The full catalog of the website is older than 31 days. | Run a full scrape, or create the URL scrape with `urls_only`. See [Scrape a URL list](/api/url-scrape.md#when-the-catalog-is-too-old). |
| `409` | `CATALOG_DELIVERED` | Your project already got the newest full scrape of the website, and that scrape ended less than 18 hours ago. | Use the current data. |
| `409` | `JOB_NOT_AWAITING_APPROVAL` | The job is not a quote any more. For a full scrape: another full or `URLS_ONLY` scrape of the website runs in the project. | Read the job. Wait for the other scrape to end. |
| `409` | `JOB_NOT_CANCELLABLE` | The job is approved, settling or finished, or it is a free matching sample. | None. An approved job runs to its end. |
| `409` | `JOB_RUNNING` | The setup job runs, so you cannot remove it. | Wait for it to end, or cancel it. |
| `409` | `SETUP_JOB_HAS_WEBSITE` | The setup job found its website. | Remove the website instead. |
| `409` | `WEBSITE_BUSY` | A job of the website runs, so you cannot remove the website. | Wait for the job to end. |
| `409` | `MATCHING_BUSY` | Another matching of the project is not finished. This includes a quote that waits for approval. | Wait, or cancel the quote. |
| `409` | `MATCHING_SAMPLE_USED` | The project already used its free matching sample. | Run a paid matching. |
| `409` | `CONTACT_REASON_MISMATCH` | The job did not fail, the job has no `contact_reason`, or the reason is not its `contact_reason`. | Send the `contact_reason` of the failed job. If the job has none, read its `error`. You can fix the cause yourself. |
### Server and providers [#server-and-providers]
| Status | Code | Cause |
| --- | --- | --- |
| `502` | `STRIPE_ERROR` | The payment provider has a problem. |
| `502` | `ACCOUNTING_ERROR` | The invoicing system has a problem. |
| `503` | `BILLING_UNAVAILABLE` | Billing is temporarily not available. |
| `503` | `MAINTENANCE` | File downloads are temporarily not available. |
| `500` | `INTERNAL_ERROR` | An unexpected error. |
If you get a `5xx` status, try again after some time. Increase the wait after each try.
### Other errors [#other-errors]
| Status | Code | Cause |
| --- | --- | --- |
| Any | `HTTP_ERROR` | An error that has no code of its own. Read the status and the `detail`. |
The `ErrorCode` list in the OpenAPI specification also has codes of the console, for example `SESSION_EXPIRED` and `CAPTCHA_FAILED`. The API endpoints in these docs do not return them.
The error codes of failed jobs are in the `error_code` field of the job, not in an error response. See [Failed jobs](/api/jobs.md#failed-jobs).
# End-to-end example (/api/example)
This script does the full flow of the [guides](/api/add-website.md) in one run:
1. It adds a website to your default project and waits for the setup.
2. It downloads the sample products.
3. It creates a full scrape and asks you to approve the quote. If the price goes up at the approval, it asks again.
4. It waits for the scrape and downloads the products of the website.
## Run it [#run-it]
You need Python 3.10 or later and the `requests` package.
```bash
pip install requests
export SCOUT_API_KEY="your-api-key"
python scout_example.py https://www.example-shop.cz CZ CZK
```
The script approves a paid scrape after you type `y`. Make sure that you have enough credit, and read the estimate before you approve it.
```txt title="Output"
1. Add the website
ANALYZING
TRAINING COLLECTING_PAGES
...
COMPLETED
2. Look at the sample
1000 sample products, for example: Bio kokosový olej 500 ml
3. Scrape the full catalog
Approve EUR 12.88, at most EUR 19.32? [y/N] y
RUNNING 38
...
COMPLETED
Charged EUR 12.57 for 17950 products
4. Download the products
products.jsonl.gz has 17950 products
```
## The script [#the-script]
```python title="scout_example.py" lineNumbers
"""Add a website, scrape its full catalog and download its products."""
import argparse
import gzip
import json
import os
import shutil
import sys
import time
import requests
BASE = "https://api.deepscout.ai/v1"
END = {"COMPLETED", "FAILED", "CANCELLED"}
api = requests.Session()
api.headers["Authorization"] = f"Bearer {os.environ['SCOUT_API_KEY']}"
class ApiError(Exception):
def __init__(self, status, body):
super().__init__(f"{status} {body.get('code')}: {body.get('detail')}")
self.code = body.get("code")
self.body = body
def call(method, path, **kwargs):
response = api.request(method, BASE + path, **kwargs)
if response.status_code >= 400:
raise ApiError(response.status_code, response.json())
return response.json() if response.content else None
def wait(path, seconds, progress):
"""Read the job until it is in an end status or waits for approval."""
while True:
job = call("GET", path)
print(f" {job['status']} {progress(job)}")
if job["status"] in END or job["status"] == "AWAITING_APPROVAL":
return job
time.sleep(seconds)
def confirm(estimate):
answer = input(f"Approve EUR {estimate['total']}, at most EUR {estimate['cap']}? [y/N] ")
if answer.strip().lower() != "y":
sys.exit("Not approved.")
def approve(job):
"""Approve the quote. Ask again if the price went up."""
confirm(job["estimate"])
while True:
try:
return call("POST", f"/jobs/scraping/{job['job_id']}/approve")
except ApiError as error:
if error.code == "ESTIMATE_CHANGED":
print("The price went up.")
confirm(error.body["estimate"])
elif error.code == "INSUFFICIENT_FUNDS":
sys.exit(f"Top up at least EUR {error.body['missing_topup']}, then run again.")
else:
raise
def main():
parser = argparse.ArgumentParser()
parser.add_argument("url")
parser.add_argument("country")
parser.add_argument("currency")
args = parser.parse_args()
project_id = call("GET", "/projects")[0]["project_id"]
print("1. Add the website")
setup = call("POST", "/jobs/setup", json={
"project_id": project_id,
"url": args.url,
"country": args.country,
"currency": args.currency,
})
setup = wait(
f"/jobs/setup/{setup['job_id']}", 60,
lambda job: job.get("train_progress", {}).get("phase", ""),
)
if setup["status"] != "COMPLETED":
sys.exit(f"The setup ended as {setup['status']}: {setup.get('error')}")
website_id = setup["website_id"]
print("2. Look at the sample")
try:
sample_url = call("GET", f"/samples/{website_id}", params={"project_id": project_id})["url"]
sample = [json.loads(line) for line in requests.get(sample_url).text.splitlines() if line]
print(f" {len(sample)} sample products, for example: {sample[0]['name']}")
except ApiError as error:
if error.code != "SAMPLE_NOT_FOUND":
raise
print(" The sample is not ready yet.")
print("3. Scrape the full catalog")
job = call("POST", "/jobs/scraping", json={"project_id": project_id, "website_id": website_id})
if job["status"] == "AWAITING_APPROVAL":
approve(job)
job = wait(
f"/jobs/scraping/{job['job_id']}", 120,
lambda job: job.get("scrape_progress", {}).get("percent", ""),
)
if job["status"] != "COMPLETED":
sys.exit(f"The scrape ended as {job['status']}: {job.get('error')}")
print(f" Charged EUR {job['settlement']['amount']} for {job['settlement']['quantity']} products")
print("4. Download the products")
params = {"project_id": project_id, "website_id": website_id}
with api.get(f"{BASE}/exports/all_products_with_details", params=params, stream=True) as response:
response.raise_for_status()
with open("products.jsonl.gz", "wb") as file:
shutil.copyfileobj(response.raw, file)
with gzip.open("products.jsonl.gz", "rt", encoding="utf-8") as file:
count = sum(1 for _ in file)
print(f" products.jsonl.gz has {count} products")
if __name__ == "__main__":
main()
```
## How it works [#how-it-works]
- `call()` sends a request and raises `ApiError` for an error status. The error has the `code` of the API. See [Errors](/api/errors.md).
- `wait()` reads a job until it is in an end status or waits for approval. The setup job is slow, so the script reads it every minute. See [Polling](/api/jobs.md#polling).
- `approve()` handles `ESTIMATE_CHANGED`: it shows the new estimate and approves again. For `INSUFFICIENT_FUNDS`, it stops and shows the top-up that you need. See [Credits and billing](/api/credits.md).
- The download follows the redirect of `GET /exports/{dataset}` to the storage URL. The `requests` package does not send the `Authorization` header to the other host.
## Next steps [#next-steps]
- To refresh some prices often, [scrape a URL list](/api/url-scrape.md).
- To compare your products with a competitor, [match products](/api/matching.md).
# Download exports (/api/exports)
The export files contain all the data of a project. When a scrape or a matching completes, the API writes the files again with the new data. The job stays in the `EXPORTING` status until the files are ready, and then changes to `COMPLETED`.
## Datasets [#datasets]
A self-service project has these datasets:
| Dataset | Content | Files |
| --- | --- | --- |
| [`all_products_with_details`](/datasets/products.md) | The products of one website, with all fields. | One file for each website. There is no file for the whole project. |
| [`matches`](/datasets/matches.md) | The product pairs from the matching jobs. | One file for the project. |
The datasets `all_products_with_products` and `prices_and_availability_by_website` are only for managed customers. To compare your prices with your competitors, join `matches` with the product files of the websites. Use `website_id` and `product_id` on both sides of a pair.
The files are JSON Lines: one JSON object for each row, one row on each line. They are compressed with gzip (`.jsonl.gz`).
## List the files [#list-the-files]
Send [`GET /exports`](/api/reference/exports/list-exports.md) with the `project_id`:
```bash tab="cURL" tab-group="lang"
curl "https://api.deepscout.ai/v1/exports?project_id=$PROJECT_ID" \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```python tab="Python"
exports = api.get(f"{BASE}/exports", params={"project_id": project_id}).json()
for file in exports["exports"]:
if file["exists"]:
print(file["dataset"], file.get("website_id"), file["size_bytes"], file["download_url"])
```
```json title="Response"
{
"enabled": true,
"generating": false,
"exports": [
{
"dataset": "all_products_with_details",
"website_id": "example-shop.cz",
"file_name": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz_all_products_with_details_example-shop.cz.jsonl.gz",
"download_url": "https://api.deepscout.ai/v1/exports/all_products_with_details?project_id=proj_01k7c2r9v8e3s0b1x6m4q5n2tz&website_id=example-shop.cz",
"exists": true,
"size_bytes": 11500000,
"updated_at": "2026-10-13T15:45:00.000Z"
},
{
"dataset": "matches",
"file_name": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz_matches.jsonl.gz",
"download_url": "https://api.deepscout.ai/v1/exports/matches?project_id=proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"exists": false
}
]
}
```
| Field | Description |
| --- | --- |
| `enabled` | `false` if the project has no exports. Then the list is empty. |
| `generating` | `true` while the API writes new files. Until it is done, the listed files are from the previous export. |
| `exists` | `false` if the file is not written yet, for example the `matches` file before the first matching. |
| `website_id` | The website of the file. The `matches` file is for the whole project, so it has no `website_id`. |
| `download_url` | The URL to download the file with your API key. |
## Download a file [#download-a-file]
Send [`GET /exports/{dataset}`](/api/reference/exports/get-export.md), or use the `download_url` from the list. The API answers with a redirect (`302`) to a temporary storage URL. Let your HTTP client follow the redirect. The `all_products_with_details` files also need `website_id`.
```bash tab="cURL" tab-group="lang"
curl -L "https://api.deepscout.ai/v1/exports/all_products_with_details?project_id=$PROJECT_ID&website_id=$WEBSITE_ID" \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-o products.jsonl.gz
```
```python tab="Python"
import shutil
with api.get(
f"{BASE}/exports/all_products_with_details",
params={"project_id": project_id, "website_id": website_id},
stream=True,
) as response:
response.raise_for_status()
with open("products.jsonl.gz", "wb") as file:
shutil.copyfileobj(response.raw, file)
```
```js tab="Node.js"
import { writeFile } from 'node:fs/promises';
const url = `https://api.deepscout.ai/v1/exports/all_products_with_details?project_id=${process.env.PROJECT_ID}&website_id=${process.env.WEBSITE_ID}`;
const response = await fetch(url, {
headers: { Authorization: `Bearer ${process.env.SCOUT_API_KEY}` },
});
await writeFile('products.jsonl.gz', Buffer.from(await response.arrayBuffer()));
```
The temporary URL is valid for 1 hour. If you only need the URL, for example to give it to another system, send [`GET /exports/{dataset}/url`](/api/reference/exports/get-export-url.md). It returns `{"url": "..."}` instead of a redirect.
With an API key for one project (`PROJECT` scope), this endpoint always uses the project of the key. You can leave out `project_id`. Such a key is good for a data warehouse or a scheduled import. See [Key scopes](/api/authentication.md#key-scopes).
## Read a file [#read-a-file]
Each line of the file is one JSON object.
```bash tab="Shell" tab-group="read"
gunzip -c products.jsonl.gz | head -n 1 | jq .
```
```python tab="Python"
import gzip
import json
with gzip.open("products.jsonl.gz", "rt", encoding="utf-8") as file:
for line in file:
product = json.loads(line)
print(product["website_id"], product["name"], product["price"])
```
The rows are not in random order. Thus the first rows of a file are not a sample of the data.
The [Datasets](/datasets.md) pages describe the fields of each dataset.
## Know when new files are ready [#know-when-new-files-are-ready]
- A scrape or a matching job is `COMPLETED` only after the export files have its data. Wait for `COMPLETED`, then download.
- `generating` in [`GET /exports`](/api/reference/exports/list-exports.md) is `true` while an export runs. When it changes to `false`, the files are new.
- `updated_at` of each file shows when the API wrote it.
The API writes the files only after a job. There is no scheduled export.
## Errors [#errors]
| Status | Code | Cause |
| --- | --- | --- |
| `400` | `PROJECT_ID_REQUIRED` | A `USER` key needs the `project_id` parameter. |
| `403` | `EXPORTS_NOT_ENABLED` | The project has no exports. |
| `404` | `DATASET_NOT_FOUND` | The project has no such dataset, or the dataset has no files for one website. |
| `404` | `EXPORT_FILE_NOT_FOUND` | The API did not write the file yet. Wait for the first completed job. |
# Scrape the full catalog (/api/full-scrape)
A full scrape collects all products of a website. Its results go to the export files of your project. A full scrape also makes a fresh **catalog**: for 31 days, [URL scrapes](/api/url-scrape.md) of the website update the prices in this catalog.
## 1. Read the estimate [#1-read-the-estimate]
When the analysis of the setup job is complete (`ESTIMATE_READY`), the website has a `full_scrape_estimate`. The API calculates it at each request, so it always shows the current price. Approve a full scrape after the setup is `COMPLETED` and you checked the [sample data](/api/sample-data.md). Read it with [`GET /websites/{website_id}`](/api/reference/websites/get-website.md):
```bash tab="cURL" tab-group="lang"
curl "https://api.deepscout.ai/v1/websites/example-shop.cz?project_id=$PROJECT_ID" \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```python tab="Python"
website = api.get(f"{BASE}/websites/{website_id}", params={"project_id": project_id}).json()
print(website["full_scrape_estimate"])
```
```json title="full_scrape_estimate"
{
"currency": "EUR",
"unit_price_per_1000": "0.70",
"quantity": 18400,
"total": "12.88",
"cap": "19.32",
"min_charge": "0.10",
"difficulty": "MEDIUM"
}
```
- `quantity` is the latest product count of the website. DeepScout counts the products at each full scrape. Before the first full scrape, the count is the estimate of the analysis.
- `difficulty` sets the price for 1,000 products. It depends on the protections of the website. See [Pricing](/api/pricing.md).
- `cap` is the maximum charge: 1.5 times the `total`.
## 2. Create the scrape job [#2-create-the-scrape-job]
Send [`POST /jobs/scraping`](/api/reference/scraping-jobs/create-scraping-job.md) with the website and the project, and without `urls`. The API creates a `FULL_SCRAPE` job with the same quote as `full_scrape_estimate`. The job waits in the `AWAITING_APPROVAL` status. Nothing is charged yet.
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/scraping \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz", "website_id": "example-shop.cz"}'
```
```python tab="Python"
job = api.post(f"{BASE}/jobs/scraping", json={
"project_id": project_id,
"website_id": website_id,
}).json()
print(job["job_id"], job["estimate"]["total"], job["estimate"]["cap"])
```
```json title="Response (201 Created)"
{
"job_id": "jobreq_01k7d0f1g2h3j4k5m6n7p8q9r0",
"job_type": "FULL_SCRAPE",
"status": "AWAITING_APPROVAL",
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"website_id": "example-shop.cz",
"variant": "FULL",
"estimate": {
"currency": "EUR",
"unit_price_per_1000": "0.70",
"quantity": 18400,
"total": "12.88",
"cap": "19.32",
"min_charge": "0.10",
"difficulty": "MEDIUM"
},
"rejected": { "invalid": 0, "other_website": 0, "duplicate": 0 },
"rejected_urls": [],
"created_at": "2026-10-13T12:00:00.000Z",
"updated_at": "2026-10-13T12:00:00.000Z"
}
```
A website has only one unfinished full scrape in a project. If one exists, the API returns it with `200 OK` and does not create a new job.
A quote that you do not approve for 7 days changes to `CANCELLED`. To remove a quote before that, send [`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md).
## 3. Approve the quote [#3-approve-the-quote]
Send [`POST /jobs/scraping/{job_id}/approve`](/api/reference/scraping-jobs/approve-scraping-job.md). The approval holds the `cap` from your credit and starts the scrape.
```bash tab="cURL" tab-group="lang"
curl -X POST https://api.deepscout.ai/v1/jobs/scraping/jobreq_01k7d0f1g2h3j4k5m6n7p8q9r0/approve \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```python tab="Python"
response = api.post(f"{BASE}/jobs/scraping/{job['job_id']}/approve")
if response.ok:
job = response.json() # status RUNNING
else:
error = response.json()
print(response.status_code, error["code"], error["detail"])
```
The approval calculates the price again with the newest data of the website. These answers need an action from you:
| Status | Code | What to do |
| --- | --- | --- |
| `409` | `ESTIMATE_CHANGED` | The price went up. The response and the job have the new `estimate`. If you accept it, approve again. |
| `402` | `INSUFFICIENT_FUNDS` | Your spendable credit is less than the `cap`. Top up at least `missing_topup` (and at least 5 EUR), then approve again. See [Credits and billing](/api/credits.md). |
| `403` | `MONTHLY_LIMIT_REACHED` | The job does not fit in the monthly spend limit of your customer. A top-up does not help. |
| `409` | `CATALOG_DELIVERED` | Your project already got the newest full scrape of the website, and that scrape ended less than 18 hours ago. |
| `409` | `JOB_NOT_AWAITING_APPROVAL` | The job is not a quote any more, or another full or `URLS_ONLY` scrape of the website runs in your project. Wait until it ends. |
If the new price is lower, the approval accepts it and you pay the lower price.
```json title="409 ESTIMATE_CHANGED"
{
"code": "ESTIMATE_CHANGED",
"detail": "the price of the job went up; approve its new quote",
"estimate": {
"currency": "EUR",
"unit_price_per_1000": "0.70",
"quantity": 21000,
"total": "14.70",
"cap": "22.05",
"min_charge": "0.10",
"difficulty": "MEDIUM"
}
}
```
## 4. Follow the progress [#4-follow-the-progress]
Read the job with [`GET /jobs/scraping/{job_id}`](/api/reference/scraping-jobs/get-scraping-job.md). The job goes through these statuses:
| Status | What happens |
| --- | --- |
| `APPROVED` | The approval is saved and the API starts the scrape. The approve response usually shows `RUNNING` already. |
| `RUNNING` | The scrape runs. `scrape_progress` shows the phase. |
| `SETTLING` | The scrape is complete. The API calculates the charge. |
| `EXPORTING` | The API writes the export files of the project. |
| `COMPLETED` | The data is in the export files. `settlement` shows the charge. |
While the status is `RUNNING`, `scrape_progress.phase` shows the step:
| `phase` | Console label | Description |
| --- | --- | --- |
| `COLLECTING_PAGES` | Downloading pages | The scraper downloads the product pages. `pages` counts them, and `percent` compares them with `pages_estimate`. |
| `PROCESSING_DATA` | Data cleaning and pre-processing | The API extracts and cleans the product data. |
| `UPDATING_CATALOG` | Updating the product catalog | The API merges the new data into the catalog of the website. |
```json title="A running full scrape"
{
"job_id": "jobreq_01k7d0f1g2h3j4k5m6n7p8q9r0",
"job_type": "FULL_SCRAPE",
"status": "RUNNING",
"variant": "FULL",
"scrape_progress": {
"phase": "COLLECTING_PAGES",
"pages": 6900,
"pages_estimate": 18400,
"percent": 38
},
"approval": {
"approved_by": "user_01k7c2r9q1w2e3r4t5y6z7x8c9",
"approved_at": "2026-10-13T12:01:00.000Z"
}
}
```
The time of a full scrape depends on the size of the catalog and on the protections of the website. A large catalog takes many hours. Read the job every 1 to 5 minutes. You also get an email when the job completes or fails.
If a full scrape of the same website runs now, or ended less than 18 hours ago, your job can use its data. This makes your job faster. The price rules do not change.
You cannot cancel a scrape after the approval.
## 5. Read the charge [#5-read-the-charge]
When the job is `COMPLETED`, `settlement` shows the charge:
```json title="settlement"
{
"quantity": 17950,
"currency": "EUR",
"amount": "12.57",
"capped": false,
"settled_at": "2026-10-13T15:40:00.000Z"
}
```
- `quantity` is the number of products that the scrape actually collected.
- `amount` is `quantity` × `unit_price_per_1000` / 1,000, at least the `min_charge`.
- `capped` is `true` if the `cap` limited the charge.
The API releases the rest of the hold at the same time. If the scrape collects no products, the job is `FAILED` with the code `SCRAPE_FAILED`, and the charge is zero.
## 6. Get the data [#6-get-the-data]
When the job is `COMPLETED`, the export files have the new data. The dataset `all_products_with_details` has one file for each website. See [Download exports](/api/exports.md).
## Scrape the catalog again [#scrape-the-catalog-again]
To refresh the catalog, create and approve a new full scrape. Your project can already have the newest full scrape of the website. If that scrape ended less than 18 hours ago, the create request and the approval answer `409 CATALOG_DELIVERED`.
A full scrape also keeps your catalog fresh for URL scrapes. URL scrapes update the catalog only for 31 days after the start of the last full scrape. See [Scrape a URL list](/api/url-scrape.md).
# Self-service API (/api)
The DeepScout API gives you the same functions as the [DeepScout Console](https://console.deepscout.ai). You add a website, look at free sample data, approve a scrape, match products across websites and download the results as files.
```txt title="Base URL"
https://api.deepscout.ai/v1
```
## How it works [#how-it-works]
The API follows the same flow as the console. Each step is a **job** that runs in the background. You start a job with a `POST` request and read its status with a `GET` request.
### Add a website [#add-a-website]
A setup job analyzes the website and learns how to read its product pages. The setup is free and gives you sample products. See [Add a website](/api/add-website.md).
### Look at the sample data [#look-at-the-sample-data]
Before you pay, download up to 1,000 sample products and make sure that the data is correct. See [Check the sample data](/api/sample-data.md).
### Scrape the full catalog [#scrape-the-full-catalog]
You get a price estimate for the full catalog. When you approve it, the scrape starts and the cost comes from your prepaid credit. See [Scrape the full catalog](/api/full-scrape.md).
### Update prices with URL lists [#update-prices-with-url-lists]
Send a list of product URLs to update only these products. You can also scrape a URL list without the full catalog. See [Scrape a URL list](/api/url-scrape.md).
### Match products across websites [#match-products-across-websites]
A matching job finds the same products on different websites. Start with a free sample. See [Match products](/api/matching.md).
### Download the exports [#download-the-exports]
Each completed scrape or matching updates the export files of the project. Download them as gzipped JSON Lines. See [Download exports](/api/exports.md).
## Start here [#start-here]
Get an API key and send your first requests.
Customers, projects, websites, jobs and credit.
Connect Claude, Cursor, Codex and other AI agents.
One Python script for the full flow, from a URL to the exports.
All endpoints with their parameters, responses and code samples.
## What you pay for [#what-you-pay-for]
Sample data is free. The first 10 setup jobs of each month are free, and each next one costs €0.50. Setup jobs, scrapes and matchings use prepaid credit in EUR. Each scrape and matching shows a quote before it starts, and the final charge is never more than 1.5 times the estimate. See [Pricing](/api/pricing.md) and [Credits and billing](/api/credits.md).
The machine-readable specification is at [`https://api.deepscout.ai/v1/openapi.json`](https://api.deepscout.ai/v1/openapi.json). Use it to generate a client in your language.
# Job lifecycle (/api/jobs)
## Job types and endpoints [#job-types-and-endpoints]
| `job_type` | Create | Read one | List |
| --- | --- | --- | --- |
| `WEBSITE_SETUP` | [`POST /jobs/setup`](/api/reference/setup-jobs/create-setup-job.md) | [`GET /jobs/setup/{job_id}`](/api/reference/setup-jobs/get-setup-job.md) | [`GET /jobs/setup`](/api/reference/setup-jobs/list-setup-jobs.md) |
| `FULL_SCRAPE`, `URL_SCRAPE` | [`POST /jobs/scraping`](/api/reference/scraping-jobs/create-scraping-job.md) | [`GET /jobs/scraping/{job_id}`](/api/reference/scraping-jobs/get-scraping-job.md) | [`GET /jobs/scraping`](/api/reference/scraping-jobs/list-scraping-jobs.md) |
| `MATCHING` | [`POST /jobs/matching`](/api/reference/matching-jobs/create-matching-job.md) | [`GET /jobs/matching/{job_id}`](/api/reference/matching-jobs/get-matching-job.md) | [`GET /jobs/matching`](/api/reference/matching-jobs/list-matching-jobs.md) |
The lists need `project_id`. The setup and the scraping lists also accept `website_id`. Each list shows the newest jobs first. `limit` sets the number of jobs, from 1 to 100, and the default is 20.
[`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md) cancels a job of any type.
## Statuses [#statuses]
Each job type goes through its own statuses.
### Setup jobs [#setup-jobs]
```txt
CREATED → ANALYZING → ESTIMATE_READY → TRAINING → COMPLETED
```
The analysis and the training move the job forward without action from you. A setup job can skip `ANALYZING` or `TRAINING` when DeepScout already knows the website. See [Add a website](/api/add-website.md#follow-the-job).
### Scrape and matching jobs [#scrape-and-matching-jobs]
```txt
AWAITING_APPROVAL → APPROVED → RUNNING → SETTLING → EXPORTING → COMPLETED
```
| Status | Description |
| --- | --- |
| `AWAITING_APPROVAL` | The job has a quote in `estimate`. It waits for your approval. |
| `APPROVED` | You approved the quote. The API holds the `cap` from your credit and starts the work. The approve response usually shows `RUNNING` already. |
| `RUNNING` | The work runs. `scrape_progress` or `matching_progress` shows the progress. |
| `SETTLING` | The work is complete. The API calculates the charge. |
| `EXPORTING` | The API writes the export files of the project. |
| `COMPLETED` | The data is in the export files. `settlement` shows the charge. |
A free matching sample has a zero estimate. It starts without your approval.
### End statuses [#end-statuses]
`COMPLETED`, `FAILED` and `CANCELLED` are end statuses. A job in an end status does not change again. `finished_at` shows when it got there.
- `FAILED` can end a job of any type. See [Failed jobs](#failed-jobs).
- `CANCELLED` ends a job that you cancelled, a quote that did not change for 7 days, or the waiting jobs of a removed website.
## Common fields [#common-fields]
All job responses have these fields. A field without a value is not in the response.
| Field | Description |
| --- | --- |
| `job_id` | The ID of the job. |
| `job_type` | `WEBSITE_SETUP`, `FULL_SCRAPE`, `URL_SCRAPE` or `MATCHING`. |
| `status` | The current status. |
| `project_id` | The project of the job. |
| `website_id` | The website of a setup or scrape job. A setup job gets it when the analysis is complete. |
| `estimate` | The quote of a paid job. See [Pricing](/api/pricing.md). |
| `approval` | Who approved the quote (`approved_by`) and when (`approved_at`). |
| `settlement` | The charge: `quantity`, `amount`, `currency`, `capped` and `settled_at`. A failed job settles at zero, except a paid analysis that fails with `CURRENCY_NOT_AVAILABLE`. |
| `error`, `error_code` | Why the job failed. |
| `contact_reason` | On a failed job: the reason to send in a contact request. A failure that you can fix yourself has none. |
| `contact_request_id` | The contact request that you sent about the job. |
| `created_at`, `updated_at`, `finished_at` | When the job was created, changed last, and ended. |
## Website state [#website-state]
[`GET /websites`](/api/reference/websites/list-websites.md) shows each website with one `job`: the newer of its setup job and its newest full scrape that is not cancelled. URL scrapes do not change it. The console shows the state of the website from this job:
| `job.status` | Console status |
| --- | --- |
| `CREATED`, `ANALYZING` | Analyzing |
| `ESTIMATE_READY`, `TRAINING` | Training |
| `COMPLETED` of the setup job, or `AWAITING_APPROVAL` | Ready |
| `APPROVED`, `RUNNING`, `SETTLING` | Scraping |
| `EXPORTING` | Exporting |
| `COMPLETED` of a full scrape | Completed |
| `FAILED` with `contact_reason` `WEBSITE_TOO_LARGE` | Custom setup |
| `FAILED` | Failed |
| `CANCELLED` | Cancelled |
The website also has `scraped_at`, the end of its last completed scrape, and `added_at`, the start of its setup.
## Polling [#polling]
The API has no webhooks yet. To follow a job, read it again at an interval:
| Job | Interval |
| --- | --- |
| Setup job | 30 to 60 seconds |
| Scrape or matching | 1 to 5 minutes |
| Top-up status | A few seconds, for up to 2 minutes |
Stop when the job is in an end status, or in `AWAITING_APPROVAL`, which waits for you.
To follow all websites of a project with one request, read [`GET /websites`](/api/reference/websites/list-websites.md). For the scrape jobs of one website, read [`GET /jobs/scraping`](/api/reference/scraping-jobs/list-scraping-jobs.md) with `website_id`.
## Cancel a job [#cancel-a-job]
Send [`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md). You can cancel:
- A setup job before it is complete. You do not get the analysis back.
- A scrape or a matching that waits for approval (`AWAITING_APPROVAL`). This removes the quote.
You cannot cancel a paid job after its approval, or a free matching sample. The API then answers `409 JOB_NOT_CANCELLABLE`.
A quote that you do not approve for 7 days changes to `CANCELLED`.
## Failed jobs [#failed-jobs]
A failed job has an `error` text and an `error_code`. Most failed jobs also have a `contact_reason`.
| `error_code` | Job type | `contact_reason` |
| --- | --- | --- |
| `WEBSITE_TOO_LARGE` | Setup | `WEBSITE_TOO_LARGE` (sales team) |
| `CURRENCY_NOT_AVAILABLE` | Setup | None. Start a new setup job with another currency. See [The website does not offer the currency](/api/add-website.md#the-website-does-not-offer-the-currency). |
| `ANALYSIS_FAILED` | Setup | `JOB_FAILED` (support team) |
| `TRAINING_FAILED` | Setup | `JOB_FAILED` (support team) |
| `SCRAPE_FAILED` | Full or URL scrape | `JOB_FAILED` (support team) |
| `MATCHING_FAILED` | Matching | `JOB_FAILED` (support team) |
A failed paid job costs nothing. The exception is a paid analysis that fails with `CURRENCY_NOT_AVAILABLE`. To ask for help, send [`POST /contact-requests`](/api/reference/contact/create-contact-request.md):
```json
{
"reason": "JOB_FAILED",
"job_id": "jobreq_01k7d0f1g2h3j4k5m6n7p8q9r0",
"phone": "+420 123 456 789",
"message": "The scrape failed two times."
}
```
- `reason` must be the `contact_reason` of the job. If not, or if the job has no `contact_reason`, the API answers `409 CONTACT_REASON_MISMATCH`.
- `phone` is optional, with 6 to 20 digits. Without it, the team writes to the email of your account.
- `message` is optional, up to 2,000 characters.
The API answers `201 Created`. A second request about the same job returns the first request with `200 OK`. After the request, the job shows its `contact_request_id`.
## Email notifications [#email-notifications]
DeepScout sends an email to the user who started the job:
| Event | Email |
| --- | --- |
| The training of a website is complete | The website is ready |
| A setup job fails | The setup failed |
| The website does not offer the currency of a setup job | Run the analysis again with another currency |
| A website has more than 2,000,000 products | An offer from the sales team |
| A scrape or a matching is `COMPLETED` | The data is ready |
| A scrape or a matching fails | The job failed |
There are no emails for quotes, cancellations or top-ups.
# Localization (/api/localization)
Each project has a localization: a country and a currency. DeepScout converts the prices of all websites in the project to this currency. Thus you can compare the prices of stores that sell in different currencies.
For example, the currency of your project is `CZK`. A German competitor sells a product for 19.99 EUR. The export of the competitor shows this price in CZK.
At this time, only the currency changes the data. The country and the locale do not change the exports.
In the console, the localization is the **Market** on **Settings → Project**.
## Fields [#fields]
| Field | Description |
| --- | --- |
| `country` | The country of the market, as a two-letter country code (ISO 3166-1 alpha-2). The API returns it in lower case, for example `cz`. |
| `currency` | The currency of the prices in the exports, as an ISO 4217 code. The API returns it in upper case, for example `CZK`. |
| `locale` | The most probable language of the country, for example `cs_CZ` or `de_AT`. The API sets it from the country. You cannot set it. |
[`GET /projects`](/api/reference/projects/list-projects.md) shows the localization of each project. A project without a localization has no `localization` field.
```json title="Response"
[
{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"name": "Default project",
"customer_id": "cust_01k7c2r9t2f8a5d3h7j9k1m6pw",
"is_default": true,
"localization": { "locale": "cs_CZ", "country": "cz", "currency": "CZK" }
}
]
```
## How a project gets its localization [#how-a-project-gets-its-localization]
- **New project.** [`POST /customers/{customer_id}/projects`](/api/reference/customers/create-project.md) needs the `localization`. In the console, the **New project** dialog asks for the country and the currency.
- **Default project.** Your first sign-in creates the default project without a localization. The first website that you add to this project sets it. The project gets the country and the currency of the setup job. If the setup job fails, the project stays without a localization.
```bash
curl https://api.deepscout.ai/v1/customers/$CUSTOMER_ID/projects \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "Germany", "localization": {"country": "DE", "currency": "EUR"}}'
```
If the first website of the default project is a foreign competitor, the project gets the currency of this competitor. If this is not your currency, change the localization.
A project without a localization does not convert prices. Each website keeps the currency of its store. The `currency` field of each product shows it.
## Change the localization [#change-the-localization]
In the console:
1. Open **Settings → Project**.
2. In **Market**, select the country and the currency.
3. Click **Save**.
With the API, send [`PATCH /projects/{project_id}`](/api/reference/projects/update-project.md). You need the `ADMIN` role in the project.
```bash
curl -X PATCH https://api.deepscout.ai/v1/projects/$PROJECT_ID \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"localization": {"country": "AT", "currency": "EUR"}}'
```
```json title="Response"
{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"name": "Default project",
"customer_id": "cust_01k7c2r9t2f8a5d3h7j9k1m6pw",
"is_default": true,
"localization": { "locale": "de_AT", "country": "at", "currency": "EUR" }
}
```
The API accepts the country and the currency in any letter case. An unknown country or currency gets `400 INVALID_LOCALE`.
A change does not convert the current export files. The next completed scrape or matching writes all export files of the project again, in the new currency. See [Know when new files are ready](/api/exports.md#know-when-new-files-are-ready).
## Currency conversion [#currency-conversion]
DeepScout uses the [euro reference rates](https://www.ecb.europa.eu/stats/policy_and_exchange_rates/euro_reference_exchange_rates/html/index.en.html) of the European Central Bank (ECB). The ECB publishes new rates on working days, at about 16:00 Central European Time. An export uses the latest rates at the time when the API writes the files.
The conversion changes these fields of the [`all_products_with_details`](/datasets/products.md) dataset:
| Field | Change |
| --- | --- |
| `price` | Converted to the currency of the project. |
| `price_standard`, `price_discounted`, `price_coupon`, `price_club`, `price_recommended` | Converted to the currency of the project. |
| The same five prices with the `_no_vat` suffix | Converted to the currency of the project. |
| `currency` | Changes to the currency of the project. |
These prices do not change:
- The prices in the `history` records. They stay in the currency of the website.
- The [sample data](/api/sample-data.md) of a setup job. It shows the prices of the store.
DeepScout does not round the converted prices. Thus a converted price can have more decimal places than its currency uses, for example `485.9569` CZK.
The conversion needs an ECB rate for the currency of the project and for the currency of each website. The ECB publishes rates for about 30 currencies, for example `EUR`, `USD`, `GBP`, `CHF`, `CZK`, `PLN` and `HUF`.
# Match products (/api/matching)
A matching job finds the same products on different websites. You select one **reference** website, usually your own store, and one or more **competitor** websites. The result is a list of product pairs, with the prices from both websites.
A paid run matches every pair of its websites: the reference website with each competitor, and also the competitors with each other. Thus the matches also show the products that only the competitors sell. See [Use the matches](#use-the-matches).
The matching first finds candidate pairs. Then an AI model verifies them.
## Before you start [#before-you-start]
- Each website in the matching must have a completed scrape in the project, a full scrape or a URL scrape.
- A project runs one matching at a time. A matching quote that waits for approval also counts.
## Get a free sample [#get-a-free-sample]
Each project gets one free sample. The sample matches some products of the reference website against the full catalogs of the competitors. It uses 20% of the reference catalog, but not more than 2,000 products. The sample starts immediately, without a quote. The sample does not match the competitors with each other.
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/matching \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"reference_website_id": "my-shop.cz",
"competitor_website_ids": ["example-shop.cz"],
"sample": true
}'
```
```python tab="Python"
job = api.post(f"{BASE}/jobs/matching", json={
"project_id": project_id,
"reference_website_id": "my-shop.cz",
"competitor_website_ids": ["example-shop.cz"],
"sample": True,
}).json()
```
If the project already used its sample, the API answers `409 MATCHING_SAMPLE_USED`. If the earlier sample failed, you can start a new one. You cannot cancel a sample.
## Run a paid matching [#run-a-paid-matching]
### 1. Create the quote [#1-create-the-quote]
Send [`POST /jobs/matching`](/api/reference/matching-jobs/create-matching-job.md) without `sample`. You can add any number of competitor websites. One run matches all pairs of its websites, also the competitors with each other.
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/matching \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"reference_website_id": "my-shop.cz",
"competitor_website_ids": ["example-shop.cz", "drogerie-example.cz"]
}'
```
```python tab="Python"
job = api.post(f"{BASE}/jobs/matching", json={
"project_id": project_id,
"reference_website_id": "my-shop.cz",
"competitor_website_ids": ["example-shop.cz", "drogerie-example.cz"],
}).json()
print(job["estimate"])
```
```json title="estimate of a first run"
{
"currency": "EUR",
"unit_price_per_1000": "7.00",
"quantity": 35000,
"total": "245.00",
"cap": "367.50",
"min_charge": "0.10",
"input_products": 35000
}
```
- `input_products` is the number of products of all websites in the run.
- `quantity` is the number of products that you pay for. The first run of a website pays for all its products. A later run pays only for new and changed products. See [Run matching again](#run-matching-again).
- `unit_price_per_1000` comes from volume bands. A large run has a lower price for 1,000 products. See [Pricing](/api/pricing.md#matching).
The API answers `400 INVALID_MATCHING_SCOPE` if the reference website is also a competitor, or if a competitor is in the list two times. It answers `409 MATCHING_BUSY` if another matching of the project is not finished. To remove an old quote, send [`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md).
### 2. Approve the quote [#2-approve-the-quote]
Send [`POST /jobs/matching/{job_id}/approve`](/api/reference/matching-jobs/approve-matching-job.md). The approval holds the `cap` from your credit. It can answer `ESTIMATE_CHANGED`, `INSUFFICIENT_FUNDS` or `MONTHLY_LIMIT_REACHED`, as for a [scrape](/api/full-scrape.md#3-approve-the-quote).
```bash
curl -X POST https://api.deepscout.ai/v1/jobs/matching/jobreq_01k7e1a2b3c4d5e6f7g8h9j0k1/approve \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
### 3. Follow the progress [#3-follow-the-progress]
Read the job with [`GET /jobs/matching/{job_id}`](/api/reference/matching-jobs/get-matching-job.md). It has the same statuses as a scrape: `APPROVED`, `RUNNING`, `SETTLING`, `EXPORTING` and `COMPLETED`. While it runs, `matching_progress.phase` shows the step:
| `phase` | Console label | Description |
| --- | --- | --- |
| `MATCHING_PRODUCTS` | Matching products | The API finds candidate pairs. `products_processed` of `products_total` products are done, and `percent` is their share. |
| `VERIFYING_MATCHES` | Verifying pairs with AI | An AI model verifies the candidate pairs. `estimated_minutes_left` shows the time left. It can also increase. |
```json title="matching_progress while the run verifies the pairs"
{
"phase": "VERIFYING_MATCHES",
"estimated_minutes_left": 40
}
```
Read the job every 1 to 5 minutes. You get an email when the job completes.
## Read the results [#read-the-results]
A completed job shows the counters of its run:
```json title="A completed matching (part)"
{
"job_type": "MATCHING",
"status": "COMPLETED",
"reference_website_id": "my-shop.cz",
"competitor_website_ids": ["example-shop.cz"],
"sample": false,
"websites": [
{ "website_id": "my-shop.cz", "products": 12400, "modified": 12400 },
{ "website_id": "example-shop.cz", "products": 22600, "modified": 22600 }
],
"overlap": [
{ "website_id": "my-shop.cz", "match_website_id": "example-shop.cz", "products": 8060 }
],
"matching_progress": {
"products_checked": 35000,
"products_skipped_not_modified": 0,
"new_matches": 8060
},
"settlement": {
"quantity": 35000,
"currency": "EUR",
"amount": "245.00",
"capped": false,
"settled_at": "2026-10-13T09:10:00.000Z"
}
}
```
| Field | Description |
| --- | --- |
| `websites` | For each website of the run: the products that it loaded (`products`), and the products that are new or changed since the last run (`modified`). |
| `overlap` | For each pair of websites: the number of products that the matches connect. It includes products that a third website connects. |
| `matching_progress.new_matches` | The number of new pairs from this run. |
The pairs are in the `matches` dataset of the project. See [Download exports](/api/exports.md) and the [Matches dataset](/datasets/matches.md).
## Use the matches [#use-the-matches]
The [Matches dataset](/datasets/matches.md) has one row for each product and its match on another website. A pair has two rows, one for each direction. A competitor product can have a match on another competitor and no match on the reference website.
To get the names and the prices, join the matches with the product files of the websites. Use `website_id` and `product_id` on both sides of a pair. The examples use the files from [Download exports](/api/exports.md), saved with short names.
```python title="Load the matches and the prices"
import pandas as pd
reference = "my-shop.cz"
websites = ["my-shop.cz", "example-shop.cz", "drogerie-example.cz"]
matches = pd.read_json("matches.jsonl.gz", lines=True)
products = pd.concat(
pd.read_json(f"all_products_with_details_{website}.jsonl.gz", lines=True)
for website in websites
).set_index(["website_id", "product_id"])[["name", "price"]]
```
### Assortment gaps [#assortment-gaps]
An assortment gap is a product that one or more competitors sell, and the reference website does not. In the matches, it is a competitor product that has matches, but no match on the reference website.
```python title="Products that the competitors sell and my-shop.cz does not"
sellers = matches.groupby(["website_id", "product_id"])["match_website_id"].agg(set)
gaps = sellers[
(sellers.index.get_level_values("website_id") != reference)
& sellers.map(lambda match_websites: reference not in match_websites)
]
# the gaps of one competitor, with the other competitors that also sell the product
example_gaps = products.join(gaps.loc[["example-shop.cz"]], how="inner")
```
A gap product of two competitors is in `gaps` one time for each competitor. To count the gap products, count them on one competitor.
### Price comparison of the market [#price-comparison-of-the-market]
Each pair of websites has its price comparison, also two competitors.
```python title="Share of the common products where the website is cheaper"
prices = products["price"]
pairs = matches.join(prices, on=["website_id", "product_id"]).join(
prices.rename("match_price"), on=["match_website_id", "match_product_id"]
)
pairs["cheaper"] = pairs["price"] < pairs["match_price"]
pairs.groupby(["website_id", "match_website_id"])["cheaper"].mean()
```
### Assortment overlap [#assortment-overlap]
The `overlap` of a completed job gives the number of products that each pair of websites shares. See [Read the results](#read-the-results).
## Run matching again [#run-matching-again]
When the catalogs change, run the matching again with the same websites. A new run pays only for the products that are new or changed since the last run of each pair. The minimum is 5% of the price of a first run. You know the exact charge when the run ends.
A website that is new in the matching of the project pays for all its products, because it has no earlier run.
If the run fails, the job is `FAILED` with the code `MATCHING_FAILED`, and the charge is zero.
# MCP server (/api/mcp)
The DeepScout MCP server gives AI agents the same functions as the API. An agent can add a website, read the sample data, scrape a catalog, match products and get the export files. You write what you want in plain language, and the agent calls the API for you.
The server follows the authenticated remote [MCP specification](https://modelcontextprotocol.io/specification). DeepScout hosts it, so you install nothing. Clients that support remote MCP servers connect directly. Other clients connect through the [`mcp-remote`](https://github.com/geelen/mcp-remote) module.
```txt title="Server URL"
https://mcp.deepscout.ai/mcp
```
## Setup [#setup]
### General [#general]
The server uses the Streamable HTTP transport. The sign-in uses OAuth 2.1 with dynamic client registration:
1. Add the server URL to your client.
2. Start the sign-in in the client. The client opens the [DeepScout Console](https://console.deepscout.ai) in your browser.
3. Sign in with Google or GitHub. Your first sign-in creates your account.
4. Read the name of the app and click **Approve**.
The app gets an API key with the scope **All my projects**. The key does not expire. You can see the app and disconnect it in the console, under **Settings → API keys → Connected apps**.
You can also send your own API key in the `Authorization` header and skip the sign-in. See [Use an API key](#use-an-api-key).
### Claude Code [#claude-code]
```bash
claude mcp add --transport http deepscout https://mcp.deepscout.ai/mcp
```
Start a Claude Code session and run `/mcp`. Select **deepscout**, then **Authenticate**.
### Claude [#claude]
In [claude.ai](https://claude.ai) or in Claude Desktop, open **Settings → Connectors** and click **Add custom connector**. Enter the name `DeepScout` and the server URL. Then click **Connect** to sign in.
In a Team or Enterprise organization, an owner adds the connector for the organization first. Then each member connects with their own DeepScout account.
### Codex [#codex]
Codex keeps one configuration for the CLI and the IDE extension. Add the server:
```bash
codex mcp add deepscout --url https://mcp.deepscout.ai/mcp
```
If Codex does not open the sign-in, run `codex mcp login deepscout`.
You can also add the server to `~/.codex/config.toml`:
```toml title="~/.codex/config.toml"
[mcp_servers.deepscout]
url = "https://mcp.deepscout.ai/mcp"
```
### Cursor [#cursor]
Add the server to `~/.cursor/mcp.json`, or to `.cursor/mcp.json` in your project:
```json title="mcp.json"
{
"mcpServers": {
"deepscout": {
"url": "https://mcp.deepscout.ai/mcp"
}
}
}
```
Open **Cursor Settings → MCP** and click **Connect** next to **deepscout**.
### Visual Studio Code [#visual-studio-code]
Add the server to `.vscode/mcp.json` in your workspace:
```json title=".vscode/mcp.json"
{
"servers": {
"deepscout": {
"type": "http",
"url": "https://mcp.deepscout.ai/mcp"
}
}
}
```
Click **Start** above the server in the file. VS Code opens the sign-in.
### Other clients [#other-clients]
If your client supports remote MCP servers, give it the server URL. If it supports only local (stdio) servers, use `mcp-remote` with these settings:
- **Command**: `npx`
- **Arguments**: `-y mcp-remote https://mcp.deepscout.ai/mcp`
- **Environment**: none
```json
{
"mcpServers": {
"deepscout": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://mcp.deepscout.ai/mcp"]
}
}
}
```
`mcp-remote` needs Node.js 18 or later.
## Tools [#tools]
Each tool is one step of the API. The tools call the API as you, with the key of the client. So they see the same projects and get the same errors as your requests.
| Tool | What it does | API |
| --- | --- | --- |
| `scout_list_projects` | Lists your projects and the customer that pays for each one. | [`GET /projects`](/api/reference/projects/list-projects.md) |
| `scout_list_websites` | Lists the websites of a project, each with its current job and `full_scrape_estimate`. | [`GET /websites`](/api/reference/websites/list-websites.md) |
| `scout_analyze_website` | Adds a website to a project. The setup job analyzes it and scrapes free sample products. | [`POST /jobs/setup`](/api/add-website.md) |
| `scout_get_job` | Reads a setup, scraping or matching job: status, progress, estimate and charge. | [`GET /jobs/{kind}/{job_id}`](/api/jobs.md) |
| `scout_get_sample` | Gives a temporary URL of the sample products of a website. | [`GET /samples/{website_id}`](/api/sample-data.md) |
| `scout_scrape_website` | Creates a scraping job for the full catalog or for a list of product URLs. | [`POST /jobs/scraping`](/api/full-scrape.md) |
| `scout_get_url_results` | Gives a temporary URL of the CSV file with the result of each URL of a URL scrape. | [`GET /jobs/scraping/{job_id}/url-results`](/api/url-scrape.md) |
| `scout_match_websites` | Creates a matching job, or the free matching sample of the project. | [`POST /jobs/matching`](/api/matching.md) |
| `scout_approve_job` | Approves a scraping or matching job. **This charges your credit.** | [`POST /jobs/{kind}/{job_id}/approve`](/api/full-scrape.md#3-approve-the-quote) |
| `scout_list_exports` | Lists the export files of a project. | [`GET /exports`](/api/exports.md) |
| `scout_get_export` | Gives a temporary URL of one export file. | [`GET /exports/{dataset}/url`](/api/exports.md) |
| `scout_get_balance` | Reads the credit of a customer: the total, the held part and the spendable part. | [`GET /customers/{customer_id}/billing/balance`](/api/credits.md#read-the-balance) |
The tools that only read data (`scout_list_*`, `scout_get_*`) have the MCP annotation `readOnlyHint`. You can let your client run them without a prompt. The other tools create or approve jobs. They do not remove or overwrite data.
The server also gives the agent the order of the steps and the usual duration of each job. A scrape of 100,000 products can take one day. The agent knows this and does not report a slow job as a problem.
Sample data, the matching sample and most website analyses are free. A scrape or a matching costs the same as through the API. See [Pricing](/api/pricing.md).
Only `scout_approve_job` charges your credit. The server tells the agent to show you the price and to wait for your answer before it approves a job. Keep the confirmation prompt of your client on for this tool.
## Example prompts [#example-prompts]
Copy a prompt into your client and change the values in brackets. Each prompt names the data that the agent needs, so the agent asks fewer questions.
**Use case**
Add a new website to a project and see if DeepScout reads its product data correctly before you pay for a scrape.
**Prompt**
> Add `[https://www.example-shop.cz]` to my DeepScout project `[Default project]`. The market is `[CZ]` and the currency is `[CZK]`. Wait until the setup job is complete. Then download the sample and tell me which fields are often empty or look wrong. Do not start a paid scrape.
**Use case**
Scrape the full catalog of a website that has a completed setup, with a price that you approve first.
**Prompt**
> In my DeepScout project `[Default project]`, create a full scrape of `[example-shop.cz]`. Show me the price estimate and the maximum charge, and wait for my approval. After I approve, follow the job until it is complete. Then give me the export files of this website.
**Use case**
Get the current prices of some products only, for example the products of a promotion, without a scrape of the full catalog.
**Prompt**
> Scrape only these product URLs on `[example-shop.cz]` in my DeepScout project `[Default project]`. Show me the price and wait for my approval. When the job is complete, read the URL results. Tell me which URLs gave no product, and the reason for each one.
>
> URLs: `[paste the product URLs]`
**Use case**
Find your products on the websites of your competitors. Look at the free matching sample first, then decide about the full matching.
**Prompt**
> In my DeepScout project `[Default project]`, `[my-shop.cz]` is my website and `[competitor-a.cz, competitor-b.cz]` are my competitors. Run the free matching sample first. When it is complete, show me 10 matched pairs from the export. Then create the full matching, show me the price and wait for my approval.
**Use case**
Make a price report from the matches export, when your client can run code or read files.
**Prompt**
> Get the matches export of my DeepScout project `[Default project]` and the product export of each website in it. For each of my products that has at least one competitor, compare my price with the lowest competitor price. List the 20 products with the largest difference in percent: my price, the lowest price, and the competitor that has it.
**Use case**
Find the products that your competitors sell and you do not. A paid matching also matches the competitors with each other, so the matches export has these products.
**Prompt**
> Get the matches export of my DeepScout project `[Default project]` and the product export of each website in it. My website is `[my-shop.cz]`. List the products that two or more competitors sell and my website does not. Give the name, the competitors and the lowest price. Sort them by the number of competitors.
**Use case**
See what runs now and how much credit is left, for example at the start of the day.
**Prompt**
> Show me the state of all websites in my DeepScout projects. For each running job, give the progress and the time left. Then show the spendable credit of my customer.
## FAQ [#faq]
The OAuth sign-in gives the app a key with the scope **All my projects**. The app can do everything that your [roles](/api/authentication.md#roles) let you do in all your projects. This includes the approval of jobs, which uses the credit of the customer.
To keep an agent inside one project, give it a key for this project. See [Use an API key](#use-an-api-key).
Yes. Create a key in the console, under **Settings → API keys**, and send it in the `Authorization` header. The server then skips the sign-in.
```bash title="Claude Code"
claude mcp add --transport http deepscout https://mcp.deepscout.ai/mcp \
--header "Authorization: Bearer $SCOUT_API_KEY"
```
```json title="mcp.json"
{
"mcpServers": {
"deepscout": {
"url": "https://mcp.deepscout.ai/mcp",
"headers": { "Authorization": "Bearer your-api-key" }
}
}
}
```
A key for one project works only in this project. With this key, `scout_get_balance` fails with `403 INSUFFICIENT_SCOPE`, and all other tools work. See [Key scopes](/api/authentication.md#key-scopes).
Open the console, go to **Settings → API keys** and find the app under **Connected apps**. Click the disconnect icon next to the app and confirm. The API revokes the key of the app. Within 5 minutes, the requests of the app get `401`.
When you sign in again from the same app, the app gets a new key. The new key replaces the earlier key of this app.
The client keeps a registration that DeepScout deleted. DeepScout deletes the registration of an app 30 days after you disconnect it. If the first sign-in of an app does not finish, DeepScout deletes its registration after 30 days. Remove the saved sign-in data from the client, then sign in again:
- **Claude Code**: run `/mcp`, select **deepscout**, then **Clear authentication**. Then select **Authenticate**.
- **mcp-remote**: remove the folder `~/.mcp-auth`, then restart the client.
- **Other clients**: remove the server and add it again.
A tool error contains the HTTP status, the error code and the details of the API error. For example, `HTTP 402 INSUFFICIENT_FUNDS` comes with `missing_topup`, the amount to add. Look up the code in [Errors](/api/errors.md).
The agent cannot top up the credit. To top up, use the console or the [top-up endpoint](/api/credits.md#top-up-the-credit).
A job without an error is not stuck. DeepScout reads the pages of an e-shop at a rate that the e-shop accepts, so a large catalog takes many hours. See [Jobs](/api/jobs.md).
You do not need to keep the conversation open. Ask again later, in a new conversation. `scout_list_websites` shows the current job of each website.
Disconnect the server in the MCP settings of your client and connect it again. Your sign-in and your data stay the same.
# Pricing (/api/pricing)
All prices are in EUR. The prices of scrapes and matching are for 1,000 products. VAT is added when you [top up](/api/credits.md#top-up-the-credit) your credit, not on the job charges. The `estimate` of each job is the price that applies. The tables on this page show the current price list.
## Website setup [#website-setup]
A setup job analyzes a website, trains the scraper and extracts sample data. The price is for the analysis:
| Setup job | Price |
| --- | --- |
| The first 10 of each month | Free |
| Each next one in the month | €0.50 |
A failed analysis costs nothing. The exception is an analysis that finds that the website does not offer your currency. See [Add a website](/api/add-website.md#the-website-does-not-offer-the-currency).
## Free [#free]
| Item | Price |
| --- | --- |
| Product sample of a website | Free |
| First matching sample of a project | Free |
## Scrapes [#scrapes]
Full scrapes and URL scrapes have the same price. The price depends on the `difficulty` of the website:
| `difficulty` | When | Price for 1,000 products |
| --- | --- | --- |
| `EASY` | The website gives its pages without a browser. | €0.40 |
| `MEDIUM` | The scraper must use a browser to read the pages. | €0.70 |
| `HARD` | The website has strong protections, for example it needs residential proxies. | €1.20 |
The analysis sets the difficulty. The `quantity` of the estimate is:
- For a full scrape: the latest product count of the website. Before the first full scrape, the count is the estimate of the analysis.
- For a URL scrape: the number of accepted URLs.
**Example.** A full scrape of an `EASY` website with 12,345 products: 12,345 × €0.40 / 1,000 = **€4.94**. The cap is €7.41.
## Matching [#matching]
The price for 1,000 products of a matching run comes from volume bands. The bands are incremental, like tax brackets: each product has the price of the band that it falls into.
| Input products | Price for 1,000 products |
| --- | --- |
| The first 50,000 | €7 |
| From 50,000 to 250,000 | €3 |
| More than 250,000 | €2 |
`input_products` in the estimate is the number of products of all websites in the run. `unit_price_per_1000` is the average price over all bands.
**Example.** A first run with 100,000 input products: 50,000 × €7 / 1,000 + 50,000 × €3 / 1,000 = **€500.00**. The unit price is €5.00 and the cap is €750.00.
### Later runs [#later-runs]
A later run of the same websites pays only for the products that are new or changed since the last run. The minimum is 5% of the price of a first run with the same products. In the example above, the minimum of a later run is €25.00.
## From the estimate to the charge [#from-the-estimate-to-the-charge]
Each paid job has an `estimate` with these values:
| Field | Description |
| --- | --- |
| `quantity` | The estimated number of products. |
| `unit_price_per_1000` | The price for 1,000 products. |
| `total` | `quantity` × `unit_price_per_1000` / 1,000, but not less than `min_charge`. |
| `cap` | 1.5 × `total`. The charge is never more than this value. |
| `min_charge` | The minimum charge of a job: €0.10. For a later matching run, 5% of the price of a first run, if that is more. |
| `difficulty` | Scrapes only: `EASY`, `MEDIUM` or `HARD`. |
| `input_products` | Matching only: the products of all websites in the run. |
The charge uses the actual number of products, at the `unit_price_per_1000` of the approved estimate:
```txt
charge = max(products × unit_price_per_1000 / 1000, min_charge), not more than cap
```
- For a scrape, `products` is the number of products that the scrape collected.
- For a matching, `products` is the number of new and changed products in the run.
If `products` is 0, the charge is 0. A scrape without products fails. A matching without new or changed products completes at €0.00.
The API rounds each amount to whole cents, with half up. The `settlement` of the job shows the final values. If the cap limited the charge, `capped` is `true`.
## Price changes [#price-changes]
The approval calculates the estimate again with the newest data. If the price goes up, the approval stops with `409 ESTIMATE_CHANGED` and gives you the new estimate. If it goes down, you pay the lower price. A job keeps the prices of its approved estimate until its settlement.
# Quickstart (/api/quickstart)
This page takes about five minutes. At the end, you have a website in your project and its analysis runs.
### Get an API key [#get-an-api-key]
1. Sign in to the [DeepScout Console](https://console.deepscout.ai). Your first sign-in creates your account.
2. Open **Settings → API keys** and click **Generate new key**.
3. In **Scope**, select **All my projects**.
4. Copy the key. The console shows it only one time.
Keep the key in an environment variable. The examples in these docs read it from `SCOUT_API_KEY`.
```bash
export SCOUT_API_KEY="your-api-key"
```
A key for one project works only in this project. It cannot read the customer or the credit, so some steps in the guides need a key with the scope **All my projects**. See [Key scopes](/api/authentication.md#key-scopes).
### Find your project [#find-your-project]
All websites, jobs and exports belong to a project. Your account has a default project. The list shows it first.
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/projects \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```python tab="Python"
import os
import requests
api = requests.Session()
api.headers["Authorization"] = f"Bearer {os.environ['SCOUT_API_KEY']}"
BASE = "https://api.deepscout.ai/v1"
projects = api.get(f"{BASE}/projects").json()
project_id = projects[0]["project_id"] # the default project
```
```json title="Response"
[
{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"name": "Default project",
"customer_id": "cust_01k7c2r9t2f8a5d3h7j9k1m6pw",
"is_default": true
}
]
```
Keep the `project_id`. Most requests need it.
### Add a website [#add-a-website]
Send the URL of the store, the country of the shoppers and the currency of the prices. This starts a [setup job](/api/add-website.md). The first 10 setup jobs of each month are free.
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/setup \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"url": "https://www.example-shop.cz",
"country": "CZ",
"currency": "CZK"
}'
```
```python tab="Python"
job = api.post(f"{BASE}/jobs/setup", json={
"project_id": project_id,
"url": "https://www.example-shop.cz",
"country": "CZ",
"currency": "CZK",
}).json()
print(job["job_id"], job["status"])
```
```json title="Response (201 Created)"
{
"job_id": "jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3",
"job_type": "WEBSITE_SETUP",
"status": "ANALYZING",
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"url": "https://www.example-shop.cz",
"country": "cz",
"currency": "CZK",
"created_at": "2026-10-13T09:00:00.000Z",
"updated_at": "2026-10-13T09:00:00.000Z"
}
```
### Follow the progress [#follow-the-progress]
Read the job again to see the progress of the analysis. The analysis usually takes a few minutes. Then a training scrape starts.
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/setup/jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3 \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```python tab="Python"
job = api.get(f"{BASE}/jobs/setup/{job['job_id']}").json()
print(job["status"], job.get("analysis_progress"))
```
When the status is `COMPLETED`, the website is ready. You can look at the [sample data](/api/sample-data.md) and get a quote for the [full scrape](/api/full-scrape.md).
## Next steps [#next-steps]
All steps of the setup job, and what to do when it fails.
One script from a URL to the export files.
# Check the sample data (/api/sample-data)
The training of a website extracts a random sample of its products. The sample is free. Use it to make sure that the data has the fields you need before you approve a paid scrape.
## Get the sample [#get-the-sample]
When the setup job is in the `QUALITY_CHECK` phase or `COMPLETED`, send [`GET /samples/{website_id}`](/api/reference/samples/get-sample.md) with the `project_id`. The API returns a temporary download URL.
```bash tab="cURL" tab-group="lang"
curl "https://api.deepscout.ai/v1/samples/example-shop.cz?project_id=$PROJECT_ID" \
-H "Authorization: Bearer $SCOUT_API_KEY"
```
```python tab="Python"
import json
url = api.get(f"{BASE}/samples/{website_id}", params={"project_id": project_id}).json()["url"]
lines = requests.get(url).text.splitlines()
products = [json.loads(line) for line in lines if line]
print(len(products), products[0]["name"], products[0]["price"])
```
```json title="Response"
{ "url": "https://...amazonaws.com/example-shop.cz.json?X-Amz-Signature=..." }
```
The URL is valid for 1 hour. Do not send the `Authorization` header to this URL. It is a presigned storage URL that needs no key.
If the sample is not ready yet, the API answers `404` with the code `SAMPLE_NOT_FOUND`. Try again after a few minutes.
## The file [#the-file]
- The file is in JSON Lines format: one JSON object for each product, one product on each line. It is not compressed.
- It has 20% of the products of the website, but not more than 1,000.
- After each full scrape of the website, a new random sample replaces the file.
```json title="One line of the sample (formatted)"
{
"name": "Bio kokosový olej 500 ml",
"price": 189,
"price_standard": 229,
"price_discounted": 189,
"currency": "CZK",
"ean": "8594173461123",
"sku": "SKU-1000",
"brand": "Country Life",
"category": "Potraviny __ Oleje",
"category_extracted": ["Potraviny", "Oleje"],
"availability": "Skladem",
"availability_extracted": "IN_STOCK",
"product_url": "https://www.example-shop.cz/p/bio-kokosovy-olej-500-ml",
"image_url": "https://www.example-shop.cz/img/1000.jpg",
"specification": { "Hmotnost": "500 g" }
}
```
## Fields [#fields]
Each line has all these fields. A field that the website does not show is `null`. The example above leaves out the `null` fields.
| Group | Fields |
| --- | --- |
| Product | `name`, `brand`, `brand_extracted`, `description`, `specification`, `variant`, `variants`, `condition`, `condition_extracted` |
| Identifiers | `ean`, `sku`, `product_url`, `image_url`, `images_gallery` |
| Category | `category`, `category_extracted` |
| Price | `price`, `currency`, `price_standard`, `price_discounted`, `price_coupon`, `price_club`, `price_recommended`, and the five detailed prices without VAT, with the suffix `_no_vat` (for example `price_standard_no_vat`) |
| Availability | `availability`, `availability_extracted` |
| History | `history` |
The fields have the same meaning as in the [Products dataset](/datasets/products.md). After a scrape, the export files have all the fields of this dataset.
## What to look for [#what-to-look-for]
- **Prices.** Compare some prices with the website. The price fields show which price types the website has, for example a discounted price or a club price.
- **Identifiers.** If you plan to match products, EAN codes make the matching more accurate.
- **Categories and specifications.** Make sure that the fields that your analysis needs are present.
If the data is not correct, do not approve a scrape. Write to [support@deepscout.ai](mailto:support@deepscout.ai) with the website ID.
# Scrape a URL list (/api/url-scrape)
A URL scrape collects only the product pages that you send. Use it for two tasks:
- **Update a catalog.** Refresh the prices and the availability of some products often, and refresh the full catalog less often.
- **Scrape only some products.** Get the products that you need, without the full catalog.
Before a URL scrape, the website must be in your project. See [Add a website](/api/add-website.md).
## Variants [#variants]
The API selects the variant of each URL scrape. `variant` on the job shows it.
| `variant` | When | The result |
| --- | --- | --- |
| `CATALOG_URLS` | Your project has a fresh full catalog of the website. | The scraped products update the full catalog. The exports have the full catalog with the new data. |
| `URLS_ONLY` | Your project has no fresh full catalog, and one of these is true: it never completed a full scrape of the website, its last completed scrape of the website was `URLS_ONLY`, or you set `urls_only`. | The exports of the website have only the products of URL scrapes. |
A full catalog is **fresh** for 31 days after the start of its full scrape. It is also fresh while a full scrape of your project runs. With a fresh catalog, the variant is always `CATALOG_URLS`, and `urls_only` has no effect.
When a `URLS_ONLY` scrape completes, the catalog products of the website leave your exports. Only the products of URL scrapes stay. The next completed full scrape brings the full catalog back.
## 1. Create the URL scrape [#1-create-the-url-scrape]
Send [`POST /jobs/scraping`](/api/reference/scraping-jobs/create-scraping-job.md) with the list in `urls`. One job accepts from 1 to 100,000 URLs.
```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/scraping \
-H "Authorization: Bearer $SCOUT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"website_id": "example-shop.cz",
"urls": [
"https://www.example-shop.cz/p/bio-kokosovy-olej-500-ml",
"https://www.example-shop.cz/p/vitamin-c-1000",
"https://www.example-shop.cz/p/vitamin-c-1000",
"https://www.other-shop.cz/p/vitamin-d3"
]
}'
```
```python tab="Python"
with open("urls.txt") as file:
urls = [line.strip() for line in file if line.strip()]
job = api.post(f"{BASE}/jobs/scraping", json={
"project_id": project_id,
"website_id": website_id,
"urls": urls,
}).json()
print(job["variant"], job["url_count"], job["rejected"], job["estimate"]["total"])
```
The API cleans the list before it creates the job:
- It adds `https://` to a URL without a scheme, changes `http://` to `https://` and changes the domain to lower case.
- It removes URLs that are not valid (`invalid`).
- It removes URLs of other websites (`other_website`). The `www.` and the bare form of the domain are both accepted. Usually, other subdomains are not.
- It removes duplicates (`duplicate`) and keeps the first copy.
`rejected` counts the removed URLs for each reason. `rejected_urls` shows the first 100 invalid and other-website URLs. The job contains only the accepted URLs, and the quote is for these URLs.
```json title="Response (201 Created)"
{
"job_id": "jobreq_01k7d5s6t7v8w9x0y1z2a3b4c5",
"job_type": "URL_SCRAPE",
"status": "AWAITING_APPROVAL",
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"website_id": "example-shop.cz",
"variant": "CATALOG_URLS",
"url_count": 2,
"estimate": {
"currency": "EUR",
"unit_price_per_1000": "0.70",
"quantity": 2,
"total": "0.10",
"cap": "0.15",
"min_charge": "0.10",
"difficulty": "MEDIUM"
},
"rejected": { "invalid": 0, "other_website": 1, "duplicate": 1 },
"rejected_urls": ["https://www.other-shop.cz/p/vitamin-d3"],
"created_at": "2026-10-13T12:00:00.000Z",
"updated_at": "2026-10-13T12:00:00.000Z"
}
```
If the API accepts no URL, it answers `400 INVALID_URL_LIST`. An empty list, or a list of more than 100,000 URLs, gets `422`. For a larger list, split it into more jobs.
Each request creates a new job, also for the same list.
## 2. Approve the quote [#2-approve-the-quote]
Approve the job with [`POST /jobs/scraping/{job_id}/approve`](/api/reference/scraping-jobs/approve-scraping-job.md). The approval works as for a [full scrape](/api/full-scrape.md#3-approve-the-quote): it holds the `cap`, and it can answer `ESTIMATE_CHANGED`, `INSUFFICIENT_FUNDS` or `MONTHLY_LIMIT_REACHED`.
The approval also selects the variant again. If your catalog became too old since the quote, the approval answers `409 CATALOG_EXPIRED`. If a full scrape made a fresh catalog since the quote, a `URLS_ONLY` job changes to `CATALOG_URLS`.
To remove a quote, send [`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md). A quote that you do not approve for 7 days changes to `CANCELLED`.
## When the catalog is too old [#when-the-catalog-is-too-old]
If the full catalog of your project is older than 31 days, the create request or the approval answers `409`. This does not occur if the last completed scrape of the website was `URLS_ONLY`.
```json title="409 CATALOG_EXPIRED"
{
"code": "CATALOG_EXPIRED",
"detail": "the FULL catalog of website example-shop.cz is older than 31 days"
}
```
Select one of two options:
Your URL list then updates a new full catalog.
1. Cancel the old URL quote, if you have one.
2. Create and approve a [full scrape](/api/full-scrape.md).
3. Create the URL scrape again. While the full scrape runs, the catalog counts as fresh, so the variant is `CATALOG_URLS`.
Only the products of URL lists stay in your exports. The catalog products leave the exports until the next full scrape.
1. Cancel the old URL quote, if you have one.
2. Create the URL scrape again with `"urls_only": true`.
```json
{
"project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
"website_id": "example-shop.cz",
"urls": ["https://www.example-shop.cz/p/vitamin-c-1000"],
"urls_only": true
}
```
## 3. Follow the progress and get the data [#3-follow-the-progress-and-get-the-data]
A URL scrape has the same statuses and the same `scrape_progress` as a [full scrape](/api/full-scrape.md#4-follow-the-progress). While it collects pages, `percent` compares `pages` with `url_count`.
To list the URL scrapes of a website, send [`GET /jobs/scraping`](/api/reference/scraping-jobs/list-scraping-jobs.md) with `project_id` and `website_id`. The list shows `url_count`. The list of URLs is only in [`GET /jobs/scraping/{job_id}`](/api/reference/scraping-jobs/get-scraping-job.md).
URL scrapes do not change the state of the website. The `job` of the website stays the setup job or the newest full scrape.
The charge is for the products that the scrape collected. A URL that is not a product page, or that does not exist any more, gives no product and costs nothing. The [URL results](#4-check-the-result-of-each-url) show which URLs gave a product. When the job is `COMPLETED`, the data is in the [export files](/api/exports.md).
While a `URLS_ONLY` scrape of a website runs, you cannot approve a full scrape of the same website. Approve it when the URL scrape ends.
## 4. Check the result of each URL [#4-check-the-result-of-each-url]
When the scrape settles, the job gets `url_results`. This field counts the URLs of the job for each result. A result without URLs is not in the field. The [job list](/api/reference/scraping-jobs/list-scraping-jobs.md) also shows `url_results`.
```json title="Part of the job"
{
"job_id": "jobreq_01k7e8h9j0k1m2n3p4q5r6s7t8",
"job_type": "URL_SCRAPE",
"status": "COMPLETED",
"variant": "CATALOG_URLS",
"url_count": 10000,
"url_results": { "OK": 9620, "NOT_FOUND": 250, "PRICE_NOT_FOUND": 80, "INVALID_DATA": 50 }
}
```
| `result` | Description |
| --- | --- |
| `OK` | The URL gave a product with a price. |
| `PRICE_NOT_FOUND` | The URL gave a product without a price. The product counts in the charge. |
| `NOT_FOUND` | The page answered with an error status, for example 404 or 410. Usually, the product does not exist any more. |
| `IGNORED_RESPONSE` | The page is not a product page, for example a file, or it did not load after all retries. |
| `IGNORED_REQUEST` | The scraper did not download the URL. |
| `INVALID_DATA` | The page loaded, but its data gave no valid product. |
| `UNKNOWN` | The scrape has no record of the URL. |
Only the `OK` and the `PRICE_NOT_FOUND` URLs gave a product. The other URLs cost nothing. If many URLs get `IGNORED_REQUEST`, `INVALID_DATA` or `UNKNOWN`, write to [support@deepscout.ai](mailto:support@deepscout.ai) with the job ID.
### Download the file [#download-the-file]
The file shows the result of each URL. Send [`GET /jobs/scraping/{job_id}/url-results`](/api/reference/scraping-jobs/get-scraping-job-url-results.md). The API returns a temporary download URL, as for the [sample data](/api/sample-data.md).
```bash tab="cURL" tab-group="lang"
URL=$(curl -s "https://api.deepscout.ai/v1/jobs/scraping/$JOB_ID/url-results" \
-H "Authorization: Bearer $SCOUT_API_KEY" | jq -r .url)
curl "$URL" -o url_results.csv
```
```python tab="Python"
import csv
import io
url = api.get(f"{BASE}/jobs/scraping/{job_id}/url-results").json()["url"]
text = requests.get(url).content.decode("utf-8")
for row in csv.DictReader(io.StringIO(text)):
if row["result"] != "OK":
print(row["result"], row["url"])
```
```json title="Response"
{ "url": "https://...amazonaws.com/url_results_jobreq_01k7e8h9j0k1m2n3p4q5r6s7t8.csv?X-Amz-Signature=..." }
```
The URL is valid for 1 hour. Do not send the `Authorization` header to this URL. It is a presigned storage URL that needs no key.
The file is in CSV format, in UTF-8, with a header row. It has one row for each URL of the job.
```csv title="url_results_jobreq_01k7e8h9j0k1m2n3p4q5r6s7t8.csv"
url,result,product_url
https://www.example-shop.cz/p/bio-kokosovy-olej-500-ml,OK,https://www.example-shop.cz/p/bio-kokosovy-olej-500-ml
https://www.example-shop.cz/p/vitamin-c-500,OK,https://www.example-shop.cz/p/vitamin-c-500-tablety
https://www.example-shop.cz/p/vitamin-c-1000,NOT_FOUND,
https://www.example-shop.cz/p/darkovy-poukaz,PRICE_NOT_FOUND,https://www.example-shop.cz/p/darkovy-poukaz
```
| Column | Description |
| --- | --- |
| `url` | A URL of the job, as the API accepted it. |
| `result` | The result of the URL. See the table above. |
| `product_url` | The URL of the product. It is different from `url` when the page redirected to another URL. It is empty when the URL gave no product. |
### When the results are available [#when-the-results-are-available]
- The job gets the URL results when it changes to `EXPORTING`. The `COMPLETED` job keeps them.
- If no URL gave a product, the job fails with `SCRAPE_FAILED` and costs nothing. This job also has URL results, so the file shows the cause.
- Before the settlement, and for a job that failed before the settlement, the API answers `404 URL_RESULTS_NOT_FOUND`.
- A full scrape has no URL results. For a full scrape, the API answers `404 JOB_NOT_FOUND`.
# Datasety (/datasets)
| Název | Dataset | Popis | Typ | Zdrojové datasety | Self-service |
|---|---|---|---|---|---|
| [Produkty](/datasets/products.md) | `all_products_with_details` | Kompletní katalog produktů | základní | — | soubor pro každý web |
| [Páry](/datasets/matches.md) | `matches` | Propojení produktů napříč weby | základní | — | ano |
| [Produkty + páry](/datasets/products-matches.md) | `all_products_with_products` | Produkty s napárovanou konkurencí | odvozený | all_products_with_details
matches | ne |
| [Ceny a dostupnost](/datasets/prices-and-availability.md) | `prices_and_availability_by_website` | Ceny a dostupnost napříč weby | odvozený | all_products_with_details
matches | ne |
Všechny datasety jsou dostupné na:
```
GET https://api.deepscout.ai/v1/exports/{dataset}
Authorization: Bearer VÁŠ_API_KLÍČ
```
Spravovaní zákazníci dostávají všechny čtyři datasety. API klíč najdete v sekci [Nastavení → API](https://reports.deepscout.cz/nastaveni).
Self-service projekty mají jen produkty, a to v samostatném souboru pro každý web, a páry. API klíč najdete v [DeepScout Console](https://console.deepscout.ai) v sekci **Settings → API keys**. Viz [Stažení exportů](/api/exports.md).
# Páry (/datasets/matches)
```
GET https://api.deepscout.ai/v1/exports/matches
Authorization: Bearer VÁŠ_API_KLÍČ
```
Páry produktů, které DeepScout identifikoval jako shodné napříč různými weby. Jeden řádek = jeden produkt a jeho pár na jiném webu. Každý pár má dva řádky, jeden pro každý směr.
Self-service matching páruje všechny weby běhu navzájem, tedy i konkurenty mezi sebou. Z párů tak zjistíte i produkty, které prodávají konkurenti a referenční web ne. Hotové příklady (mezery v sortimentu, srovnání cen trhu) jsou v [Match products](/api/matching.md#use-the-matches).
**Kdo ho dostane:** spravovaní zákazníci i self-service projekty, jeden soubor za celý projekt.
## Pole [#pole]
| Pole | Typ | Popis |
| --- | --- | --- |
| `website_id` | string | Web zdrojového produktu |
| `product_id` | string | ID zdrojového produktu |
| `match_website_id` | string | Web napárovaného produktu |
| `match_product_id` | string | ID napárovaného produktu |
## Příklad [#příklad]
```json
{
"website_id": "eshop-a.cz",
"product_id": "abc-123",
"match_website_id": "eshop-b.cz",
"match_product_id": "def-456"
}
```
## Přesnost párů [#přesnost-párů]
Párování DeepScout neporovnává jen názvy nebo EAN. U každého páru porovnává všechny údaje o produktu: technické parametry a specifikaci, obrázky, popis, všechny kódy produktu (SKU, EAN, kód výrobce), značku, variantu a velikost balení. Pár je shoda, jen když mají oba produkty značnou shodu napříč těmito signály. Najde tak stejný produkt i pod jiným názvem a odmítne produkty, které mají skoro stejný název, ale jsou jiné.
Když váš LLM nebo AI agent při kontrole hlásí chybné páry, většinou neporovnal produkty dostatečně komplexně napříč celým katalogem jako DeepScout Matching.
# Ceny a dostupnost (/datasets/prices-and-availability)
```
GET https://api.deepscout.ai/v1/exports/prices_and_availability_by_website
Authorization: Bearer VÁŠ_API_KLÍČ
```
**Kdo ho dostane:** jen spravovaní zákazníci. Self-service projekty ho nemají. Stejná data získáte spojením datasetů [Páry](/datasets/matches.md) a [Produkty](/datasets/products.md) podle `website_id` a `product_id`.
Odvozený dataset ze základních datasetů [`all_products_with_details`](/datasets/products.md) a [`matches`](/datasets/matches.md): přehledná tabulka cen a dostupností napříč všemi sledovanými weby. Obsahuje jen podmnožinu polí — kompletní data najdete v základních datasetech.
Každý řádek je jeden produkt vašeho webu s cenami a dostupnostmi napárované konkurence ve sloupcích podle webu (`price_*`, `availability_*`).
## Pole [#pole]
| Pole | Typ | Popis |
| --- | --- | --- |
| `website_id` | string | Identifikátor vašeho webu |
| `product_url` | string | URL produktu |
| `image_url` | string | URL obrázku |
| `own_product_id` | string | ID produktu v databázi vašeho webu |
| `name` | string | Název produktu |
| `price_{website_id}` | number | Cena na daném webu |
| `availability_{website_id}` | string | Dostupnost na daném webu |
| `__url_{website_id}` | string | URL napárovaného produktu |
| `__sku_{website_id}` | string | SKU napárovaného produktu |
| `competitors_count` | number | Počet webů s cenou tohoto produktu, **včetně vašeho**. Hodnota 1 znamená, že žádný konkurent cenu nemá. |
Dynamické sloupce (`price_*`, `availability_*`, `__url_*`, `__sku_*`) se generují pro každý web ve vašem projektu.
## Prázdné hodnoty [#prázdné-hodnoty]
Hodnota `null` ve sloupci konkurenta znamená, že u konkurenta nemáme spárovaný produkt s cenou. Konkurent produkt nemusí vést, nebo ho párování nespojilo (například jiný název nebo chybějící EAN). `null` proto neznamená jistě, že konkurent produkt nemá.
Výpočty nad sloupci konkurentů proto platí jen pro spárované produkty.
## Příklad [#příklad]
```json
{
"website_id": "eshop-a.cz",
"product_url": "https://eshop-a.cz/produkt",
"name": "Název produktu",
"own_product_id": "SKU-001",
"price_eshop-a.cz": 399,
"price_eshop-b.cz": 449,
"price_eshop-c.cz": null,
"availability_eshop-a.cz": "IN_STOCK",
"availability_eshop-b.cz": "IN_STOCK",
"availability_eshop-c.cz": "OUT_OF_STOCK",
"__url_eshop-b.cz": "https://eshop-b.cz/produkt",
"__url_eshop-c.cz": "https://eshop-c.cz/produkt",
"competitors_count": 2
}
```
# Produkty + páry (/datasets/products-matches)
```
GET https://api.deepscout.ai/v1/exports/all_products_with_products
Authorization: Bearer VÁŠ_API_KLÍČ
```
**Kdo ho dostane:** jen spravovaní zákazníci. Self-service projekty ho nemají. Stejná data získáte spojením datasetů [Páry](/datasets/matches.md) a [Produkty](/datasets/products.md) podle `website_id` a `product_id`.
Odvozený dataset ze základních datasetů [`all_products_with_details`](/datasets/products.md) a [`matches`](/datasets/matches.md). Obsahuje jen podmnožinu polí — kompletní data najdete v základních datasetech.
Dataset obsahuje produkty všech sledovaných webů. Každý řádek je jeden produkt s vnořeným objektem napárovaného produktu pro každý web, nebo `null` pokud pár neexistuje.
Sloupce s napárovanými produkty se generují dynamicky podle webů ve vašem projektu. Každý sloupec je pojmenovaný podle `website_id`.
Skupina N spárovaných produktů má v datasetu N řádků, jeden pro každý produkt. Skupiny proto nepočítejte podle počtu řádků.
Dataset obsahuje z údajů o produktu jen název, obrázek, SKU a EAN. Párování ale porovnává mnohem víc údajů, viz [Přesnost párů](/datasets/matches.md#přesnost-párů).
## Pole řádku [#pole-řádku]
| Pole | Typ | Popis |
| --- | --- | --- |
| `website_id` | string | Identifikátor webu, kterému produkt patří |
| `product_id` | string | Unikátní identifikátor produktu |
| `{website_id}` | object \| null | Vnořený objekt s napárovaným produktem z daného webu, nebo `null` pokud pár neexistuje |
## Pole vnořeného objektu [#pole-vnořeného-objektu]
| Pole | Typ | Popis |
| --- | --- | --- |
| `website_id` | string | Identifikátor webu |
| `product_id` | string | Unikátní identifikátor produktu |
| `product_url` | string | URL detailní stránky produktu |
| `name` | string | Název produktu |
| `price` | number | Aktuální cena produktu |
| `availability_extracted` | `IN_STOCK` \| `ORDERED` \| `ON_DEMAND` \| `IN_STORE_ONLY` \| `OUT_OF_STOCK` \| `UNKNOWN` \| `DATE` | Normalizovaný stav dostupnosti (viz [Produkty](/datasets/products.md#dostupnost)) |
| `image_url` | string | URL hlavního obrázku produktu |
| `discontinued` | number | Kdy jsme produkt označili jako vyřazený, protože jsme ho několikrát po sobě neviděli (timestamp v ms) |
| `sku` | string | SKU produktu (EAN, GTIN, ISBN, kód produktu apod.) |
| `ean` | string | EAN (případně GTIN) produktu |
| `own_product_id` | string | ID produktu v databázi daného webu. Produkty různých webů podle něj nespojujte. |
## Příklad [#příklad]
```json
{
"website_id": "eshop-a.cz",
"product_id": "abc-123",
"eshop-a.cz": {
"website_id": "eshop-a.cz",
"product_id": "abc-123",
"product_url": "https://eshop-a.cz/produkt",
"name": "Název produktu",
"price": 399,
"availability_extracted": "IN_STOCK",
"image_url": "https://eshop-a.cz/img/produkt.jpg"
},
"eshop-b.cz": {
"website_id": "eshop-b.cz",
"product_id": "def-456",
"product_url": "https://eshop-b.cz/produkt",
"name": "Název produktu",
"price": 449,
"availability_extracted": "IN_STOCK",
"image_url": "https://eshop-b.cz/img/produkt.jpg"
},
"eshop-c.cz": null
}
```
# Produkty (/datasets/products)
```
GET https://api.deepscout.ai/v1/exports/all_products_with_details
Authorization: Bearer VÁŠ_API_KLÍČ
```
Kompletní katalog produktů napříč sledovanými weby. Jeden řádek = jeden produkt.
**Kdo ho dostane:** spravovaní zákazníci jeden soubor za celý projekt. Self-service projekty jeden soubor pro každý web, s parametrem `website_id`. Viz [Stažení exportů](/api/exports.md).
Přibyla nová granulární cenová pole (`price_standard`, `price_discounted`, `price_coupon`, `price_club`, `price_recommended`). Pole `price` je automaticky syntetizované z těchto polí. Pole `price_before_sale` bylo odstraněno.
## Pole [#pole]
| Pole | Typ | Popis |
| --- | --- | --- |
| `website_id` | string | Identifikátor webu |
| `product_id` | string | Unikátní identifikátor produktu |
| `product_url` | string | URL detailní stránky produktu |
| `image_url` | string | URL hlavního obrázku produktu |
| `images_gallery` | string[] | URL dalších obrázků produktu |
| `own_product_id` | string | ID produktu v databázi daného webu. Produkty různých webů podle něj nespojujte, k tomu slouží [`matches`](/datasets/matches.md). |
| `name` | string | Název produktu |
| `brand` | string | Značka produktu |
| `brand_extracted` | string | Značka (normalizovaná) |
| `category` | string | Text odkazů z drobečkové navigace |
| `category_extracted` | string[] | Seřazený seznam kategorií od nejobecnější po nejkonkrétnější |
| `description` | string | Popis produktu |
| `variant` | string | Varianta produktu – barva, velikost, objem, model, velikost balení apod. |
| `condition` | string | Stav produktu (prázdné = nový, nebo: new, used, refurbished, open box apod.) |
| `price` | number | Finální syntetizovaná cena (vč. DPH). Automaticky počítaná z granulárních cenových polí ve výchozím pořadí: kupónová → po slevě → běžná |
| `price_standard` | number | Běžná prodejní cena bez jakékoli slevy (vč. DPH) |
| `price_discounted` | number | Cena po slevě (vč. DPH) |
| `price_coupon` | number | Cena po uplatnění kupónu/kódu dostupného na webu (vč. DPH) |
| `price_club` | number | Cena pro členy klubu, po přihlášení, při nákupu přes mobilní aplikaci apod. (vč. DPH) |
| `price_recommended` | number | Doporučená maloobchodní cena (RRP) od dodavatele (vč. DPH) |
| `currency` | string | Měna (např. USD, EUR, Kč apod.) |
| `availability` | string | Dostupnost produktu (přesný text z webu) |
| `availability_extracted` | `IN_STOCK` \| `ORDERED` \| `ON_DEMAND` \| `IN_STORE_ONLY` \| `OUT_OF_STOCK` \| `UNKNOWN` \| `DATE` | Normalizovaný stav dostupnosti (viz [níže](#dostupnost)) |
| `sku` | string | SKU produktu (EAN, GTIN, ISBN, kód produktu apod.) |
| `ean` | string | EAN (případně GTIN) produktu |
| `language` | string | Jazyk dat produktu ve formátu ISO 639-1 (např. en, de, fr) |
| `rating` | number | Hodnocení produktu |
| `reviews_count` | number | Počet recenzí |
| `specification` | string | Specifikace |
| `first_seen` | number | Kdy byl produkt poprvé zachycen na webu (timestamp v ms) |
| `last_seen` | number | Kdy byl produkt naposledy zachycen na webu (timestamp v ms) |
| `discontinued` | number | Kdy jsme produkt označili jako vyřazený, protože jsme ho několikrát po sobě neviděli (timestamp v ms) |
| `history` | object[] | Historie změn dat produktu v čase (viz [níže](#historie)) |
| `ai_fields` | object | Pole extrahovaná pomocí AI (viz níže) |
Když pole v řádku chybí nebo má hodnotu `null`, e-shop tento údaj většinou vůbec neuvádí. Například doporučenou cenu (`price_recommended`) zobrazuje jen část e-shopů.
### Historie [#historie]
Pole `history` obsahuje jen změny. Záznam přibude, když se změní cena, dostupnost, název nebo jiné sledované pole. Záznam obsahuje původní hodnoty změněných polí a tato data:
| Pole | Typ | Popis |
| --- | --- | --- |
| `valid_for_date` | number | Kdy jsme původní hodnotu viděli naposledy (timestamp v ms) |
| `changed_date` | number | Kdy jsme novou hodnotu viděli poprvé (timestamp v ms) |
Hodnotu pole k datu T najdete v nejstarším záznamu s `valid_for_date` ≥ T, který toto pole obsahuje. Když takový záznam není, platí aktuální hodnota. Historie webu začíná dnem, kdy jsme web přidali do projektu.
### Dostupnost [#dostupnost]
Pole `availability_extracted` je normalizovaná hodnota textu z pole `availability`.
| Hodnota | Význam | Příklady textů |
| --- | --- | --- |
| `IN_STOCK` | E-shop má produkt teď a teď ho odešle. | „skladem“, „in stock“, „5 ks“, „poslední kus“ |
| `ORDERED` | E-shop produkt objednal a produkt je na cestě do e-shopu. | „na cestě“, „očekáváme“, „objednáno u dodavatele“ |
| `ON_DEMAND` | Zákazník objedná první, e-shop produkt nedrží. Patří sem předobjednávky, výroba na zakázku a produkty na dotaz. | „na objednávku“, „předobjednávka“, „na dotaz“, „u výrobce“ |
| `IN_STORE_ONLY` | Produkt jde koupit jen na kamenné prodejně. E-shop ho neodešle. | „pouze na prodejně“, „iba v predajni“, „csak üzletben kapható“ |
| `OUT_OF_STOCK` | Produkt teď nejde koupit. | „vyprodáno“, „prodej ukončen“, „coming soon“ |
| `UNKNOWN` | Text dostupnost neříká. | název produktu, cena, akce |
| `DATE` | Vzácné. Text obsahuje datum, ale den a měsíc nejsou jasné. | |
Pravidla:
1. Datum nebo doba dodání rozhoduje první.
2. Doba dodání („do 3 dnů“): do 7 dní je `IN_STOCK`, 8 až 29 dní je `ORDERED`, 30 a více dní je `OUT_OF_STOCK`. Rozsah se počítá podle nižšího čísla: „5–10 dní“ je 5 dní, tedy `IN_STOCK`. Jedna hranice se počítá podle svého čísla: „max 2 týdny“ je 14 dní, tedy `ORDERED`.
3. U data dodání použijeme nejdřívější datum rozsahu.
### AI pole [#ai-pole]
Stahují se jednorázově, pokud byla služba sjednána. Získávají se pouze údaje, které jsou dostupné přímo na webu.
| Pole | Typ | Popis |
| --- | --- | --- |
| `name_en` | string | Plný název produktu přeložený do angličtiny v nejběžnějším pojmenování |
| `primary_comparison_metric` | string | Nejvhodnější metrika pro porovnání s podobnými produkty. Právě jedno slovo: „count", „weight", „volume" nebo „size" |
| `count` | string | Celkový počet kusů v prodejní jednotce (číslo, případně číslo + jednotka) |
| `weight` | string | Celková hmotnost produktu v prodejní jednotce |
| `volume` | string | Celkový objem produktu v prodejní jednotce |
| `size` | string | Rozměr nebo standardizovaný kód velikosti jedné prodejní jednotky |
| `variant` | string | Barva, příchuť, velikost, balení apod. Kombinace oddělené čárkou (např. „100g,chocolate,bottle") |
| `year` | string | Rok vydání, publikace nebo ročník produktu podle kategorie |
| `bundle_content` | string | Obsah balení – všechny položky oddělené čárkou |
| `age_group` | string | Cílová věková skupina (např. baby, adult, senior) |
| `gender` | string | Cílové pohlaví. Hodnoty: „none", „male", „female", „unisex" |
| `category_normalized` | string | Hlavní kategorie produktu podle Google Product Taxonomy |
| `product_type` | string | Standardizovaná klasifikace GS1 Global Product Classification (GPC) |
## Příklad [#příklad]
```json
{
"website_id": "eshop-a.cz",
"product_id": "abc-123",
"product_url": "https://eshop-a.cz/produkt-xyz",
"name": "Název produktu",
"brand": "Značka",
"category": "Home __ Electronics __ Audio __ Headphones",
"category_extracted": ["Electronics", "Audio", "Headphones"],
"price": 1990,
"price_standard": 2490,
"price_discounted": 1990,
"price_coupon": null,
"price_club": 1890,
"price_recommended": 2690,
"currency": "CZK",
"availability_extracted": "IN_STOCK",
"ean": "8801234567890",
"first_seen": 1700000000000,
"last_seen": 1706000000000
}
```
# Pole feedu (/feed-fields)
## Povinná pole [#povinná-pole]
| Pole | Typ | Popis |
| --- | --- | --- |
| `product_url` | string | URL produktu na webu |
| `name` | string | Název produktu |
| `price` | number | Současná cena produktu |
| `image_url` | string | URL hlavního obrázku produktu |
| `description` | string | Popis produktu (nemusí být strukturovaný) |
| `specification` | string | Technické parametry produktu (nemusí být strukturované, může být součástí popisu) |
| `ean` | string | Unikátní EAN / GTIN produktu. Pokud máte více EAN/GTIN kódů, uveďte je v poli `sku` |
Pole `specification` a `ean` jsou povinná pouze v případě, že jsou pro daný produkt dostupná.
## Volitelná pole [#volitelná-pole]
| Pole | Typ | Popis |
| --- | --- | --- |
| `availability` | string | Dostupnost produktu |
| `brand` | string | Značka produktu |
| `category` | string | Kategorie produktu |
| `condition` | string | Stav produktu (`NEW`, `OPEN_BOX`, `LIKE_NEW`, `REFURBISHED`, `USED`, `UNKNOWN`). Pokud nebude vyplněno, předpokládá se `NEW` |
| `currency` | string | Měna ceny (např. USD, EUR, Kč apod.) |
| `images_gallery` | string | Seznam URL všech produktových obrázků |
| `own_product_id` | string | ID produktu ve vaší databázi — usnadní zpětné napárování výsledků ve vašem systému |
| `price_before_sale` | number | Původní cena produktu před slevou |
| `sku` | string | SKU nebo jiný produktový kód. Může jít o seznam více hodnot |
| `variant` | string | Varianta produktu — barva, velikost, model, velikost balení apod. |
# Produktový feed (/feeds)
Produktový feed je soubor s daty o vašich produktech, který DeepScout pravidelně stahuje a zpracovává. Díky němu máte kontrolu nad tím, jaká data do systému vstupují.
Feedy nastavuje DeepScout pro projekty, které spravuje. Pro self-service projekty feedy připravujeme.
## Formát [#formát]
Feed může být v jednom z těchto formátů:
| Formát | Popis |
|---|---|
| **XML** | Nejběžnější formát — fungují i standardní feedy pro Google Shopping nebo Heureku |
| **CSV** | Tabulkový formát s oddělovačem |
| **JSON** | JSON pole, nebo JSONL (jeden objekt na řádek) |
| **XLSX** | Excelový soubor |
## Kde má feed být [#kde-má-feed-být]
Feed musí být dostupný na veřejné URL adrese, ze které ho DeepScout pravidelně stahuje. Podporujeme protokoly HTTP a HTTPS, v případě potřeby i s autentizací (Basic Auth, OAuth2 nebo vlastní hlavičky).
## Frekvence aktualizace [#frekvence-aktualizace]
Frekvenci stahování feedu nastavíme podle vašich potřeb:
| Frekvence | Popis |
|---|---|
| **Denní** | Feed se stahuje jednou denně ve zvolený čas |
| **Týdenní** | Feed se stahuje ve vybrané dny v týdnu |
| **Měsíční** | Feed se stahuje ve vybrané dny v měsíci |
Pokud potřebujete specifický rozvrh, je možné nastavit i vlastní plán.
## Existující feedy [#existující-feedy]
Pokud už máte produktový feed pro jiný systém (Google Shopping, Heureka, Zboží.cz apod.), můžete ho využít i pro DeepScout — stačí, aby obsahoval [požadovaná pole](/feed-fields.md). Mapování polí z vašeho feedu na DeepScout zajistíme na naší straně.
## Pole [#pole]
Přehled všech polí, která feed může obsahovat, najdete na stránce [Pole feedu](/feed-fields.md).
# Úvod (/)
DeepScout má dva druhy zákazníků. Každý z nich má vlastní část dokumentace.
## Spravovaní zákazníci [#spravovaní-zákazníci]
Projekt pro vás nastavuje a spravuje DeepScout. Data a nastavení najdete v aplikaci [reports.deepscout.cz](https://reports.deepscout.cz).
### Exporty [#exporty]
Stahujte produktová data přes API — kompletní katalog, párování produktů, ceny a dostupnost napříč konkurencí.
- [Jak aktivovat exporty](/activation.md)
- [Přehled datasetů](/datasets.md)
### Feedy [#feedy]
Nahrajte svůj produktový feed a DeepScout ho pravidelně zpracuje. Fungují i existující feedy pro Google Shopping nebo Heureku.
- [Produktový feed](/feeds.md)
- [Pole feedu](/feed-fields.md)
## Self-service [#self-service]
Weby si přidáváte sami v [DeepScout Console](https://console.deepscout.ai), přes REST API nebo přes MCP server pro AI agenty. Stáhněte si zdarma ukázková data, spusťte scraping a párování a stáhněte výsledky. Dokumentace je v angličtině.
- [Self-service API](/api.md)
- [Quickstart](/api/quickstart.md)
- [MCP server](/api/mcp.md)
- [Stažení exportů](/api/exports.md)