# Add a website (/en/api/add-website)

A setup job adds a website to your project. The job does three things:

1. **Analysis.** It finds the protections of the website, the language, the type of products and the size of the catalog. This usually takes a few minutes.
2. **Training.** It scrapes a small part of the website to learn how to read the product pages. This can take up to about two hours.
3. **Sample.** It extracts up to 1,000 sample products, so that you can examine the data before you pay.

## Before you start [#before-you-start]

Each new setup job uses one website analysis of your customer. The customer gets 10 free analyses each month. After them, an analysis costs €0.50 from your [credit](/en/api/credits.md). To see the free analyses that remain and the price, read the customer with a `USER` key:

```bash
curl https://api.deepscout.ai/v1/customers/$CUSTOMER_ID \
  -H "Authorization: Bearer $SCOUT_API_KEY"
```

```json title="Response"
{
  "customer_id": "cust_01k7c2r9t2f8a5d3h7j9k1m6pw",
  "name": "Example Ltd",
  "role": "OWNER",
  "quotas": {
    "analyses_remaining": 9,
    "analyses_per_month": 10,
    "currency": "EUR",
    "analysis_price": "0.50"
  }
}
```

The `customer_id` of a project is in [`GET /projects`](/en/api/reference/projects/list-projects.md).

When `analyses_remaining` is `0`, the next setup job is paid:

1. When the API creates the job, it holds `analysis_price` from your credit. If the spendable credit is too low, the API answers `402 INSUFFICIENT_FUNDS`. If the price does not fit in the [monthly spend limit](/en/api/credits.md#monthly-spend-limit), it answers `403 MONTHLY_LIMIT_REACHED`.
2. When the analysis adds the website to your project, the API charges the price.
3. If the analysis fails, the API releases the hold and charges nothing.

A failed analysis also gives a free analysis back. At the start of each month (UTC), the free analyses go back to 10. Free analyses that you do not use do not carry over.

## Start the setup job [#start-the-setup-job]

Send [`POST /jobs/setup`](/en/api/reference/setup-jobs/create-setup-job.md):

| Field | Description |
| --- | --- |
| `project_id` | The project that gets the website. |
| `url` | The URL of the store, for example `https://www.example-shop.cz`. The API uses only the domain. It ignores the path and the query. |
| `country` | The country of the shoppers, as a two-letter country code (ISO 3166-1 alpha-2), for example `CZ`. `UK` is also accepted for `GB`. The scraper browses the store as a shopper from this country. |
| `currency` | The currency of the prices, as an ISO 4217 code, for example `CZK`. |

```bash tab="cURL" tab-group="lang"
curl https://api.deepscout.ai/v1/jobs/setup \
  -H "Authorization: Bearer $SCOUT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
    "url": "https://www.example-shop.cz",
    "country": "CZ",
    "currency": "CZK"
  }'
```

```python tab="Python"
job = api.post(f"{BASE}/jobs/setup", json={
    "project_id": project_id,
    "url": "https://www.example-shop.cz",
    "country": "CZ",
    "currency": "CZK",
}).json()
```

The API answers `201 Created` with the new job. If the project already has an open setup job for the same website, the API returns this job with `200 OK`. It does not use an analysis.

The API accepts the country and the currency in any letter case. It returns the country in lower case (`cz`) and the currency in upper case (`CZK`).

If the project has no localization, the setup job gives it this country and this currency when it adds the website. See [Localization](/en/api/localization.md#how-a-project-gets-its-localization).

<Callout title="Only some products?">
The setup is always for the whole store. To scrape only some products, add the website first. Then [scrape a URL list](/en/api/url-scrape.md).
</Callout>

## Follow the job [#follow-the-job]

Read the job with [`GET /jobs/setup/{job_id}`](/en/api/reference/setup-jobs/get-setup-job.md). The job goes through these statuses:

| Status | What happens |
| --- | --- |
| `CREATED` | The job is in the queue. |
| `ANALYZING` | The analysis runs. `analysis_progress` shows each step. |
| `ESTIMATE_READY` | The analysis is complete. The website is now in the project with its `full_scrape_estimate`, and the job has its `website_id`. |
| `TRAINING` | The training scrape runs. `train_progress` shows the phase and the percent. |
| `COMPLETED` | The website is ready, with its sample data. |

If DeepScout already knows the website, the job skips the analysis. If the scraper of the website is already trained, the job also skips the training and completes in a few seconds.

### Analysis [#analysis]

`analysis_progress` has four steps. Each step has a `state`: `RUNNING`, `DONE` or `FAILED`. A step that has not started is not in the response.

| Step | What it finds | Result fields |
| --- | --- | --- |
| `config` | The protections of the website and how to get past them. | `challenge` on the website |
| `homepage` | If the website is a store, its language and its type of products. | `is_ecommerce`, `language`, `product_type` |
| `sitemap` | The URLs in the sitemap. | `sitemap_size` |
| `size` | The number of products in the catalog. | `products_estimate` |

```json title="A setup job during the analysis"
{
  "job_id": "jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3",
  "job_type": "WEBSITE_SETUP",
  "status": "ANALYZING",
  "project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
  "url": "https://www.example-shop.cz",
  "country": "cz",
  "currency": "CZK",
  "analysis_progress": {
    "config": { "state": "DONE" },
    "homepage": { "state": "DONE" },
    "sitemap": { "state": "RUNNING" },
    "is_ecommerce": true,
    "product_type": "health and household products",
    "language": "cs"
  },
  "created_at": "2026-10-13T09:00:00.000Z",
  "updated_at": "2026-10-13T09:02:10.000Z"
}
```

### Training [#training]

When the analysis is complete, the job gets its `website_id` and the website shows in [`GET /websites`](/en/api/reference/websites/list-websites.md). Then the training scrape starts. `train_progress` shows how far it is:

| `phase` | Console label | Typical time left |
| --- | --- | --- |
| `COLLECTING_PAGES` | Collecting pages | About 2 hours |
| `ANALYZING_PAGES` | Analyzing pages | Less than 1 hour |
| `UNDERSTANDING_PRODUCTS` | Understanding products | Less than 30 minutes |
| `EXTRACTING_SAMPLE` | Extracting sample products | A few minutes |
| `PREPARING_SAMPLE` | Preparing your sample | A few minutes |
| `QUALITY_CHECK` | Final check | A few minutes |

```json title="train_progress during the training"
{
  "phase": "COLLECTING_PAGES",
  "percent": 10,
  "pages": 800,
  "pages_estimate": 2600,
  "pages_limit": 5000
}
```

`percent` is the progress of the whole training. While the phase is `COLLECTING_PAGES`, `pages` is the number of pages that the scraper downloaded.

### Wait for the result [#wait-for-the-result]

A setup job takes minutes to hours, so read it every 30 to 60 seconds. You also get an email when the training completes or fails.

```python
import time

TERMINAL = {"COMPLETED", "FAILED", "CANCELLED"}

while job["status"] not in TERMINAL:
    time.sleep(30)
    job = api.get(f"{BASE}/jobs/setup/{job['job_id']}").json()
    print(job["status"], job.get("train_progress", {}).get("percent"))

website_id = job.get("website_id")
```

When the status is `COMPLETED`:

- [Check the sample data](/en/api/sample-data.md).
- Read the full-scrape estimate in `full_scrape_estimate` of [`GET /websites/{website_id}`](/en/api/reference/websites/get-website.md).
- [Scrape the full catalog](/en/api/full-scrape.md) or [scrape a URL list](/en/api/url-scrape.md).

## When the setup fails [#when-the-setup-fails]

A failed job has the status `FAILED`, an `error` text and an `error_code`. It also has a `contact_reason`, which tells you which team can help.

| `error_code` | Cause | What to do |
| --- | --- | --- |
| `WEBSITE_TOO_LARGE` | The catalog has more than 2,000,000 products, or the website is a large marketplace. | Ask the sales team for a custom offer. You get the analysis back. |
| `ANALYSIS_FAILED` | The analysis was not able to read the website, for example because the website blocked it. | Make sure that the URL is correct, then start a new setup job. You get the analysis back. |
| `TRAINING_FAILED` | The training was not able to read the product pages. | Ask the support team for help. |

To ask for help, send [`POST /contact-requests`](/en/api/reference/contact/create-contact-request.md) with the `contact_reason` and the `job_id` of the failed job:

```bash
curl https://api.deepscout.ai/v1/contact-requests \
  -H "Authorization: Bearer $SCOUT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "reason": "WEBSITE_TOO_LARGE",
    "job_id": "jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3",
    "phone": "+420 123 456 789",
    "message": "We need only the pharmacy category."
  }'
```

The team contacts you by email, or by phone if you give a number. `phone` and `message` are optional. See [Failed jobs](/en/api/jobs.md#failed-jobs).

## Remove a website [#remove-a-website]

To remove a website from the project, send [`DELETE /websites/{website_id}`](/en/api/reference/websites/remove-website.md) with the `project_id`. You need the `ADMIN` role in the project.

```bash
curl -X DELETE "https://api.deepscout.ai/v1/websites/example-shop.cz?project_id=$PROJECT_ID" \
  -H "Authorization: Bearer $SCOUT_API_KEY"
```

- The API refuses the request with `409 WEBSITE_BUSY` while a job of the website runs.
- The API cancels the quotes of the website that wait for approval.
- The current export files keep the products of the website until the next export of the project.
- To add the website again, start a new setup job. It uses one analysis.

A setup job that failed before it found its website has no `website_id`. These jobs show in [`GET /jobs/setup`](/en/api/reference/setup-jobs/list-setup-jobs.md). To remove one, send [`DELETE /jobs/setup/{job_id}`](/en/api/reference/setup-jobs/remove-setup-job.md).

To stop a setup job that runs, send [`POST /jobs/{job_id}/cancel`](/en/api/reference/jobs/cancel-job.md). You do not get the analysis back.