Add a website

A setup job adds a website to your project. The job does three things:

1. Analysis. It finds the protections of the website, the language, the type of products and the size of the catalog. This usually takes a few minutes.
2. Training. It scrapes a small part of the website to learn how to read the product pages. This can take up to about two hours.
3. Sample. It extracts up to 1,000 sample products, so that you can examine the data before you pay.

Before you startCopy Anchor Link

Each new setup job uses one website analysis of your customer. The customer gets 10 free analyses each month. After them, an analysis costs €0.50 from your credit. To see the free analyses that remain and the price, read the customer with a USER key:

curl https://api.deepscout.ai/v1/customers/$CUSTOMER_ID \
  -H "Authorization: Bearer $SCOUT_API_KEY"

{
  "customer_id": "cust_01k7c2r9t2f8a5d3h7j9k1m6pw",
  "name": "Example Ltd",
  "role": "OWNER",
  "quotas": {
    "analyses_remaining": 9,
    "analyses_per_month": 10,
    "currency": "EUR",
    "analysis_price": "0.50"
  }
}

The customer_id of a project is in GET /projects.

When analyses_remaining is 0, the next setup job is paid:

1. When the API creates the job, it holds analysis_price from your credit. If the spendable credit is too low, the API answers 402 INSUFFICIENT_FUNDS. If the price does not fit in the monthly spend limit, it answers 403 MONTHLY_LIMIT_REACHED.
2. When the analysis adds the website to your project, the API charges the price.
3. If the analysis fails, the API releases the hold and charges nothing.

A failed analysis also gives a free analysis back. At the start of each month (UTC), the free analyses go back to 10. Free analyses that you do not use do not carry over.

Start the setup jobCopy Anchor Link

Send POST /jobs/setup:

curl https://api.deepscout.ai/v1/jobs/setup \
  -H "Authorization: Bearer $SCOUT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
    "url": "https://www.example-shop.cz",
    "country": "CZ",
    "currency": "CZK"
  }'

The API answers 201 Created with the new job. If the project already has an open setup job for the same website, the API returns this job with 200 OK. It does not use an analysis.

The API accepts the country and the currency in any letter case. It returns the country in lower case (cz) and the currency in upper case (CZK).

If the project has no localization, the setup job gives it this country and this currency when it adds the website. See Localization.

Follow the jobCopy Anchor Link

Read the job with GET /jobs/setup/{job_id}. The job goes through these statuses:

If DeepScout already knows the website, the job skips the analysis. If the scraper of the website is already trained, the job also skips the training and completes in a few seconds.

AnalysisCopy Anchor Link

analysis_progress has four steps. Each step has a state: RUNNING, DONE or FAILED. A step that has not started is not in the response.

{
  "job_id": "jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3",
  "job_type": "WEBSITE_SETUP",
  "status": "ANALYZING",
  "project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
  "url": "https://www.example-shop.cz",
  "country": "cz",
  "currency": "CZK",
  "analysis_progress": {
    "config": { "state": "DONE" },
    "homepage": { "state": "DONE" },
    "sitemap": { "state": "RUNNING" },
    "is_ecommerce": true,
    "product_type": "health and household products",
    "language": "cs"
  },
  "created_at": "2026-10-13T09:00:00.000Z",
  "updated_at": "2026-10-13T09:02:10.000Z"
}

TrainingCopy Anchor Link

When the analysis is complete, the job gets its website_id and the website shows in GET /websites. Then the training scrape starts. train_progress shows how far it is:

{
  "phase": "COLLECTING_PAGES",
  "percent": 10,
  "pages": 800,
  "pages_estimate": 2600,
  "pages_limit": 5000
}

percent is the progress of the whole training. While the phase is COLLECTING_PAGES, pages is the number of pages that the scraper downloaded.

Wait for the resultCopy Anchor Link

A setup job takes minutes to hours, so read it every 30 to 60 seconds. You also get an email when the training completes or fails.

import time

TERMINAL = {"COMPLETED", "FAILED", "CANCELLED"}

while job["status"] not in TERMINAL:
    time.sleep(30)
    job = api.get(f"{BASE}/jobs/setup/{job['job_id']}").json()
    print(job["status"], job.get("train_progress", {}).get("percent"))

website_id = job.get("website_id")

When the status is COMPLETED:

• Check the sample data.
• Read the full-scrape estimate in full_scrape_estimate of GET /websites/{website_id}.
• Scrape the full catalog or scrape a URL list.

When the setup failsCopy Anchor Link

A failed job has the status FAILED, an error text and an error_code. It also has a contact_reason, which tells you which team can help.

To ask for help, send POST /contact-requests with the contact_reason and the job_id of the failed job:

curl https://api.deepscout.ai/v1/contact-requests \
  -H "Authorization: Bearer $SCOUT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "reason": "WEBSITE_TOO_LARGE",
    "job_id": "jobreq_01k7c3a4b5c6d7e8f9g0h1j2k3",
    "phone": "+420 123 456 789",
    "message": "We need only the pharmacy category."
  }'

The team contacts you by email, or by phone if you give a number. phone and message are optional. See Failed jobs.

Remove a websiteCopy Anchor Link

To remove a website from the project, send DELETE /websites/{website_id} with the project_id. You need the ADMIN role in the project.

curl -X DELETE "https://api.deepscout.ai/v1/websites/example-shop.cz?project_id=$PROJECT_ID" \
  -H "Authorization: Bearer $SCOUT_API_KEY"

• The API refuses the request with 409 WEBSITE_BUSY while a job of the website runs.
• The API cancels the quotes of the website that wait for approval.
• The current export files keep the products of the website until the next export of the project.
• To add the website again, start a new setup job. It uses one analysis.

A setup job that failed before it found its website has no website_id. These jobs show in GET /jobs/setup. To remove one, send DELETE /jobs/setup/{job_id}.

To stop a setup job that runs, send POST /jobs/{job_id}/cancel. You do not get the analysis back.