# Job lifecycle (/api/jobs)

## Job types and endpoints [#job-types-and-endpoints]

| `job_type` | Create | Read one | List |
| --- | --- | --- | --- |
| `WEBSITE_SETUP` | [`POST /jobs/setup`](/api/reference/setup-jobs/create-setup-job.md) | [`GET /jobs/setup/{job_id}`](/api/reference/setup-jobs/get-setup-job.md) | [`GET /jobs/setup`](/api/reference/setup-jobs/list-setup-jobs.md) |
| `FULL_SCRAPE`, `URL_SCRAPE` | [`POST /jobs/scraping`](/api/reference/scraping-jobs/create-scraping-job.md) | [`GET /jobs/scraping/{job_id}`](/api/reference/scraping-jobs/get-scraping-job.md) | [`GET /jobs/scraping`](/api/reference/scraping-jobs/list-scraping-jobs.md) |
| `MATCHING` | [`POST /jobs/matching`](/api/reference/matching-jobs/create-matching-job.md) | [`GET /jobs/matching/{job_id}`](/api/reference/matching-jobs/get-matching-job.md) | [`GET /jobs/matching`](/api/reference/matching-jobs/list-matching-jobs.md) |

The lists need `project_id`. The setup and the scraping lists also accept `website_id`. Each list shows the newest jobs first. `limit` sets the number of jobs, from 1 to 100, and the default is 20.

[`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md) cancels a job of any type.

## Statuses [#statuses]

Each job type goes through its own statuses.

### Setup jobs [#setup-jobs]

```txt
CREATED → ANALYZING → ESTIMATE_READY → TRAINING → COMPLETED
```

The analysis and the training move the job forward without action from you. A setup job can skip `ANALYZING` or `TRAINING` when DeepScout already knows the website. See [Add a website](/api/add-website.md#follow-the-job).

### Scrape and matching jobs [#scrape-and-matching-jobs]

```txt
AWAITING_APPROVAL → APPROVED → RUNNING → SETTLING → EXPORTING → COMPLETED
```

| Status | Description |
| --- | --- |
| `AWAITING_APPROVAL` | The job has a quote in `estimate`. It waits for your approval. |
| `APPROVED` | You approved the quote. The API holds the `cap` from your credit and starts the work. The approve response usually shows `RUNNING` already. |
| `RUNNING` | The work runs. `scrape_progress` or `matching_progress` shows the progress. |
| `SETTLING` | The work is complete. The API calculates the charge. |
| `EXPORTING` | The API writes the export files of the project. |
| `COMPLETED` | The data is in the export files. `settlement` shows the charge. |

A free matching sample has a zero estimate. It starts without your approval.

### End statuses [#end-statuses]

`COMPLETED`, `FAILED` and `CANCELLED` are end statuses. A job in an end status does not change again. `finished_at` shows when it got there.

- `FAILED` can end a job of any type. See [Failed jobs](#failed-jobs).
- `CANCELLED` ends a job that you cancelled, a quote that did not change for 7 days, or the waiting jobs of a removed website.

## Common fields [#common-fields]

All job responses have these fields. A field without a value is not in the response.

| Field | Description |
| --- | --- |
| `job_id` | The ID of the job. |
| `job_type` | `WEBSITE_SETUP`, `FULL_SCRAPE`, `URL_SCRAPE` or `MATCHING`. |
| `status` | The current status. |
| `project_id` | The project of the job. |
| `website_id` | The website of a setup or scrape job. A setup job gets it when the analysis is complete. |
| `estimate` | The quote of a paid job. See [Pricing](/api/pricing.md). |
| `approval` | Who approved the quote (`approved_by`) and when (`approved_at`). |
| `settlement` | The charge: `quantity`, `amount`, `currency`, `capped` and `settled_at`. A failed job settles at zero. |
| `error`, `error_code` | Why the job failed. |
| `contact_reason` | On a failed job: the reason to send in a contact request. |
| `contact_request_id` | The contact request that you sent about the job. |
| `created_at`, `updated_at`, `finished_at` | When the job was created, changed last, and ended. |

## Website state [#website-state]

[`GET /websites`](/api/reference/websites/list-websites.md) shows each website with one `job`: the newer of its setup job and its newest full scrape that is not cancelled. URL scrapes do not change it. The console shows the state of the website from this job:

| `job.status` | Console status |
| --- | --- |
| `CREATED`, `ANALYZING` | Analyzing |
| `ESTIMATE_READY`, `TRAINING` | Training |
| `COMPLETED` of the setup job, or `AWAITING_APPROVAL` | Ready |
| `APPROVED`, `RUNNING`, `SETTLING` | Scraping |
| `EXPORTING` | Exporting |
| `COMPLETED` of a full scrape | Completed |
| `FAILED` with `contact_reason` `WEBSITE_TOO_LARGE` | Custom setup |
| `FAILED` | Failed |
| `CANCELLED` | Cancelled |

The website also has `scraped_at`, the end of its last completed scrape, and `added_at`, the start of its setup.

## Polling [#polling]

The API has no webhooks yet. To follow a job, read it again at an interval:

| Job | Interval |
| --- | --- |
| Setup job | 30 to 60 seconds |
| Scrape or matching | 1 to 5 minutes |
| Top-up status | A few seconds, for up to 2 minutes |

Stop when the job is in an end status, or in `AWAITING_APPROVAL`, which waits for you.

To follow all websites of a project with one request, read [`GET /websites`](/api/reference/websites/list-websites.md). For the scrape jobs of one website, read [`GET /jobs/scraping`](/api/reference/scraping-jobs/list-scraping-jobs.md) with `website_id`.

## Cancel a job [#cancel-a-job]

Send [`POST /jobs/{job_id}/cancel`](/api/reference/jobs/cancel-job.md). You can cancel:

- A setup job before it is complete. You do not get the analysis back.
- A scrape or a matching that waits for approval (`AWAITING_APPROVAL`). This removes the quote.

You cannot cancel a paid job after its approval, or a free matching sample. The API then answers `409 JOB_NOT_CANCELLABLE`.

A quote that you do not approve for 7 days changes to `CANCELLED`.

## Failed jobs [#failed-jobs]

A failed job has an `error` text, an `error_code` and a `contact_reason`.

| `error_code` | Job type | `contact_reason` |
| --- | --- | --- |
| `WEBSITE_TOO_LARGE` | Setup | `WEBSITE_TOO_LARGE` (sales team) |
| `ANALYSIS_FAILED` | Setup | `JOB_FAILED` (support team) |
| `TRAINING_FAILED` | Setup | `JOB_FAILED` (support team) |
| `SCRAPE_FAILED` | Full or URL scrape | `JOB_FAILED` (support team) |
| `MATCHING_FAILED` | Matching | `JOB_FAILED` (support team) |

A failed paid job costs nothing. To ask for help, send [`POST /contact-requests`](/api/reference/contact/create-contact-request.md):

```json
{
  "reason": "JOB_FAILED",
  "job_id": "jobreq_01k7d0f1g2h3j4k5m6n7p8q9r0",
  "phone": "+420 123 456 789",
  "message": "The scrape failed two times."
}
```

- `reason` must be the `contact_reason` of the job. If not, the API answers `409 CONTACT_REASON_MISMATCH`.
- `phone` is optional, with 6 to 20 digits. Without it, the team writes to the email of your account.
- `message` is optional, up to 2,000 characters.

The API answers `201 Created`. A second request about the same job returns the first request with `200 OK`. After the request, the job shows its `contact_request_id`.

## Email notifications [#email-notifications]

DeepScout sends an email to the user who started the job:

| Event | Email |
| --- | --- |
| The training of a website is complete | The website is ready |
| A setup job fails | The setup failed |
| A website has more than 2,000,000 products | An offer from the sales team |
| A scrape or a matching is `COMPLETED` | The data is ready |
| A scrape or a matching fails | The job failed |

There are no emails for quotes, cancellations or top-ups.