Match products

A matching job finds the same products on different websites. You select one reference website, usually your own store, and one or more competitor websites. The result is a list of product pairs, with the prices from both websites.

A paid run matches every pair of its websites: the reference website with each competitor, and also the competitors with each other. Thus the matches also show the products that only the competitors sell. See Use the matches.

The matching first finds candidate pairs. Then an AI model verifies them.

Before you startCopy Anchor Link

• Each website in the matching must have a completed scrape in the project, a full scrape or a URL scrape.
• A project runs one matching at a time. A matching quote that waits for approval also counts.

Get a free sampleCopy Anchor Link

Each project gets one free sample. The sample matches some products of the reference website against the full catalogs of the competitors. It uses 20% of the reference catalog, but not more than 2,000 products. The sample starts immediately, without a quote. The sample does not match the competitors with each other.

curl https://api.deepscout.ai/v1/jobs/matching \
  -H "Authorization: Bearer $SCOUT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
    "reference_website_id": "my-shop.cz",
    "competitor_website_ids": ["example-shop.cz"],
    "sample": true
  }'

If the project already used its sample, the API answers 409 MATCHING_SAMPLE_USED. If the earlier sample failed, you can start a new one. You cannot cancel a sample.

Run a paid matchingCopy Anchor Link

1. Create the quoteCopy Anchor Link

Send POST /jobs/matching without sample. You can add any number of competitor websites. One run matches all pairs of its websites, also the competitors with each other.

curl https://api.deepscout.ai/v1/jobs/matching \
  -H "Authorization: Bearer $SCOUT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "proj_01k7c2r9v8e3s0b1x6m4q5n2tz",
    "reference_website_id": "my-shop.cz",
    "competitor_website_ids": ["example-shop.cz", "drogerie-example.cz"]
  }'

{
  "currency": "EUR",
  "unit_price_per_1000": "7.00",
  "quantity": 35000,
  "total": "245.00",
  "cap": "367.50",
  "min_charge": "0.10",
  "input_products": 35000
}

• input_products is the number of products of all websites in the run.
• quantity is the number of products that you pay for. The first run of a website pays for all its products. A later run pays only for new and changed products. See Run matching again.
• unit_price_per_1000 comes from volume bands. A large run has a lower price for 1,000 products. See Pricing.

The API answers 400 INVALID_MATCHING_SCOPE if the reference website is also a competitor, or if a competitor is in the list two times. It answers 409 MATCHING_BUSY if another matching of the project is not finished. To remove an old quote, send POST /jobs/{job_id}/cancel.

2. Approve the quoteCopy Anchor Link

Send POST /jobs/matching/{job_id}/approve. The approval holds the cap from your credit. It can answer ESTIMATE_CHANGED, INSUFFICIENT_FUNDS or MONTHLY_LIMIT_REACHED, as for a scrape.

curl -X POST https://api.deepscout.ai/v1/jobs/matching/jobreq_01k7e1a2b3c4d5e6f7g8h9j0k1/approve \
  -H "Authorization: Bearer $SCOUT_API_KEY"

3. Follow the progressCopy Anchor Link

Read the job with GET /jobs/matching/{job_id}. It has the same statuses as a scrape: APPROVED, RUNNING, SETTLING, EXPORTING and COMPLETED. While it runs, matching_progress.phase shows the step:

{
  "phase": "VERIFYING_MATCHES",
  "estimated_minutes_left": 40
}

Read the job every 1 to 5 minutes. You get an email when the job completes.

Read the resultsCopy Anchor Link

A completed job shows the counters of its run:

{
  "job_type": "MATCHING",
  "status": "COMPLETED",
  "reference_website_id": "my-shop.cz",
  "competitor_website_ids": ["example-shop.cz"],
  "sample": false,
  "websites": [
    { "website_id": "my-shop.cz", "products": 12400, "modified": 12400 },
    { "website_id": "example-shop.cz", "products": 22600, "modified": 22600 }
  ],
  "overlap": [
    { "website_id": "my-shop.cz", "match_website_id": "example-shop.cz", "products": 8060 }
  ],
  "matching_progress": {
    "products_checked": 35000,
    "products_skipped_not_modified": 0,
    "new_matches": 8060
  },
  "settlement": {
    "quantity": 35000,
    "currency": "EUR",
    "amount": "245.00",
    "capped": false,
    "settled_at": "2026-10-13T09:10:00.000Z"
  }
}

The pairs are in the matches dataset of the project. See Download exports and the Matches dataset.

Use the matchesCopy Anchor Link

The Matches dataset has one row for each product and its match on another website. A pair has two rows, one for each direction. A competitor product can have a match on another competitor and no match on the reference website.

To get the names and the prices, join the matches with the product files of the websites. Use website_id and product_id on both sides of a pair. The examples use the files from Download exports, saved with short names.

import pandas as pd

reference = "my-shop.cz"
websites = ["my-shop.cz", "example-shop.cz", "drogerie-example.cz"]

matches = pd.read_json("matches.jsonl.gz", lines=True)
products = pd.concat(
    pd.read_json(f"all_products_with_details_{website}.jsonl.gz", lines=True)
    for website in websites
).set_index(["website_id", "product_id"])[["name", "price"]]

Assortment gapsCopy Anchor Link

An assortment gap is a product that one or more competitors sell, and the reference website does not. In the matches, it is a competitor product that has matches, but no match on the reference website.

sellers = matches.groupby(["website_id", "product_id"])["match_website_id"].agg(set)
gaps = sellers[
    (sellers.index.get_level_values("website_id") != reference)
    & sellers.map(lambda match_websites: reference not in match_websites)
]

# the gaps of one competitor, with the other competitors that also sell the product
example_gaps = products.join(gaps.loc[["example-shop.cz"]], how="inner")

A gap product of two competitors is in gaps one time for each competitor. To count the gap products, count them on one competitor.

Price comparison of the marketCopy Anchor Link

Each pair of websites has its price comparison, also two competitors.

prices = products["price"]
pairs = matches.join(prices, on=["website_id", "product_id"]).join(
    prices.rename("match_price"), on=["match_website_id", "match_product_id"]
)
pairs["cheaper"] = pairs["price"] < pairs["match_price"]
pairs.groupby(["website_id", "match_website_id"])["cheaper"].mean()

Assortment overlapCopy Anchor Link

The overlap of a completed job gives the number of products that each pair of websites shares. See Read the results.

Run matching againCopy Anchor Link

When the catalogs change, run the matching again with the same websites. A new run pays only for the products that are new or changed since the last run of each pair. The minimum is 5% of the price of a first run. You know the exact charge when the run ends.

A website that is new in the matching of the project pays for all its products, because it has no earlier run.

If the run fails, the job is FAILED with the code MATCHING_FAILED, and the charge is zero.