Manycrawl

Use cases / Market research

Stop re-checking forty competitor pricing pages by hand

Pricing, plan limits, whether there is a free tier: obvious to a person, invisible to a generic scraper, and usually collected by hand into a spreadsheet that is stale a month later. Scrape each page once, ask an AI column for the field, and every competitor lands in one column you can sort and re-run.

Where it stops working — the AI reads the page's text, so a price drawn into an image or hidden behind a toggle may simply not be there.

1 fetchper page, and the text is kept, so a new question does not re-fetch
No selectorsthe AI reads the page, so a redesign does not break a column
1 columnper field you ask for, so filtering and sorting still work

The recipeFour steps

  1. 1

    Google results

    Find the pages. Operators work, so site:competitor.com pricing gets you straight to the page you want rather than the homepage.

  2. 2

    Scrape

    Reads every URL and keeps the page as clean text. Turn on scrolling for pricing pages that load their plans as you go.

  3. 3

    Analyze and write

    One column per field, reading the scraped text: “the monthly price of the cheapest paid plan, as a number”, “yes or no: is there a free tier”. Say the shape you want back and the column stays filterable.

  4. 4

    Check the preview before you run it

    The step people skip. The preview renders your prompt against the first row, which is far cheaper than fixing a column after it ran across forty.

Picking the toolScrape, agent, or crawl

IfUseBecause
You have the exact URLscrape + aiOne fetch, one AI call per field, nothing wasted looking around
You only have the domainsite-agentFinding the page is the hard part, and it browses the site to do it
You want every page, not onecrawlStart at the homepage and collect what is underneath it
The answer is not on one siteweb-agentIt navigates and cross-references rather than reading once

What it costsPay for the work, not the seat

Scraping forty competitor pages is 40 actions, and each AI column over them is another 40. Pro is $39/month with no action limit; Lite is $19/month with 5,000 actions. The AI calls are billed by your provider on your own key, or cost nothing extra if the column runs on Claude Code or Codex as a local agent. The fetching runs on your own machine, from your own IP, so the price stays flat however long the list gets — unlike a credit-priced tool, where every row enriched is another charge.

How that compares with Clay

Where this recipe stops working

An AI column is only as good as the text Scrape handed it. Prices rendered into an image, a monthly/annual toggle that swaps the numbers in the browser, or a plan table that only appears after a click can all leave the answer missing from the text, and a well-written prompt will then say “not found” rather than guess. Two runs are not guaranteed to word an answer identically, so ask for a number or a yes/no when you plan to compare months. For a list of genuinely mixed sites, Site Agent copes better and costs more. Crawl depth is what multiplies: max pages caps the damage, but depth is the dial that runs away.

FAQQuestions people actually ask

Does every row cost an AI call?

Yes, one per row per AI column. The scrape is not repeated, so asking a second question of the same pages costs only the AI side. A small model is usually enough to read a price off a page.

What if a competitor changes their page?

Nothing breaks, because there are no selectors to go stale. Refresh the scrape and re-run the AI column, and it reads the new page the same way it read the old one.

Whose AI key does this use?

Yours. OpenAI, Anthropic or Google, added in Settings, billed by them at their price. We take no cut and never see the key. Or pick Claude Code or Codex as the column's model and it runs on the subscription you already pay for, with no key at all.