Scrape: read each row's web page
The Scrape column opens each row’s website and fills in what it finds there: title, description, page text, links, social profiles, emails and phone numbers, and optionally the technology the site uses. It needs no key and no login, and each row is one action.
Add the column
- Open a table that has a column of web addresses and click Enrichment. Choose Scrape.
- Under Column with URLs, pick the column. An address without
https://gets it added. - Tick what you want under What to scrape. Only ticked items are stored, and you can add more later with + Column.
- Set Parallel scrapes (1–8), 4 by default. Run a few rows first, then the rest.
What you can collect
| Tick this | You get a column called | On by default |
|---|---|---|
| Final URL (after redirects) | Redirect URL | Yes |
| Title | Title | Yes |
| Meta description | Description | Yes |
| Body text (plain text, best for AI) | Body text | Yes |
| Markdown (structured, larger) | Markdown | No |
| Full HTML | HTML Content | No |
| Page links | Links | Yes |
| Social profiles | Social media | Yes |
| Email addresses | Emails | Yes |
| Phone numbers | Phone numbers | Yes |
| Tech stack (CMS, analytics, payments, framework, hosting) | Tech stack | No |
| SSL cert issuer (a hosting or CDN hint) | SSL cert issuer | No |
| Founder / owner name | Founder | No |
The last three are under Business intelligence, which starts collapsed. Tech stack, SSL cert issuer and Founder are filled by the desktop app’s engine and stay empty without it. The Tech stack cell lists technology names, with a version when one is found.
How it works
Manycrawl loads each page in a background browser, so pages that build themselves with JavaScript still work. It retries a page that shows a bot-protection challenge. It cannot read pages that need you to be signed in.
Empty cells and errors
- An empty cell means the page does not contain that item. Not every site shows an email or a founder name.
- “Missing URL” or “No records with a valid URL to scrape”: the address cell is empty or not a web address.
- “403 Forbidden”, “404 Not Found”, “429 Too Many Requests”: the site refused, the page does not exist, or it is rate-limiting you. Try again later, or lower Parallel scrapes.
- “The engine is still starting”: wait a few seconds and run again. If it keeps happening, restart Manycrawl.
To go beyond one page per row, use Crawl.