Find every page on a website
Enter a domain. We read its sitemaps and follow its links, then count every page we find and group them by section.
Free, no signupUp to 5,000 pages
What it costs
- Here, right now
- Free, no key
- Through the API
- 1 credit a site
- On signup (work email)
- 500 free credits
Every sitemap, not just one
Reads robots.txt for declared sitemaps and opens sitemap indexes all the way down, which is where large sites keep most of their pages.
Grouped by section
Counts pages under /blog, /docs, /products and the rest, so you see the shape of a site before you read a single URL.
The full list as a CSV
The first 25 pages show here. A free account opens the whole list, up to 5,000 pages, ready to download.
Questions
Start with the sitemap, which lists the pages a site wants found, then follow the links on the site to catch pages the sitemap leaves out. This tool does both and merges them, so you get one deduplicated list.
A site: search shows an approximate sample of what Google chose to index. This shows what the site itself publishes in its sitemaps and links, which is usually more, and includes pages Google has not indexed.
Only if they are in a sitemap. A page that is neither linked nor listed cannot be discovered from outside the site, by this tool or any other.
The free tool shows the total and the first 25 pages. With a free account you can see and download up to 5,000 pages per site, and each lookup costs one credit however many pages come back.
Yes. The same lookup is one call to the map endpoint of the API, which returns every URL with its last-modified date when the sitemap has one.
[ Start ]
Found the pages? Now read them.
Crawl or scrape any of them into clean markdown, one credit a page, failures never billed.
Success rate
Median scrape
ms
