Free site crawler and SEO audit. No signup.

Crawl your website, see what search engines see, and fix what the checks find. DIY SEO on your own data: up to 5,000 pages a scan, and the whole crawl downloads as a SQLite database you keep.

One email when the scan finishes, with the score and every check. No newsletter.

More options: paths, depth, JavaScript rendering, privacy

What to crawlOnly crawl paths matching (one per line)Leave out paths matching (one per line)Maximum link depth from the start pageDrop these query parameters (comma-separated, * allowed)

Drop tracking parameters utm_*, gclid, fbclid and the like, so tagged links are not crawled twice.Stay inside the start folder Start at /blog/ and only pages under /blog/ are crawled.Crawl subdomains of this host Starting at example.com includes shop.example.com and blog.example.com. Starting at blog.example.com includes only that host and hosts beneath it.Lower-case paths Treat /About and /about as the same page.

Include JavaScript rendering checks Each page is also loaded in a real browser and compared with the HTML the server sent. Slower.Keep this scan private No public report page, no place in the rankings or the sitemap.

Crawls up to 5000 pages plus their images, scripts and stylesheets. No account needed: you get a private link to the results.

No account
No credit card
Up to 5,000 pages
Download the database

Get Started Free Sign In

Everything a DIY SEO audit needs

Site Spider

Crawls up to 5,000 pages plus every image, script and stylesheet, and keeps all of it in one SQLite database you can download.

Sixteen SEO checks

Broken links, redirect chains, titles, meta descriptions, headings, indexability, sitemap coverage, image alt text, heavy assets, orphan pages and more, run as often as you like.

Yours to keep

No account is needed to run a scan. Sign up to keep scans in your dashboard, or take the database and query it with any SQLite tool.

How the Site Spider works

1
Enter your domain
Type your website address on this page or on the Site Spider page. You get a private, unguessable link to the results straight away.
2
We crawl it in the background
The crawler follows internal links, reads your sitemap, respects robots.txt and stores every page, image, script and stylesheet it fetches. You can watch progress live.
3
Run the checks
Each SEO tool queries the stored crawl, so you can run them as many times as you like. Download the SQLite file to dig deeper yourself.

Doing this for the first time? The DIY SEO guide walks through the whole job step by step, from taking a baseline to earning links, and points at the tool for each step.

Questions people ask

Do I need an account?: No. Anyone can start a scan. An account lets you keep scans in one place and find them again later. Scans you started before signing up are attached to your account automatically.
How big a site can it crawl?: Up to 5,000 pages and 1,000 assets per scan, staying on the domain you entered, and the same for everyone: there is no paid tier with higher limits. Files over 10 MB are skipped and counted rather than stored.
Will it load my server?: It starts with one request at a time and a short pause between them, and only goes up to four at once while your server answers quickly; a slow answer, an error or a 429 slows it straight back down. It identifies itself as DIYSEOHubBot, follows your robots.txt and honours a Crawl-delay in it.
How does it compare with Screaming Frog?: Screaming Frog's free version stops at 500 URLs and cannot save, schedule or compare crawls, render JavaScript or extract data; a scan here goes to 5,000 pages and does all of it, with nothing to install. Its paid version still does more, such as unlimited crawls, logging in to a site and Google API integrations. See the feature-by-feature comparison with Screaming Frog, free against free.
What is in the download?: A single SQLite database with the crawl history, every fetched page and asset, extracted titles, headings, links and images, and the tables the checks run against.

Ready to see what search engines see?

Scan your site now, or create a free account to keep every scan in one place.

Start Your Free Account