Crawlability and indexability answer two separate questions
Crawlability is whether a search engine can reach and fetch a page at all. Indexability is a later, separate decision: whether that page, once reached, is allowed into the index. A page can pass one test and fail the other, and most technical SEO problems trace back to treating them as one thing.
Definition
Crawlability is a page's ability to be found and retrieved by a search engine's crawler: no robots.txt block standing in the way, no broken path to reach it, content the crawler can actually read. Indexability is a separate, subsequent decision: whether a page the crawler already fetched is allowed into the search index, governed by directives like noindex and canonical tags, plus the engine's own judgment about whether the page is worth storing. A page needs to clear both gates to appear in search results, and clearing one says nothing about the other.
What crawlability actually controls
Crawlability is entirely about access. A robots.txt disallow rule, a server error, a broken redirect chain, or a page with no internal links pointing to it can each stop a crawler from ever reaching a URL. None of these say anything about whether the page, once reached, would have been welcome in the index.
Content rendering is part of the same gate. A crawler that does not execute JavaScript sees only what exists in the raw HTML response. Anything that only appears after the page hydrates in a browser is, to that crawler, simply not there, which is a crawlability failure even though the server returned the page successfully. See do AI crawlers read JavaScript for how unevenly this applies across different crawlers.
What indexability actually controls
Indexability starts only after crawlability has already succeeded. A meta robots noindex tag, a canonical tag pointing to a different URL, or thin and duplicate content can each keep a perfectly reachable page out of the index. The crawler saw the page fine; the search engine simply decided not to store it, or to store a different version instead.
This is the more common failure of the two in practice. A site with clean, working URLs and no crawl errors can still be leaking pages from search results because of directives that have nothing to do with whether those pages could be reached.
Reaching a page and being allowed to keep it are two different decisions, made at two different points.
A few thousand
is roughly where Google draws the line: sites under that many URLs are, in its own words, crawled efficiently most of the time. Crawl budget is a large-site problem, not the reason a typical small business page is missing from results.
Google Search Central, crawl budget guidanceWhere the two get confused in practice
The most common mistake pairs a robots.txt disallow with a noindex tag on the same URL, expecting the two to stack. They do the opposite: a robots.txt block stops the crawler from ever fetching the page, so it never sees the noindex instruction sitting in the page's own head. Google's own guidance is explicit that this combination should never be used together, because the URL can still surface in results with no content shown, built entirely from external links pointing to it.
The fix is ordering, not more rules. A page meant to be excluded from the index should stay crawlable and carry a noindex tag, so the crawler can actually read the instruction. Robots.txt is for pages that should never be fetched at all, not for pages that should be fetched and then quietly dropped.
Common questions
What is the one-line difference between crawlability and indexability?
Crawlability is whether a search engine can reach and fetch a page. Indexability is whether, once fetched, that page is allowed into the index. A page needs to pass both tests to appear in search results, and failing one says nothing about the other.
Can a page be indexable but not crawlable?
Not in a way that helps it. If a crawler cannot reach a page, it never learns whether that page carries a noindex tag or any other indexability signal. A page can still appear in results with no content shown, sourced entirely from links pointing to it, but that is a defect, not a working state.
Can a page be crawlable but not indexable?
Yes, and this is the far more common case. A page can be perfectly reachable, load fine, return a 200 status, and still be excluded from the index by a noindex tag, a canonical tag pointing elsewhere, or a quality judgment the search engine makes on its own.
Does blocking a page in robots.txt guarantee it stays out of search results?
No, and this is the most common mix-up between the two concepts. Robots.txt blocks crawling, not indexing. If other pages link to a blocked URL, it can still be indexed and shown in results, just with no content, because the crawler was never allowed in to see a noindex tag or anything else. Google's own guidance says never combine noindex with a robots.txt disallow on the same URL for exactly this reason: the noindex instruction has to be crawled to be seen.
Does this distinction matter for AI crawlers the same way it does for Google?
The crawlability half matters more, not less. AI crawlers behind ChatGPT and Claude fetch pages but do not execute JavaScript, so content that only renders client-side is effectively uncrawlable to them even when Googlebot handles it fine. The indexability half, noindex and canonical tags, is Google-specific machinery; most AI systems have their own separate rules for what they cite.
AI-readable websites
The structural discipline both crawlability and indexability sit inside.
What is a canonical URL
The main tool for steering which of several duplicate pages gets indexed.
Do AI crawlers read JavaScript
Why crawlability specifically breaks down on client-rendered pages.
What is structured data
What a page should say once it has cleared both gates.
Find out whether your site has a crawlability problem or an indexability one.