Indexability vs Crawlability: Two Different Questions — Audit Both Layers
Crawlability means a search engine can reach and fetch a page; indexability means it is allowed to store and rank it. A page can be crawlable but noindex, or indexable but blocked by robots.txt — so a correct audit checks both layers independently rather than assuming one implies the other.
Crawlability and indexability answer different questions. Conflating them is a common cause of silent visibility loss: a page appears 'fine' but is excluded from results because one layer blocks it.
What controls each
| Layer | Controlled by | Effect |
|---|---|---|
| Crawlability | robots.txt | can the crawler fetch it? |
| Indexability | noindex meta/header | can it be stored and ranked? |
The two failure modes
A page can be crawlable but noindex — fetchable, but not stored. Or it can be indexable but blocked by robots.txt — allowed to rank, but never discovered because the crawler cannot fetch it to read the directive. Each is a distinct problem with a distinct fix.
Audit both
A correct audit reports robots directives and noindex directives separately per page, so a page that is crawlable but accidentally noindex is flagged rather than silently excluded.
Frequently asked questions
Can a page be crawlable but not indexable?
Yes. If it returns 200 and is allowed by robots.txt but carries a noindex directive, it is fetchable but will not be stored in the index.
If I disallow a page in robots.txt, is it deindexed?
Not reliably. Disallowing blocks crawling, which can prevent the crawler from reading your noindex tag. To remove a page, use noindex and let it be crawled.
About the author
RankoraXPress Team — Search Visibility Editorial
The RankoraXPress editorial team publishes original daily briefs on search visibility — covering SEO, AEO, GEO, and developer SEO. Every brief is written in-house and never scraped or copied from third-party sources.
Expertise: SEO, AEO, GEO, and developer SEO