Fetches robots.txt and tests if homepage, /admin/, and /api/ are crawlable.
Want all 12 checks at once? Run a full site scan →