Every site's robots.txt file lives at the root: yoursite.com/robots.txt. This is a web standard—crawlers like Googlebot always check that exact path first before indexing any pages. Just type your domain followed by /robots.txt into the address bar. If the file exists, you'll see plain text directives. If you get a 404, the site has no robots.txt, which means crawlers treat it as "allow everything." For sites you own, the actual file sits in your web server's root directory. On cPanel hosting, that's usually public_html/robots.txt. On AWS or DigitalOcean, it's wherever your document root points—often /var/www/html/ or similar. If you're on WordPress, it's in the same folder as wp-config.php. Shopify and Wix generate robots.txt automatically, so you access it via the URL but can't always edit the raw file. To check what Google actually sees, use Google Search Console's robots.txt Tester tool under the old crawl settings, or just fetch yoursite.com/robots.txt and compare. Sometimes caching or CDN rules create discrepancies between what you uploaded and what bots fetch. We've seen cases where Cloudflare page rules accidentally blocked /robots.txt, returning a 403—Google then assumes disallow-all and tanks visibility. Common mistakes: placing robots.txt in a subdirectory like /assets/robots.txt doesn't work; crawlers only check the root. Also, if you have multiple subdomains, each needs its own robots.txt at subdomain.yoursite.com/robots.txt. The root domain's file doesn't cascade. At Ottawa SEO, we verify robots.txt during every technical audit because a single "Disallow: /" left over from staging has wiped out indexing for clients more than once. Always double-check after site migrations or CMS updates—deployment scripts sometimes overwrite it.