Your robots.txt file must live at the root of your domain: yourdomain.com/robots.txt. Search engines check this exact location when they first crawl your site, and they won't look anywhere else. Placing it in a subfolder like /seo/robots.txt or /wp-content/robots.txt renders it completely invisible to crawlers. For most sites, this means uploading the file via FTP or your hosting control panel to the public_html, www, or htdocs folder—whatever your host uses as the web root. On WordPress, that's the same directory where you find wp-config.php and the wp-content folder. On static sites or platforms like Shopify, you'll typically use the platform's built-in editor or settings panel, since direct file access may be restricted. Common mistakes we see at Ottawa SEO: - Placing robots.txt inside a CMS theme folder or subdirectory, expecting it to propagate up. - Creating separate robots.txt files for different sections of the site—only the root file counts. - Using a subdomain setup where blog.yourdomain.com needs its own robots.txt at blog.yourdomain.com/robots.txt, separate from the main domain. If you're running a multisite or have staging environments, each distinct hostname needs its own robots.txt at its own root. Staging sites should almost always disallow all crawling unless you're testing specific bot behavior. One protocol detail: robots.txt is case-sensitive on some servers. Stick with lowercase "robots.txt" rather than Robots.txt or ROBOTS.TXT to avoid edge-case failures. After uploading, test it immediately by visiting yourdomain.com/robots.txt in a browser—you should see plain text. Then verify in Google Search Console under the URL Inspection tool or the old robots.txt tester to confirm Google reads it correctly. If you see a 404 or your directives aren't respected, the file isn't in the root or there's a server misconfiguration blocking txt files.