Type the domain followed by /robots.txt directly into your browser's address bar. For example, ottawaseo.com/robots.txt or amazon.ca/robots.txt. The file loads as plain text immediately if it exists. If you see a 404 error, the site hasn't created one, which means crawlers assume everything is allowed by default. The robots.txt file lives at the root level of a domain, never in a subdirectory. If you're checking a subdomain like blog.example.com, you'd use blog.example.com/robots.txt because subdomains can have their own rules separate from the main domain. HTTPS versus HTTP doesn't change the path, but modern sites typically redirect HTTP to HTTPS automatically. What you'll see inside varies widely. Large sites like Amazon or government portals show hundreds of lines blocking specific crawlers, limiting access to filters or session URLs, and pointing to multiple sitemap files. Smaller business sites might have five lines blocking /admin/ or /wp-admin/ and one sitemap reference. Some sites accidentally block everything with "User-agent: * Disallow: /" which is a catastrophic SEO mistake we've seen on new WordPress installs where a plugin or developer left the site in noindex mode. When we audit a client's technical SEO, checking robots.txt is step one. We look for over-blocking (disallowing important pages), conflicting rules, or missing sitemap declarations. We also verify that staging environments use a full disallow while production allows legitimate crawlers. If you're analyzing competitors, their robots.txt reveals which sections they're hiding from Google, which tools they use (like Semrush or Ahrefs bots getting blocked), and whether they're sophisticated enough to manage crawl budget on large sites. Browser extensions exist for this, but typing the URL manually is faster and works universally. If you need to check hundreds of domains, tools like Screaming Frog or a simple Python script can pull them in bulk, but for everyday checks, the manual method takes five seconds.