User-agent: * Disallow: /orderconftemplates/ Disallow: /wp-content/uploads/usps_label_ # ALLOW FAVICONS FIRST Allow: /wp-content/uploads/cropped-degree-fav-dark-*.png # Google indexed /wp-content/uploads/export_temp/ while it was browsable. Those # URLs now return 410 Gone, but a crawler that is disallowed cannot fetch a URL # and therefore never learns it is gone, so the blanket rule below would keep # them in the index indefinitely. This Allow is longer than that rule and so # wins for this prefix only; everything else under uploads stays disallowed. # Remove once the URLs have dropped out of the index. Allow: /wp-content/uploads/export_temp/ # Slab Replay share cards live here and are the og:image for /replay/. # Applebot honours robots.txt when it fetches a preview image, so while this path # sat under the blanket rule below iMessage showed a bare link with no card at # all, where Facebook and Discord still previewed it because they largely ignore # robots.txt for unfurls. Same longest-match trick as the rule above: this Allow # is longer than the blanket Disallow and so wins for this prefix only. # These files carry grades and rarity and never customer identity, and the pages # that embed them stay noindex, so allowing the image does not put anything into # a search index. Allow: /wp-content/uploads/replay/ # The path those cards were served from before the feature was renamed from # DEGREE Wrapped to Slab Replay. It holds no files now, only a 301 to the line # above, and a crawler that is disallowed cannot fetch a URL and so never learns # it has moved. Kept so an already-sent link still unfurls. Allow: /wp-content/uploads/wrapped/ # Certification page card renders. Every /certification// page names one # of these as its og:image and as the Product schema image, so with the blanket # rule below Google, Bing and the AI crawlers could read the grade but never # fetch the picture of the card. Same longest-match trick as the rules above. # These files carry the card, cert number and grade, never customer identity. Allow: /wp-content/uploads/dgcp-og/ # THEN BLOCK EVERYTHING ELSE Disallow: /wp-content/uploads/ # XML sitemap for search engines and AI crawlers Sitemap: https://degreegrading.com/sitemap_index.xml # LLM-friendly site summary (proposed llmstxt.org convention) # https://degreegrading.com/llms.txt