User-agent: * Crawl-delay: 2 Disallow: /cdn-cgi/* Disallow: /*? Allow: /*.css Allow: /*.js Sitemap: https://www.challies.com/sitemap.xml User-agent: Twitterbot Allow: /wp-content/themes/20th/images/* Allow: /wp-content/uploads/* Allow: /media/*