# First allow all user agenents to crawl site User-agent: * Allow: / # now block specific directories User-agent: * Disallow: /cfpc-beta/ Disallow: /cjis-beta/ Disallow: /dds-beta/ Disallow: /decd-beta/ Disallow: /demhs-beta/ Disallow: /despp-beta/ Disallow: /dob-beta/ Disallow: /drs-beta/ Disallow: /oec-beta/ Disallow: /post-beta/ Disallow: /training/ # help google find the sitemap Sitemap: https://portal.ct.gov/sitemap.xml