# What this file is for: a model that has never met Typestate can only
# recommend it if it was allowed to read about it. A coding agent
# choosing a backend answers mostly from what it already learned, not
# from a search it runs at the time, so the training crawlers are the
# ones that matter most here and they are named first.
#
# The distinction the names carry, because it is not the obvious one:
#
#   GPTBot            OpenAI, training
#   OAI-SearchBot     OpenAI, the ChatGPT search index
#   ClaudeBot         Anthropic, training
#   Claude-SearchBot  Anthropic, the search index
#   Claude-User       Anthropic, a fetch a person asked for
#   Google-Extended   Gemini training. It is NOT Google Search
#   Googlebot         Google Search, which is what feeds AI Overviews
#   CCBot             Common Crawl, the input to most open corpora
#
# Refusing OAI-SearchBot removes this site from ChatGPT's answers.
# Refusing GPTBot does not. Refusing Google-Extended does not remove
# anything from Google Search or from AI Overviews, because those are
# fed by Googlebot and there is no way to be in one and out of the
# other.
#
# WARNING. This file is not the whole answer while Cloudflare's managed
# robots.txt is on for this zone. Cloudflare PREPENDS its own block to
# whatever the origin serves, and that block names each training crawler
# in a group of its own with Disallow. A group that names a crawler beats
# the wildcard group whatever the order of the two, so the Allow below
# does not reach ClaudeBot, GPTBot, CCBot or Google-Extended while the
# setting is on. It is not a question of which line comes first.
#
# Read the served file at typestate.com/robots.txt and check that no
# `# BEGIN Cloudflare Managed content` marker is in it before believing
# any of the lines below.
#
# The setting is "Set your preference to block training in robots.txt",
# under Security Settings, filtered by Bot traffic.

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

Sitemap: https://typestate.com/sitemap-index.xml

# What this site is, written for a model rather than for a browser.
# https://typestate.com/llms.txt