How ChatGPT finds and cites brands
In short
ChatGPT answers from OpenAI’s own web search, and a site is found for that search by a crawler called OAI-SearchBot. Training is a separate crawler, GPTBot, with a separate switch. This page explains how Agentcites asks ChatGPT, how ChatGPT reads the web, and what a brand can do to be read by it.
How Agentcites asks ChatGPT
Agentcites sends each of your buyers’ questions exactly as written. Nothing of ours is added: no instruction to mention you, and no hint about what we hope to hear.
The question goes to an OpenAI model through OpenRouter, a service that reaches many AI providers through one account. Web search is switched on, and it is OpenAI’s own search, the one its product uses. If that search is not available, the request is refused and recorded as a failure, rather than quietly answered without it.
Every answer is stored word for word, with every link ChatGPT gave and the model that wrote it. When OpenAI changes its model, the answers before and after the change can be told apart.
How ChatGPT uses the web
When ChatGPT searches, it reads pages found by OpenAI’s search and can attach links to the sentences it wrote from them. Agentcites keeps those links with the answer, so you can see which pages it drew on.
The app a person signs into can add things the API does not see, such as their past chats or their location. So an answer measured by Agentcites can differ from the one a particular buyer gets. What it does show is what ChatGPT says to a question with nothing else attached.
Which robot reads your site
OpenAI runs three robots, each with its own name in robots.txt, the file a site uses to tell robots what they may read. OpenAI documents them on its crawler page.
| Robot | What OpenAI says it is for | Follows robots.txt |
|---|---|---|
| OAI-SearchBot | Surfacing websites in search results in ChatGPT’s search features | Yes |
| GPTBot | Making OpenAI’s generative AI foundation models more useful and safe | Yes |
| ChatGPT-User | Certain actions in ChatGPT taken when a user asks | Not always |
OpenAI says robots.txt rules may not apply to ChatGPT-User, because its visits are started by a user.
OAI-SearchBot is the one that decides whether you can be cited. OpenAI says: “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”
The switches are separate. OpenAI says a site “can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot” to keep its pages out of training. It also says a change to robots.txt can take about 24 hours to reach its systems.
What you can do to be cited by ChatGPT
- Allow OAI-SearchBot in robots.txt. Blocking it keeps your pages out of ChatGPT’s search answers.
- Decide about GPTBot on its own. Blocking it is about training, and OpenAI treats it as a separate choice from search.
- Check your firewall as well as robots.txt. OpenAI publishes the addresses its robots come from, and a firewall that blocks them undoes an allow in robots.txt.
- Answer the buyer’s question on the page in plain words. A model that searches can only quote what a page actually says.
- Give each question its own page where you can. A page that answers one thing clearly is easier to cite than a page that touches on ten.
Questions
Does blocking GPTBot stop ChatGPT from citing me?
Not by itself. OpenAI says each robot is set independently, and search uses OAI-SearchBot. Blocking GPTBot says your content should not be used in training. See OpenAI's crawler page.
Is the answer Agentcites records the one a buyer sees?
It is ChatGPT’s answer to the question with nothing added, using OpenAI’s own search. The app can add a person’s history or location, so a buyer’s answer can differ. That is why Agentcites asks every week rather than once.
How long after I change robots.txt will it count?
OpenAI says it can take about 24 hours for its systems to adjust. After that, it still has to search and choose your page for a given question.
Can I see which of my pages ChatGPT used?
Yes, when it gave a link. Agentcites keeps every link ChatGPT returned with each answer and marks which are your pages and which are a competitor’s.