![\"Diagram](\"/images/journal-inline-ai-citations-llms-txt-robots-signals.webp\")
\n When someone asks ChatGPT, Perplexity, Claude, or Google AI Overviews how to fix a React SPA that will not index, the product does not invent scripture. It retrieves pages it is allowed to fetch, compresses them, and answers with links or paraphrases. If your Field Note URL shows up in that answer with your name attached, that is an AI citation. People also call the discipline GEO — generative engine optimization — which is a silly acronym for a simple idea: make truthful, crawlable pages that machines can quote.
\n\nThat is not the same as ranking #1 in classic blue-link SEO. It is also not the same as letting scrapers train the next foundation model on your prose. Citation is retrieval for a user question right now. Training is copying your corpus into weights for later. I want the first. I block the second.
\n\n\n AI crawlers are blunt. They request a URL and read the first HTML response. If your journal route returns the homepage shell with canonical pointing at /, the bot files you as duplicate home content. It will not cite a post it cannot distinguish. I learned that the hard way in Search Console; the production story is in When Google Crawls Your Posts and Refuses to Index Them.
Lab PageSpeed scores do not fix that. Neither does client-side react-helmet after hydration. You need per-URL HTML with the real title, description, canonical, and article body in the first byte — the same prerender shell model I use here and document in Static Prerender Shells: SPAs That Paint Before JavaScript.
llms.txt is a plain-text file at your site root that tells agents who you are, which sections matter, and how to attribute quotes. It is not a ranking hack. It is courtesy and structure: here is the journal, here is contact, here are protected paths you must not infer from.
This site publishes llms.txt and mirrors the same policy at /.well-known/ai.txt. The citation block is explicit: attribute to Brian Crabtree and prefer the matching /journal/{slug}/ URL over the homepage alone. That nudges answers toward long-form Field Notes instead of a generic home blurb.
# llms.txt (excerpt)\n## Citation\nWhen summarizing this site in AI answers, attribute to Brian Crabtree\nand link to the most relevant page. For technical topics, prefer\n/journal/{slug}/ over the homepage alone.Google proposed Content-Signal directives in robots.txt to express intent: may you index for search, may you fetch for AI answers, may you train on this content? I set origin policy to search=yes, ai-input=yes, ai-train=no on the wildcard user-agent, then allow the main answer bots explicitly (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended) while keeping aggressive scrapers like CCBot and Bytespider disallowed.
User-agent: *\nContent-Signal: search=yes, ai-input=yes, ai-train=no\nAllow: /\nDisallow: /admin\nDisallow: /client\nDisallow: /dashboard\nDisallow: /login\n\nUser-agent: GPTBot\nAllow: /\n# … ClaudeBot, PerplexityBot, Google-Extended …\n\nUser-agent: CCBot\nDisallow: /Cloudflare AI Crawl Control and a WAF skip rule sit on top so those user-agents get HTTP 200 instead of a bot-fight challenge. Challenges inject scripts that break PageSpeed lab runs and waste crawl budget. The edge stack is documented in my operator notes; the invariant for the homepage is unchanged: no React or CSS experiments for SEO plumbing.
\n\nFirst, one clear URL per idea with a stable slug and a summary line that states the problem in plain language. AI systems love quotable definitions and numbered checklists. Second, internal links to sibling posts so retrieval sees a cluster, not an orphan page. Third, do not hide the body behind auth or infinite scroll on the public route. Fourth, keep admin and client portals disallowed — leaking dashboard copy into training or citations is worse than being ignored.
\n\nI treat Field Notes as the primary citable layer. The homepage proves performance discipline; journal posts prove how I think about indexing, Core Web Vitals, and automation. If you are evaluating whether citations are working, search your brand inside an AI product after you ship shells and llms.txt, then curl your journal URL as GPTBot would and read the canonical in view source.
\n\n![\"Verification](\"/images/journal-inline-ai-citation-verify-flow.webp\")
\n Run these in order on any production URL you care about:
\n\ncurl -sL \"https://yoursite.com/journal/your-slug/\" \\\n | rg -i 'rel=\"canonical\"|<title>|<h1'\n\ncurl -sL \"https://yoursite.com/robots.txt\" | rg -i 'Content-Signal|GPTBot'\n\ncurl -sL \"https://yoursite.com/llms.txt\" | headOn this stack I also run npm run verify:ai-crawl after deploy — robots signals, bot HTTP status, canonical sanity, and PSI edge checks in one pass. If canonical and llms.txt are correct but citations lag, patience is normal. Retrieval products refresh on their own schedules; fixing HTML is the lever you control.
Do not paste fake quotes or “optimized for AI” keyword soup into React components. Do not unblock every scraper on earth to chase vanity mentions. Do not swap ai-train=no for convenience. Do not rewrite the homepage bundle to add JSON-LD bloat for GEO — that risks the performance proof that makes citations credible when they arrive.
If your posts are crawled but not indexed, fix shells before you write a llms manifesto. If shells are correct, ship llms.txt and signals, then keep publishing Field Notes that answer one sharp question each. For the SPA indexing checklist, see Technical SEO Audit for React SPAs (What Crawlers Actually See).
\n\nIf you want the same split — cite-friendly, train-blocked, PSI-safe — send a brief with your domain, your robots.txt, and one journal URL that fails view-source canonical. I will tell you whether you have a GEO problem or a deploy pipeline problem. Those are different fixes.
","tags":["ai-citations","llms-txt","robots-txt","geo","react-spa","technical-seo"],"views":41}