{"id":"g4fx27rrgtkxc4a","title":"When Google Thinks Every URL Is Your Homepage","slug":"react-spa-crawled-not-indexed-fix","summary":"A broken deployment flag stopped generating static HTML shells for journal posts, causing every URL to serve a generic homepage fallback that Google…","imageUrl":"https://briancrabtree.me/images/journal-react-spa-crawled-not-indexed-fix.webp","category":"Engineering","date":"2026-06-03T18:00:00.000Z","featured":false,"likes":17,"author":"Brian Crabtree","content":"<h2>The symptom in Search Console</h2>\n\n<p>Search Console told me twelve pages were indexed and sixty-five were not. That ratio sat unchanged for weeks while I fixed things I thought mattered — sitemap resubmits, trailing-slash canonicals, nginx redirects for retired slugs. The Page indexing report still showed a last update date from late May. Google was not lying. It was just slow. But the reasons bucket was honest: crawled – currently not indexed, duplicate without user-selected canonical, and a fat block of not found (404) URLs that mostly already 301 on the live server.</p>\n\n<p>What frustrated me was not the lag. It was that the URLs I cared about — journal posts with real search intent — looked healthy in a normal browser session. Title correct. Content there. Share preview fine. URL Inspection sometimes showed a stale 404 from an old crawl while the live test passed. That split is the whole story: Google indexes the first HTML response, not the React tree you see after hydration.</p>\n\n<figure>\n  <img src=\"/images/journal-react-spa-crawl-before-after.webp\" alt=\"Before and after: SPA fallback serves homepage canonical; prerender shell serves post title and article HTML\" width=\"1200\" height=\"675\" loading=\"lazy\" />\n  <figcaption>Google indexes what the first HTML response says — not what React paints a second later.</figcaption>\n</figure>\n\n<h2>What Chrome lied about</h2>\n\n<p>Open DevTools on a React SPA route and everything looks intentional. The router swapped components. Meta tags updated client-side. Canonical link points at the journal slug. Disable JavaScript and the picture changes — or skip JS entirely and curl the URL. That is the test I should have run daily during deploy week.</p>\n\n<p>When journal prerender shells were missing from production, nginx fell through to the SPA fallback. Every post URL returned the same <code>index.html</code> as the homepage. View source showed <code>&lt;link rel=\"canonical\" href=\"https://briancrabtree.me/\"&gt;</code> on paths that should have been unique articles. Google crawled them, saw duplicates of home, and filed them under crawled – not indexed or duplicate canonical. The hydrated React app corrected the head after load. Crawlers that execute JS might eventually notice. Many do not wait that long.</p>\n\n<pre><code>curl -sL \"https://yoursite.com/journal/my-post/\" \\\n  | rg -i 'rel=\"canonical\"|property=\"og:url\"|&lt;title&gt;'</code></pre>\n\n<p>Run that on every template you ship. If the canonical is wrong before the bundle runs, your SEO stack is wrong regardless of Lighthouse scores.</p>\n\n<h2>The deploy mistake</h2>\n\n<p>The regression was boring. Production deploys used <code>--no-build</code> to save time. That path skipped the step that clones <code>dist/index.html</code> per journal slug and injects post-specific title, description, canonical, and Open Graph tags. The webroot still had the React bundle. It did not have one hundred two slug directories under <code>/journal/</code>. Google kept recrawling URLs from the sitemap and getting homepage HTML.</p>\n\n<p>Full deploy restored shells. A guard in deploy now fails prod <code>--no-build</code> when the journal slug count drops below fifty. That is not elegant architecture poetry. It is a seatbelt. I would rather fail a deploy than silently teach Google that every Field Note is the homepage.</p>\n\n<pre><code>&lt;!-- Wrong: SPA fallback — every route looks like home --&gt;\n&lt;link rel=\"canonical\" href=\"https://briancrabtree.me/\" /&gt;\n\n&lt;!-- Right: per-post shell from injectJournalMetaCli --&gt;\n&lt;link rel=\"canonical\" href=\"https://briancrabtree.me/journal/my-post/\" /&gt;\n&lt;title&gt;My Post Title | Brian Crabtree&lt;/title&gt;</code></pre>\n\n<pre><code># deploy.sh — fail prod --no-build if journal shells missing\njournal_count=$(find \"$WEBROOT/journal\" -mindepth 1 -maxdepth 1 -type d | wc -l)\n[ \"$journal_count\" -ge 50 ] || exit 1</code></pre>\n\n<h2>Three checks before blaming Google</h2>\n\n<p>First: curl canonical and title on the exact trailing-slash URL in the sitemap. Second: URL Inspection live test on the URL-prefix property — not the domain property if you only care about the main site KPI. Third: confirm sitemap discovered count matches the number of live journal slugs plus static routes. When mine jumped from forty-six to one hundred nine, I knew the feed was finally honest even though indexed count did not move yet.</p>\n\n<figure>\n  <img src=\"/images/journal-gsc-url-inspection-flow.webp\" alt=\"Editorial illustration of Google Search Console URL Inspection: paste URL, test live URL, request indexing\" width=\"1200\" height=\"675\" loading=\"lazy\" />\n  <figcaption>Live test confirms what Googlebot would fetch today — not what the index tab remembered from February.</figcaption>\n</figure>\n\n<p>Request indexing helps on priority URLs. Quota is real — roughly ten to fifteen a day. I batch hub pages and high-intent posts first, skip URLs already indexed, and stop when the dialog says quota exceeded. Indexing requests do not replace correct HTML. They nudge recrawl after you fix the crawl signal.</p>\n\n<h2>The fix without rewriting the app</h2>\n\n<p>I did not add JSON-LD widgets or marketing tags to React for this. The fix was infrastructure: full build, per-slug shells, trailing-slash nginx, legacy slug 301s in server config only, IndexNow ping on deploy. Article body HTML already lives in content files; React enhances navigation and likes. That matches the prerender model I wrote about in <a href=\"/journal/static-prerender-shells-spa/\">Static Prerender Shells: SPAs That Paint Before JavaScript</a> — real copy and meta in the first byte, hydration without hiding the shell.</p>\n\n<p>For a deeper audit checklist — render budget, hydration mismatches, dynamic meta — read <a href=\"/journal/technical-seo-audit-react-spa/\">Technical SEO Audit for React SPAs (What Crawlers Actually See)</a>. This post is the production war story that checklist is trying to prevent.</p>\n\n<h2>What to do while you wait</h2>\n\n<p>Indexed count will not jump overnight. GSC Page indexing reports lag fixes by days or weeks. Keep daily indexing requests on P0 URLs. Do not merge domain-property not-indexed totals with URL-prefix KPIs — domain includes microsites and legacy hosts. Watch for 404 buckets shrinking as 301s recrawl. Do not touch homepage bundle or CSS chasing SEO; server-side redirects and GSC hygiene are enough.</p>\n\n<p>If your React posts are crawled but stuck, start with view source, not a content rewrite. If the canonical is home on a journal URL, you do not have an indexing mystery. You have a deploy pipeline bug.</p>\n\n<h2>Verify it yourself</h2>\n\n<p>Pick one journal URL. Curl the canonical. Compare to what React shows after load. If they disagree, fix shells before you pay for more content. If you want the same discipline on your stack, <a href=\"/contact?ref=journal\">send a brief</a> with your URL and what Search Console shows — not what you wish it showed.</p>\n","tags":["react","seo","spa","google-search-console"],"views":38}