Autopulto
Google Search Console told us 61 pages on autopulto.com were not indexed. At the time, the site had about 26 URLs that should be indexed at all. So the number was not a coverage gap — it was more than double the entire site, made of addresses we did not know we were publishing.
Not one of them was a content problem. Here is the whole breakdown, and what it actually turned out to be.
What the 61 were made of
| Reason | Pages |
|---|---|
| Page with redirect | 19 |
| Duplicate without user-selected canonical | 15 |
| Excluded by "noindex" tag | 11 |
| Alternate page with proper canonical tag | 9 |
| Other 4xx | 4 |
| 401 | 2 |
| Crawled — currently not indexed | 1 |
Read that list as a whole and the diagnosis writes itself. Nothing there says "thin content" or "low quality". Every line says the same thing in a different dialect: we asked Google to index an address, and something else at that address disagreed about which URL the page really lives at.
That is a URL identity problem. The site was serving the same page at several addresses and never saying which one was real.
If you run a product site and you have a scary not-indexed number, check this before you rewrite a single paragraph. Content problems show up as "Crawled — currently not indexed" and "Discovered — currently not indexed". We had exactly one of those.
The canonical that pointed at a page nobody linked to
The worst one was ours, in our own code, and it had been shipping quietly for months.
Three different places in the codebase compute the URL prefix for a blog. Two of them handled the platform blog correctly and returned /blog. The third always returned /b/<slug> — the internal, multi-tenant form of the same address.
That third function was the one that fed the value we bake into the database at publish time. So every post published at autopulto.com/blog/<slug> carried a stored canonical pointing at autopulto.com/b/autopulto/<slug>. Meanwhile the sitemap submitted the /blog/ form.
We were submitting one address and telling Google the real page was a different one. Google did exactly what it was told: it filed the submitted URLs as "Alternate page with proper canonical tag". That is the 9.
The blog index did the reverse — /b/autopulto/ canonicalised to /blog/ — so the two halves of the blog pointed at each other and neither was clearly the original.
The part worth your attention is why it shipped. There was a test for this. It asserts that posts must never regress to the /b/<slug> form. It passed the whole time. Its fixture created posts with the stored canonical left unset, so the code fell through to a fallback that computes the correct value, and the stored-column path — the one that actually runs in production — was never executed once.
A green test on a code path your fixture never reaches is worse than no test, because it buys you confidence you did not earn. If you write a test that asserts a value is never wrong, seed it with the wrong value first and watch it fail.
Fixing the function was three lines. Fixing the data took a migration to null out every poisoned canonical we had already written.
The other three addresses
www and apex both answered. Both served the site, 200, no redirect. Two complete copies of every page. Now www 301s to the apex.
Trailing slashes went to the login page. /pricing/ did not serve pricing and did not redirect to /pricing. It redirected to /login. Every marketing page had a slashed twin that led somewhere else entirely — which is a good share of those 19 "Page with redirect" entries, submitted by anyone who had ever linked to us with a trailing slash. Now it is a 301 to the unslashed URL, with the query string preserved, because our blog attribution rides in a query parameter and a redirect that drops it destroys the data.
No page said what it was. No marketing or legal page carried a self-referencing canonical tag. That tag is how a page settles the argument about its own identity before anything else can start one.
The fix we deliberately did not make
Eleven pages were excluded by a noindex tag: login, register, password reset, email verification. The obvious move is to add them to robots.txt as well — belt and braces.
That would have made things worse. Those pages render a noindex tag, and Google has to crawl a page to see it. Disallowing them in robots.txt stops the crawl, which means the tag never gets read, which risks the genuinely bad state: "Indexed, though blocked by robots.txt" — a page in the index that you have no way to remove because you have blocked the mechanism that would tell Google to drop it.
Disallow is for routes that render no content of their own. noindex is for pages that render but should not rank. Those 11 were already correct. The right action was to leave them alone and stop counting them as a problem.
The bug that only appeared once we fixed the bug
Adding the trailing-slash redirect introduced a redirect loop on one class of URL, and the cause is a good one to know about.
Request /pricing%2F. The server decodes %2F to / in the path it hands your code, but leaves it encoded in the raw path. Our gate checked the decoded path — which ends in a slash, so it fired — and built the redirect target from the raw path, which has no trailing slash to strip. The result was a 301 whose Location was byte-for-byte the URL that had just been requested. Infinite loop, in a browser: ERR_TOO_MANY_REDIRECTS.
The fix is to require both forms to end in a slash before redirecting. The general rule: if you make a routing decision on a decoded value and construct the response from a raw one, you have two different ideas of the same URL in one function, and they will disagree eventually.
A second one from the same week: the redirect built its Location from the request scheme. Behind a tunnel the app sees plain HTTP, so it was issuing http:// redirects on an HTTPS site — a bonus hop, and an insecure one, on every canonical redirect we had just added. Build redirect targets from your configured public base URL, not from what the request claims to be.
What it looks like now
Every one of these is live and checkable yourself:
www.autopulto.com→301→autopulto.com/pricing/→301→/pricing, and the same for every marketing page- every marketing page carries a self-referencing canonical
- every blog post canonicalises to its own
/blog/URL - the internal
/b/autopulto/<slug>address still answers, and now points at the canonical one http://→301→https://
One imperfection is left, and we know about it because our own check reports it: http://www.autopulto.com takes two hops to reach the canonical page — first to https://www, then to the apex. It is a low-severity finding and it stays visible until we collapse it.
The part that actually matters
Every bug above was a one-time fix. None of them would have stayed fixed.
URL identity breaks the way it broke the first time: not from a decision, but from a deploy. A framework's default redirect behaviour changes. A new page ships without a canonical. A CDN setting adds a hop. Someone adds a helper that computes a URL prefix and gets the fourth copy of that logic subtly wrong. Nobody sees any of it, because nothing is broken from a browser — the page loads fine. It only shows up in Search Console, weeks later, as a number.
So the day after this pass, we shipped a daily check that looks at the site the way a crawler does. It compares what the sitemap claims against what the URLs actually answer: redirect chains, canonical disagreements, noindex where it does not belong, status codes that do not match the plan. Twenty-three kinds of finding.
We held it to one release gate before it went anywhere near a customer's site. Replay the recorded responses of autopulto.com from before this pass and the check has to produce all seven of the problems we found by hand. Replay the site as deployed after the fixes and it has to produce no high or medium findings and exactly one low — the http://www hop. Both passed. The first live run against the real site matched the replay exactly, and as of today the check holds exactly one open finding on autopulto.com — that http://www hop, severity low.
The other half of the gate was the quiet one: eleven consecutive simulated daily runs, to prove that a problem you have already been told about never emails you again. A monitor that re-reports a standing issue every morning is a monitor you will mute inside a week, and then it is worth nothing. You get told when something changes.
What we do not know yet
Whether it worked.
The fixes went to production on 16 September. This is being published two days later. Google recrawls on its own schedule, and a 61-page identity mess takes weeks to unwind, not days. We are not going to tell you the number moved when we cannot see it move.
We will publish the follow-up with the real numbers, whatever they say — including the case where the not-indexed count barely drops, which would mean the diagnosis was wrong and worth writing up just as much.
If you want to run the same diagnosis on your own site: open Search Console, go to Pages, and read the exclusion reasons as a group rather than one at a time. If yours are mostly redirect, duplicate, and alternate-canonical entries, you do not have a content problem either. You have several addresses arguing about which one is the page.