crawling us as hard as ever

On Reddit’s earnings call last week, Steve Huffman said Google is reading the site as much as ever and sending far fewer people to it. His answer is a login wall. The click was never just traffic. It was the return gift that kept the open web open, and you have the same problem inside your own company.

Two sentences from Reddit’s third-quarter call on 29 October describe the end of a deal that ran the web for twenty-five years.

“Google is crawling us as hard as ever, so the demand hasn’t changed from their point of view.”

And, on Google’s AI Mode: “The reality is the AI Mode does not drive traffic. Not for us, and not to anybody as far as I know.”

Read them together. Google wants the content as much as it ever did. It has stopped paying for it in the one currency the web ran on.

the numbers

Reddit isn’t in trouble the way that sounds. Revenue was $847 million for the quarter, up 54%. The trouble is in the visits. In October, Reddit says, search referrals fell by more than a million a week on average. In the US, the logged-out daily audience, the people who arrive from a search and never sign in, dropped from 30.1 million to 27.3 million in one quarter. The logged-in audience barely moved, 23.1 million to 23.4 million. Reddit warned its total daily audience could fall by a low-double-digit percentage in the fourth quarter. The stock fell about 18% after hours.

Huffman called Google’s AI Overviews and AI Mode “the biggest part” of Reddit’s working hypothesis. That’s his word, hypothesis, and these are Reddit’s own numbers. But the pattern is easy to picture. Someone searches “best budget espresso grinder reddit”. The word Reddit is still among the most searched words on Google. The answer comes back as a paragraph built from Reddit threads, and the reader never visits one.

His response is to close the door. Starting this quarter Reddit will test a logged-in-first experience in some markets outside the US, to find out how much a search visitor can see before a hard login prompt. Huffman was blunt: “We are going to make this transition.” And: “At this point, we’re the only platform that doesn’t do this.”

the gift

The sociologist Marcel Mauss wrote The Gift in 1925 about exchange before markets, drawing on accounts from Polynesia, Melanesia and the Pacific Northwest. His finding was that nothing in those economies was free. A gift carried three obligations: to give, to receive, and to give back, and the third was the one that mattered most. The return gift was what turned an exchange into a relationship. Failing to give back cost you standing. Refusing the cycle altogether was, in Mauss’s reading, close to a declaration of hostility.

The open web was this kind of economy, and the return gift was the click. Sites let crawlers in for free. They gave away the content, robots.txt open, no login, no meter. In return the search engine sent readers back. Nobody wrote it down as a contract, because it didn’t need to be one. Both sides were paid every time someone clicked a blue link.

AI Mode keeps the receiving and drops the giving back. Google reads as much as it ever did, answers the question, and sends nobody. In Mauss’s terms that isn’t a better search product. It’s a gift kept without a return, and the predictable answer is the one Reddit announced. The giver stops giving to strangers, and the door that was open becomes a door you need an account to pass.

Licensing doesn’t fix this, though it looks like it should. Google’s data deal with Reddit was reported at $60 million a year in 2024, and Huffman says new talks with Google and OpenAI are ongoing. But a sale is the opposite of a gift. Paying for something settles the debt and ends the relationship. A license pays Reddit for the archive. It does nothing for the anonymous stranger who wrote the grinder answer at 1 a.m. for other strangers, got thanked in a reply, and came back to answer the next one. That person was paid in visits and recognition. That’s the part of the economy that just stopped, and it’s the part that produced the archive in the first place.

So here’s my prediction. The walls go up across the web, and the AI answers slowly start citing a web that stopped being written in 2025. The crawlers will still be crawling hard. There will just be less behind the login prompt that they’re allowed to see, and less written for strangers at all.

you are Reddit

Now take this indoors.

Your company has probably bolted an assistant onto Slack, the wiki, the ticket tracker and the code review history. It crawls everything you’ve written there as hard as ever. The colleague who would have DM’d you, hey, how does the billing retry work?, now asks the assistant. They get a paragraph built from your design doc, your three-year-old Slack answer and your PR comments. They never visit you.

Think about what that visit did for you. It was how reputation got made at work. Go ask Priya, she knows billing. Every one of those was a referral. It’s how people learned your name, how your manager heard you were the one people go to, and how “unblocks others” ended up in your review with examples attached. The assistant keeps the answer and drops the referral. Your work keeps getting used. It just stopped coming back to you.

Reddit’s response, the wall, is the wrong move for you. An engineer who hoards answers to stay needed gets managed out, and should. But the debt is real, so collect it the honest way:

• Sign what you write. Put your name and the date at the top of the design doc. A page with an author gives the summary someone to cite. A page without one gives it nobody.
• Keep your own referral log. When your doc, runbook or answer unblocks someone, write it down with a link. AI Mode won’t send you the traffic report, and neither will the internal tool. Review season needs it.
• Answer in public channels, not DMs. A DM is a gift nobody else sees. The same answer in a channel gets thanked where people can see it, and gets crawled with your name on it.

The crawl is a compliment. Make sure it comes with a return address.

Sources: Search Engine Journal: Reddit’s Q3 audience numbers and login test ↗ · PPC Land: Huffman’s quotes, revenue, after-hours move and licensing deals ↗ · Marcel Mauss, The Gift (Wikipedia summary) ↗