ThreadPilot
Solution Seeking
r/WebScrapingInsiderPosted Nov 4, 2026

get ALL posts and comments matching a keyword?

by u/Feeling-Plane3434 · 6 upvotes · 5 comments

The post

As far as I know, there are two main ways to search on Reddit. The first is the site-wide search from the Reddit homepage. For each query/sort combination it only lists up to 250 results (about 490 on the newer search endpoint). For comments it doesn't even reach 490 — it stops at 250. There are about 25 possible sort/time combinations, so by collecting the list for each combination I've confirmed (by actually testing it) that you can get up to roughly 3,000 results. The second is searching inside a specific subreddit, but it has exactly the same caps. Because of the 250 (490) limit, I had to split the search into many combinations there too. This is really frustrating, because there's probably a lot of great information hidden beyond what these lists return. What I want is to see every post and every comment that contains a given keyword, across Reddit's entire history. Is there really no way to do this within Reddit itself? Are third-party archive services or paid APIs (Reddit's official paid API or third-party ones) the only options? Do I really have to download the full dumps (tens of GB of posts and comments) and search them locally?
View the original thread on Reddit →

Why this is an opportunity

  • ✓Low comment competition

Never miss a thread like this again

ThreadPilot finds Reddit posts where people ask for products like yours, scores them, and drafts a helpful reply.

Try ThreadPilot free