How to Build a Programmatic SEO Site With Thousands of Pages (2026)

To build a programmatic SEO site with thousands of pages you need four things working together: a structured data source that gives you one row per page, a template that turns a row into a page that is genuinely useful on its own, a writer step that adds real per-page content instead of swapping nouns, and a publishing loop that keeps URLs stable, rebuilds hubs and sitemaps, and tells search engines what changed. Everything else, from hosting to analytics, is secondary to those four.

I know this because I built one. PageNearby is a generator that creates a themed one-page website for small Indian businesses that have a phone number and good reviews but no website of their own. It published more than 2,800 pages within a day of launch and has rewritten over 30,000 since, running unattended on a single PC with a 16 GB GPU. This post is the playbook I wish I had read before starting, including the mistakes that cost me the most time.

What makes a programmatic SEO site worth indexing?

The honest answer is: each page must answer a query that a real person types, better than the nothing that currently exists for it. For PageNearby the query is a business name plus a city, or a category plus an area ("plumber in Patia Bhubaneswar"). If the business has no site, the search result is a map pin and nothing else. A clean page with the hours, services, reviews summary and a way to contact them is an improvement for the searcher, and that is the test I apply before adding a page type.

What does not work is the classic "city name times keyword" grid where every page is the same paragraph with a different noun. Search engines have been demoting that pattern for years, and in 2026 the scaled-content guidance is explicit about it. So the design rule I follow is that every page must contain information that exists only for that entity: its specific services, its rating profile, its neighbourhood, the questions people ask about that category in that area.

How do you get a data source with one row per page?

Programmatic SEO lives or dies on the dataset. Mine comes from scraping public business listings per city and category, then filtering hard. A listing survives only if it has a phone number, a rating above a threshold, a minimum number of reviews, and no website field. That filter does two jobs at once: it keeps the dataset useful (people can actually contact the business) and it keeps me out of competing with businesses that already have a site.

Three lessons from the scraping layer:

• Expect your scraper to break at a layer you do not own. The release binary I used failed on Windows and the cause was four dependency layers down, where a closed-page check was inverted and the browser crashed in single-process mode. The fix was a patched fork and a rebuilt binary. Budget time for this.
• Treat memory as a shared resource. Scrapers running next to a local language model will fight over RAM. My watchdog pauses scrapers under memory pressure and resumes them later.
• Store raw listings separately from the rows you publish. You will want to re-filter later without re-scraping.

How do you write unique content for thousands of pages without an API bill?

This is where most programmatic SEO projects either go bankrupt on API calls or ship thin pages. My approach was a local mixture-of-experts model (35 billion parameters with about 3 billion active, which fits a 16 GB consumer GPU) producing a schema-constrained JSON brief for each business. The brief contains a headline, a short description, a services list, a neighbourhood note, and a set of FAQs specific to that category.

The schema constraint is the important part. I do not ask the model for a page; I ask it for structured fields that a validator can reject. A brief that fails validation is regenerated, not published. That keeps the output consistent enough to template and stops the model from inventing opening hours or prices that are not in the source listing.

For throughput I run several writer backends as independent workers: the local model does the bulk, and hosted models pick up overflow when the queue is long. Each worker claims rows from the same queue, so adding capacity is a matter of starting another process. A watchdog restarts any worker that is missing every ten minutes; the system survived a mains power cut and resumed on its own.

How should you structure URLs and templates at scale?

Pick a URL scheme once and never change it. Mine is city, then area, then a business slug, which gives three natural hub levels: a city hub listing areas, an area hub listing businesses, and the business page itself. Hubs are what make the site crawlable; without them you have thousands of orphan pages that only a sitemap knows about.

The templating layer is a plain Jinja renderer with a handful of themes selected by category, so a bakery and a hardware store do not look identical. Every page gets structured data for a local business, canonical tags, and an internal link back to its area and city hubs. Legal pages, a removal-by-email clause and a privacy page are generated as part of the same build, because at this scale you will get removal requests and you need a documented path for them.

One rule I added after an incident: the path registry that maps a business to its URL now raises an error on any sign of corruption instead of silently continuing. A corrupted registry once handed one business's URL to another, which is the worst possible failure for a directory. Fail loudly on identity data.

How do you handle contact details and leads responsibly?

A directory that prints every phone number in plain HTML is a scraper magnet and arguably not fair to the business. PageNearby never puts the phone number in the page source. The page shows a masked number and a lead form; after the visitor enters an email and consents, a small server endpoint reveals the number and logs the lead. That gives the business a record of who contacted them and gives me a way to measure whether a page is doing anything useful. It also means the "thousands of pages" claim is backed by a real interaction, not just impressions.

I deliberately held back ad placement on business pages until I had reviewed the site against scaled-content guidelines. Programmatic sites that monetise before they prove usefulness tend to get caught in exactly the review they were trying to avoid.

How do you get thousands of pages indexed quickly?

Three mechanics, in order of impact:

1. Hubs and internal links, so a crawler can discover pages by following links, not just by reading a sitemap.
2. Sitemaps split by city, regenerated on every build, with accurate last-modified dates.
3. IndexNow pings for changed URLs, sent by the same packer that uploads files to hosting. The packer only ships files whose content hash changed, so a 30,000-page site uploads in minutes, not hours.

Verification in Search Console happened on day one and the site was receiving organic impressions within the first week. I track indexed pages per city to spot areas where the template or the data is weak.

How I built this: PageNearby, the autonomous business-website generator

PageNearby is the project behind everything above. The workflow it runs, unattended, is:

1. A scraper harvests business listings per city and category and writes raw rows to a local store.
2. A filter keeps businesses with a phone, a rating, reviews and no website, and assigns each a stable URL in the registry.
3. Writer workers (a local Qwen mixture-of-experts model first, hosted models as overflow) produce a schema-validated JSON brief per business; invalid briefs are regenerated.
4. The Jinja renderer builds the themed page, then rebuilds the affected area and city hubs and the sitemaps.
5. A packer diffs the output against the last upload, ships only changed files to hosting and pings IndexNow.
6. On the live page, a consent-gated lead form reveals the masked phone number and logs the lead.

Design decisions that mattered: structured briefs over free text, a registry that fails loudly, a watchdog instead of a long-running monolith, and keeping phone numbers out of HTML. The outcome is a live, indexed site with thousands of pages, city hubs, legal pages and a working lead path, built and maintained by one machine. The rank and crawl monitoring runs on my self-hosted SEO stack, which is a separate project; if you are weighing a build like this against paid tooling, my services page explains how I scope it.

FAQ

How many pages can one PC generate per day?

With a local 16 GB GPU model doing most of the writing and a few hosted workers for overflow, I sustained a few thousand pages per day including scraping, rendering and upload. The bottleneck moved between scraping, generation and memory depending on the hour, which is why the watchdog pauses and resumes stages rather than running everything flat out.

Will Google penalise a programmatic SEO site?

Google penalises scaled content that exists only to rank, not scale itself. Large directories, job boards and listing sites are all programmatic. The difference is whether each page carries entity-specific information that helps the searcher. Validate your content against that test before you worry about volume.

Do I need a local language model, or can I use an API?

You can use an API, and I use hosted models as overflow. But at tens of thousands of pages, per-page API cost becomes the project's biggest line item, and a schema-constrained local model produces briefs that are good enough for this use. The local model also keeps running when a provider has an outage.

How do you handle businesses that want their page removed?

Every page links to a removal policy with an email address. Removal requests are processed by deleting the row, which removes the page on the next build and marks the URL as gone in the registry so it is never reassigned.

If you want a programmatic site like this built for your data or your niche, tell me about it on the contact page.