Next.js vs Astro vs Webflow for a marketing site AI search can read

By Roshan, founder of SearchMaxx - Sep 27, 2026 - updated Oct 18, 2026 - 7 min read

All three can produce a marketing site that ChatGPT, Claude, Perplexity and Google can read, because all three can send finished HTML in the first response. Astro does it by default with the least JavaScript. Next.js does it by default too, but a few common patterns quietly send crawlers less than you think. Webflow sends real HTML without any code, and is the best fit when marketers, not developers, edit the site.

The reason this matters: Vercel's study of AI crawlers found that OpenAI's crawlers, Anthropic's ClaudeBot and PerplexityBot do not run JavaScript. Google's crawling does. Whatever your framework sends before JavaScript runs is all those AI crawlers will ever see.

Side by side

Feature	Next.js	Astro	Webflow
Rendering model	React. Pages are Server Components by default and are rendered to HTML on the server or at build time	Renders pages to static HTML and strips client-side JavaScript by default	Visual builder that publishes pages as HTML and CSS on Webflow's hosting
What crawlers get without JavaScript	Full text, if content is rendered on the server. Empty spots wherever content is fetched in the browser	Full text. JavaScript only loads for components you mark as interactive	Full text for pages built in the designer. Script-based embeds may arrive empty
Titles and descriptions	Built-in `metadata` object and `generateMetadata` function	You write the `<head>` tags in a layout	Per-page fields in page settings
Sitemap	`app/sitemap.ts` file convention	Official `@astrojs/sitemap` integration	Generated automatically
robots.txt	`app/robots.ts` file convention	A static file you add	Edited in site settings, on paid plans
Structured data (JSON-LD)	A `<script type="application/ld+json">` tag in the page or layout	Same, in the layout	A schema markup field in page settings, or custom code
JavaScript sent to visitors	React runtime plus your interactive components	None by default, then only the islands you add	Depends on interactions and embeds you add
Who edits content	Developers, or marketers through a headless CMS you connect	Developers, or marketers through a headless CMS you connect	Marketers, in Webflow's visual editor
Cost	Free and open source. You pay for hosting	Free and open source. You pay for hosting	From $15 a month per site, billed yearly, plus seats

Next.js

Rendering. The Next.js docs on Server and Client Components say that "by default, layouts and pages are Server Components." Even Client Components are used to prerender HTML on the server. On first load, "HTML is used to immediately show a fast non-interactive preview," and JavaScript then makes it interactive. So a default Next.js App Router page sends crawlers real text.

Where it goes wrong. Three patterns send AI crawlers less than a browser sees:

Content fetched in the browser. A component that loads pricing or testimonials in a useEffect renders an empty box on the server. The text appears only after JavaScript runs.
Large client-only sections. Next.js says that once a file is marked "use client", all of its imports go into the client bundle. That does not hide the text, but it adds JavaScript every visitor downloads.
Streaming metadata on dynamic pages. For dynamically rendered pages, the metadata docs say Next.js streams metadata and injects it once generateMetadata resolves, instead of placing it in the <head> up front. Streaming is turned off for a list of bots that expect metadata in the head. As of today, the default list in the Next.js source includes Bingbot, Applebot, several Google crawlers and social preview bots. It does not include GPTBot, OAI-SearchBot, ClaudeBot or PerplexityBot. Prerendered pages are not affected, because their metadata resolves at build time.

If your marketing pages render dynamically, the htmlLimitedBots option lets you add those bots, or set it to /.*/ to turn streaming metadata off entirely. Note that setting it replaces the default list, so include the bots already on it.

Metadata, sitemap and schema. Next.js has the most built in. A sitemap.ts file generates sitemap.xml from code, and a robots.ts file does the same for robots.txt. For structured data, the JSON-LD guide recommends a plain <script> tag in layout.js or page.js, with < escaped to avoid injection.

Astro

Rendering. Astro's islands docs say it renders "the majority of your page to fast, static HTML" and works by "stripping out all client-side JavaScript automatically." To make a component interactive you add a client:* directive, and only that component ships JavaScript. Directives like client:visible delay it until the component scrolls into view.

What crawlers get. The whole page as text. Of the two frameworks, Astro is the harder one to accidentally break for AI crawlers, because JavaScript is something you opt into per component rather than something you have to remove.

Metadata, sitemap and schema. Less is built in than in Next.js. You write title, description and JSON-LD tags in a layout component yourself. The official @astrojs/sitemap integration installs with npx astro add sitemap, needs your site URL in the config, and writes sitemap-index.xml and sitemap-0.xml at build time.

Ownership. Cloudflare announced on January 16, 2026 that it was acquiring the company behind Astro. The release says "Astro will remain open source," and Astro's CEO says it will stay the best way to build content sites "whether they host on Cloudflare or elsewhere."

Webflow

Rendering. Webflow publishes the pages you design as HTML and CSS, and its paid Workspace plans can export that code. Text you place in the designer is in the HTML a crawler downloads.

Where it goes wrong. Anything added through an embed or a third-party script, such as review widgets, pricing calculators or FAQ tools, may load with JavaScript and reach AI crawlers as an empty box. Check those sections without JavaScript.

Metadata, sitemap and schema. The pricing page lists custom titles and meta descriptions per page, canonical tags, an auto-generated XML sitemap, robots.txt and llms.txt controls, crawler access controls, and schema markup support. Webflow's help center says the sitemap is generated by default, and that page settings include a schema markup field where you can paste your own JSON-LD. On the free Starter plan, the sitemap, robots.txt and schema features need a paid Workspace plan.

Who edits. This is Webflow's main advantage. Marketers can change copy, publish CMS posts and build pages without a developer or a deploy.

Cost

Next.js and Astro are free and open source. Next.js is MIT licensed. Your costs are hosting and developer time. Hosts such as Vercel, Netlify and Cloudflare all run them. Vercel's Pro plan is listed at $20 a month.

Webflow charges per site and per seat. As of October 2026, its pricing page lists:

Webflow plan	Price	What you get
Starter	Free	webflow.io domain, 2 static pages, 50 CMS items
Basic	$15 a month, billed yearly	Custom domain, 300 static pages, no CMS
Premium	$25 a month, billed yearly	Everything in Basic, plus the CMS (up to 20,000 items) and 50 GB bandwidth
Team	$2,500 a month, annual contract	Everything in Premium, plus localization, publishing workflows and 5 full and 5 limited seats
Extra full seat	$39 a month per seat, billed yearly	Design pages and manage settings
Extra limited seat	$15 a month per seat, billed yearly	Edit content and build pages from components

Prices are in US dollars, per site, before tax. A small marketing team with a blog, one designer and two editors would pay for a Premium site plus seats, so check the current page before you budget.

Speed

We have not benchmarked these three ourselves, so here is only what follows from how they work. Astro sends no JavaScript unless you add it, so a plain Astro page is as light as its HTML, CSS and images. Next.js sends the React runtime and the code for every Client Component, which it says Server Components help reduce. Webflow serves sites from a global CDN, and page weight depends on the interactions, fonts and embeds you add. For crawlers, the bigger risk is not speed but missing text.

Verdict by team type

Your team	Pick	Why
Marketers own the site, no developer time	Webflow	Real HTML without code, built-in sitemap and schema fields, and editors can publish alone. Audit embeds for empty boxes.
Developers own the site, mostly content pages and a blog	Astro	Static HTML and zero JavaScript by default. Hardest to break for AI crawlers.
Your product is already in Next.js and you want one codebase	Next.js	Best built-in metadata, sitemap and robots tools. Prerender marketing pages, render content on the server, and add AI bots to `htmlLimitedBots` if any page renders dynamically.
Marketers edit, developers build	Astro or Next.js with a headless CMS, or Webflow	Pick based on who you can spare. A framework plus a CMS costs developer time. Webflow costs money per seat.
Moving off a single-page React app	Next.js or Astro	Both fix the empty-HTML problem. See migrating a React SPA to Next.js.

Whichever you choose, test the result the way an AI crawler sees it. AI crawlers don't run JavaScript has a one-line curl command that counts the words your server sends, and our free check runs the same test plus crawler access in about 10 seconds. For how each AI engine then finds and cites those pages, see ChatGPT vs Perplexity vs Google AI Overviews.

Want this done for your site?

Every day we make one change to your website that helps AI engines name you, and a person checks each one. Email me your website and the question your buyers ask AI.

Email Roshan to get started