Screaming Frog SEO Spider vs WebPageTest in 2026: Full-site crawler vs deep performance diagnostics
Screaming Frog crawls every URL on a site and finds what is structurally broken. WebPageTest tests one page at a time and shows exactly why it is slow. Most technical SEO workflows need both, not one instead of the other.
Screaming Frog SEO Spider
WebPageTest- Screaming Frog crawls an entire site for structural and content issues; WebPageTest tests individual URLs for performance and does not crawl a site at all.
- WebPageTest captures LCP, CLS, INP, and TTFB plus full waterfall and filmstrip detail. Screaming Frog has no built-in Core Web Vitals reporting of its own, though it can pull PageSpeed Insights data via its Google integrations.
- Screaming Frog's paid license is £199 a year with unlimited URLs and no recurring monthly cost. WebPageTest is free for on-demand public testing, with a Pro API starting at $9.89 a month for priority queuing and automation.
- Server log analysis, which maps real Googlebot crawl behaviour, is included in Screaming Frog's standard license. WebPageTest has no log analysis feature; it only tests what happens when a browser requests a page.
- WebPageTest tests real browsers across 30-plus global locations with configurable connection speeds. Screaming Frog runs locally on one machine, so its crawl speed and vantage point are tied to your own hardware and network.
- Neither tool tracks AI search visibility, AI Overviews citations, or LLM brand mentions; both are scoped to traditional crawl and performance diagnostics.
- WebPageTest is open source and can be self-hosted for testing staging or internal environments. Screaming Frog is closed-source commercial software with no self-hosting option, since it already runs locally by design.
Screaming Frog SEO Spider and WebPageTest get compared because they both sit in a technical SEO consultant's toolkit, but they answer different questions. Screaming Frog is a desktop crawler that walks an entire site and surfaces broken links, redirect chains, missing metadata, duplicate content, and hreflang problems, with server log analysis included in the £199 per year license. WebPageTest is a free, open-source performance testing tool that runs a single URL through a real browser at one of 30-plus global locations and hands back a full request waterfall, a filmstrip, and Core Web Vitals detail that Screaming Frog does not attempt to produce on its own. One tells you what is wrong across a thousand pages; the other tells you exactly why one page is slow.
The tools at a glance
Screaming Frog SEO Spider
The industry-standard desktop crawler for technical SEO audits.
Screaming Frog SEO Spider crawls a site the way Googlebot does: following links, checking status codes, and cataloguing every title tag, canonical, and redirect it finds along the way. It runs locally rather than in the cloud, so the free version stops at 500 URLs and the £199 a year license removes that limit entirely, with JavaScript rendering via Chromium and custom XPath, CSS, or regex extraction included in the same price.
The feature that separates it from most crawlers at this price is server log analysis, included in the standard license rather than sold as an add-on. Upload Apache, Nginx, or IIS logs and the Spider maps which URLs Googlebot is actually visiting against your site structure, which is a different and often more useful question than what your crawler thinks the site looks like.
What it will not do is tell you why a page is slow beyond a surface-level PageSpeed Insights score pulled through its Google integrations. There is no waterfall, no filmstrip, and no multi-location testing. For crawl-scale structural auditing it has few real peers at the price; for diagnosing a specific slow page, it is the wrong tool.
| Feature | Free Free (500 URL cap) | Single License £199/year | 5-9 Licenses £189/license/year | 10-19 Licenses £179/license/year | 20+ Licenses £169/license/year |
|---|---|---|---|---|---|
| URL crawl limit | 500 | Unlimited | Unlimited | Unlimited | Unlimited |
| Server log analysis | No | Yes | Yes | Yes | Yes |
| JavaScript rendering | No | Yes | Yes | Yes | Yes |
| Custom extraction (XPath/CSS/regex) | No | Yes | Yes | Yes | Yes |
| Google Analytics / Search Console integration | No | Yes | Yes | Yes | Yes |
WebPageTest
The open-source gold standard for deep web performance diagnostics, trusted by engineers at Google, Mozilla, and every serious web team.
WebPageTest runs a single URL through a real browser instance, not a headless emulation, at one of over 30 global locations, and returns a full waterfall of every request the page made, complete with DNS, connection, and TTFB timing for each one. The filmstrip view shows frame by frame what a visitor actually sees as the page loads, which is a level of detail no crawler produces because crawlers are not built to render and time a single page this precisely.
It captures LCP, CLS, INP, and TTFB alongside the lower-level metrics that explain them, and every run can include a full Lighthouse audit for SEO and accessibility findings in the same report. The No-Code Experiments feature lets you test the effect of removing a third-party script or switching to a self-hosted font without touching a codebase, which is useful for building a case before engineering time gets spent.
The trade-off is scope: WebPageTest tests pages you point it at, one at a time, and does nothing resembling a site crawl. The public queue is genuinely free but can be slow during peak hours, and continuous monitoring or API automation requires the Pro tier starting at $9.89 a month.
| Feature | Free Free | Pro API (Starter) $9.89/month |
|---|---|---|
| On-demand tests | Shared queue | Priority access |
| Global test locations | 30+ | 30+ |
| Filmstrip and video replay | Yes | Yes |
| Lighthouse integration | Yes | Yes |
| API access | No | Yes |
| Continuous monitoring | No | Yes |
Head-to-head feature comparison
| Feature | Screaming Frog SEO Spider | WebPageTest |
|---|---|---|
| Primary use case | Full-site technical SEO crawl and audit | Single-page deep performance diagnostics |
| Full-site crawling | Yes | No |
| Server log analysis | Yes, included | No |
| Core Web Vitals / performance metrics | Via PageSpeed Insights integration only | Yes (LCP, CLS, INP, TTFB, and more) |
| Request waterfall / filmstrip | No | Yes |
| JavaScript rendering | Yes | Yes (real browser rendering) |
| Structured data / hreflang validation | Yes | No (Lighthouse SEO/accessibility checks only) |
| Custom data extraction | Yes (XPath, CSS, regex) | No |
| Google Analytics / Search Console integration | Yes | No |
| Global test locations | No (runs on your own machine) | Yes (30+ locations) |
| API access | No dedicated public API | Yes (Pro API) |
| Deployment model | Desktop app (Windows, macOS, Linux) | Public cloud instance, self-hostable |
| Free tier | Yes (500 URL cap) | Yes (unlimited public queue) |
| Starting price | Free / £199/yr | Free / $9.89/mo |
Which should you choose?
Screaming Frog SEO SpiderWebPageTestScreaming Frog SEO SpiderWebPageTestScreaming Frog SEO SpiderWebPageTestScreaming Frog SEO SpiderThese are not competing purchases, they are sequential ones. A typical technical SEO workflow starts with Screaming Frog to find every structural problem across the site, then moves to WebPageTest on the specific pages that turn out to be slow, since Screaming Frog was never built to explain why a page underperforms and WebPageTest was never built to check a thousand URLs for broken canonicals. Buying one instead of the other only makes sense if the job in front of you is genuinely one-dimensional.
Bottom line
Get the Screaming Frog license first if you are starting from zero, £199 a year for unlimited crawling and included log analysis covers more of the day-to-day technical SEO workload than anything else at that price. Add WebPageTest, which costs nothing to start, the moment a Core Web Vitals problem or a specific slow page shows up in that crawl. Only upgrade to the WebPageTest Pro API once you need scheduled monitoring or you are automating tests across a client portfolio; most agencies get by on the free public queue far longer than they expect.
Frequently asked questions
Should I use Screaming Frog or WebPageTest for a Core Web Vitals audit?
Use WebPageTest for the actual Core Web Vitals measurement, since it captures LCP, CLS, INP, and TTFB directly with full waterfall detail explaining the cause. Screaming Frog has no native Core Web Vitals reporting of its own and only surfaces performance data through its PageSpeed Insights integration, which is a much shallower score than what WebPageTest produces.
Can Screaming Frog replace WebPageTest for diagnosing slow pages?
No. Screaming Frog is a site-wide crawler with no request waterfall, filmstrip, or multi-location testing, so it cannot show you why a specific page is slow beyond a basic PageSpeed Insights score. For genuine performance diagnosis, waterfall analysis and filmstrip playback are what WebPageTest is built for and Screaming Frog is not.
Is WebPageTest a substitute for a full technical SEO crawl?
No, WebPageTest tests one URL at a time and has no crawling function at all, so it cannot find broken links, duplicate titles, or redirect chains across a site. Screaming Frog is the tool for full-site structural auditing; WebPageTest is a companion for the performance layer once specific pages need deeper diagnosis.
Which tool is cheaper for a freelancer on a tight budget?
WebPageTest is free to use on the public instance with no account required, which makes it the cheaper starting point for occasional performance checks. Screaming Frog's free tier caps out at 500 URLs, so most freelancers doing real audit work will need the £199 a year license fairly quickly, though that is still a one-time annual cost rather than a recurring monthly one.
Does either Screaming Frog or WebPageTest track AI Overviews or ChatGPT citations?
No. Both tools are scoped to traditional crawl and performance diagnostics, Screaming Frog for site structure and content, WebPageTest for page speed and rendering. Neither has any AI search visibility, AI Overviews tracking, or LLM citation monitoring feature, so a separate AI visibility tool is needed if that data matters for your reporting.
Can I run server log analysis in WebPageTest the way I can in Screaming Frog?
No, WebPageTest has no server log analysis capability at all; it only measures what happens when a browser requests a page in real time. Server log analysis, which shows actual Googlebot crawl behaviour over time, is a Screaming Frog feature included in its standard £199 a year license.