Building an open-source Semrush on Cloudflare Workers is technically possible but the hard part is data, not infrastructure. Cloudflare handles compute and hosting cheaply. The bottleneck is getting reliable SERP data, backlink data, and keyword volume at a cost that makes the project viable. Here is what actually works and what does not.
What Cloudflare handles well
- Workers for API endpoints and cron triggers (free tier generous)
- D1 or Turso for SQLite-compatible keyword/rank storage
- R2 for storing historical SERP snapshots
- Pages for the dashboard frontend
The data problem
Semrush's moat is not software -- it is a decade of crawl data, backlink indexes, and keyword volume estimates. You cannot replicate that with a weekend project. What you can build: a rank tracker with fresh SERP data, keyword gap analysis, and AI Overview monitoring.
Realistic architecture
# cloudflare_seo_worker.py
# Daily rank tracking with SERP API + Cloudflare D1
import requests
def track_keywords(keywords: list, domain: str) -> list:
"""Track keyword rankings for a domain."""
results = []
for kw in keywords:
resp = requests.post(
"https://api.scavio.dev/api/v1/search",
headers={"x-api-key": "YOUR_KEY"},
json={
"query": kw,
"num_results": 20,
"include_ai_overview": True
}
)
data = resp.json()
rank = None
for r in data.get("organic_results", []):
if domain in r.get("url", ""):
rank = r["position"]
break
ai_cited = False
for c in data.get("ai_overview", {}).get("citations", []):
if domain in c.get("url", ""):
ai_cited = True
results.append({
"keyword": kw,
"rank": rank,
"ai_cited": ai_cited,
"date": "2026-05-21"
})
return results
# Run daily via Cloudflare Cron Trigger
keywords = ["search api comparison", "serp api pricing"]
report = track_keywords(keywords, "yourdomain.com")What you get vs Semrush
Feature | DIY on Cloudflare | Semrush ($119/mo)
Rank tracking | Yes (fresh data) | Yes (delayed)
AI Overview track | Yes | No
Keyword volume | No (need 3rd party)| Yes
Backlink data | No | Yes (huge index)
Competitor gaps | Partial | Yes
Site audit | No | Yes
Cost (100 kw) | ~$15/mo | $119/mo// Cloudflare Worker: daily rank check cron
export default {
async scheduled(event, env) {
const keywords = ["best seo api", "serp api pricing 2026"];
const domain = "mydomain.com";
for (const kw of keywords) {
const resp = await fetch("https://api.scavio.dev/api/v1/search", {
method: "POST",
headers: {
"x-api-key": env.SCAVIO_KEY,
"Content-Type": "application/json"
},
body: JSON.stringify({
query: kw,
num_results: 20,
include_ai_overview: true
})
});
const data = await resp.json();
const rank = data.organic_results?.findIndex(
r => r.url.includes(domain)
) + 1 || null;
// Store in D1
await env.DB.prepare(
"INSERT INTO rankings (keyword, rank, date) VALUES (?, ?, ?)"
).bind(kw, rank, new Date().toISOString().slice(0, 10)).run();
}
}
};Honest assessment
You can build a useful rank tracker and AI visibility monitor on Cloudflare for $15-50/month. You cannot build a Semrush replacement. The backlink index and keyword volume database require infrastructure that costs millions to build. Focus on what SERP APIs give you that Semrush does not: real-time AI Overview tracking and multi-platform search data.