Google Custom Search Engine free tier limits you to 50 specific domains. If your app needs to search the full web, you either pay Google $5 per 1,000 queries or switch providers. This tutorial migrates a 50-domain Google CSE setup to Scavio, which searches the entire web with no domain cap at $0.005 per query. The migration preserves your existing result format so downstream code needs zero changes.
Prerequisites
- Python 3.9+ or Node.js 18+
- Your current Google CSE configuration (CX ID, domain list)
- A Scavio API key from scavio.dev
Walkthrough
Step 1: Export your Google CSE domain list
Extract the domains from your Google CSE configuration. These become optional site filters in Scavio if you want to limit scope.
Python
# Your 50 Google CSE domains (example)
google_cse_domains = [
'docs.python.org', 'stackoverflow.com', 'github.com',
'developer.mozilla.org', 'reactjs.org', 'nextjs.org',
'vuejs.org', 'angular.io', 'nodejs.org', 'npmjs.com',
# ... up to 50 domains
]
print(f'Migrating {len(google_cse_domains)} domains from Google CSE')
print(f'Google CSE: limited to these {len(google_cse_domains)} domains')
print(f'Scavio: searches entire web (no domain cap)')
print(f'Cost comparison:')
print(f' Google CSE paid: $5.00 per 1,000 queries')
print(f' Scavio: $5.00 per 1,000 queries ($0.005 each)')
print(f' Scavio free: 250 queries/month included')Step 2: Build the migration adapter
Create a function with the same interface as your Google CSE calls but using Scavio. Optionally restrict to your domain list using site: queries.
Python
import requests, os
SCAVIO_KEY = os.environ['SCAVIO_API_KEY']
def search(query: str, num: int = 10, restrict_domains: list = None) -> list:
"""Drop-in replacement for Google CSE. Same return shape."""
search_query = query
if restrict_domains:
# Use site: operator to limit to specific domains
sites = ' OR '.join(f'site:{d}' for d in restrict_domains[:5])
search_query = f'{query} ({sites})'
resp = requests.post('https://api.scavio.dev/api/v1/search',
headers={'x-api-key': SCAVIO_KEY, 'Content-Type': 'application/json'},
json={'query': search_query, 'country_code': 'us', 'num_results': num})
resp.raise_for_status()
# Return in Google CSE items[] format
return [{'title': r['title'], 'link': r['link'],
'snippet': r.get('snippet', ''),
'displayLink': r['link'].split('/')[2] if '/' in r['link'] else ''}
for r in resp.json().get('organic_results', [])]
# Full web search (no domain limit)
results = search('python asyncio tutorial', num=5)
print(f'Full web: {len(results)} results')
for r in results:
print(f' {r["displayLink"]}: {r["title"][:50]}')Step 3: Validate migration with comparison tests
Run the same queries through both providers to ensure result quality is equivalent before fully cutting over.
Python
queries = ['python asyncio', 'react hooks tutorial', 'docker compose networking']
for q in queries:
# With domain restriction (mimics old Google CSE behavior)
restricted = search(q, num=3, restrict_domains=['docs.python.org', 'stackoverflow.com', 'github.com'])
# Full web (new capability)
full = search(q, num=3)
print(f'Query: {q}')
print(f' Restricted ({len(restricted)} results): {restricted[0]["displayLink"] if restricted else "none"}')
print(f' Full web ({len(full)} results): {full[0]["displayLink"] if full else "none"}')
print()Python Example
Python
import requests, os
SCAVIO_KEY = os.environ['SCAVIO_API_KEY']
def search(query, num=10):
resp = requests.post('https://api.scavio.dev/api/v1/search',
headers={'x-api-key': SCAVIO_KEY, 'Content-Type': 'application/json'},
json={'query': query, 'country_code': 'us', 'num_results': num})
return [{'title': r['title'], 'link': r['link'], 'snippet': r.get('snippet', '')}
for r in resp.json().get('organic_results', [])]
results = search('python asyncio tutorial')
for r in results[:5]:
print(f'{r["title"]}: {r["link"]}')JavaScript Example
JavaScript
const SCAVIO_KEY = process.env.SCAVIO_API_KEY;
async function search(query, num = 10) {
const resp = await fetch('https://api.scavio.dev/api/v1/search', {
method: 'POST',
headers: { 'x-api-key': SCAVIO_KEY, 'Content-Type': 'application/json' },
body: JSON.stringify({ query, country_code: 'us', num_results: num })
});
const data = await resp.json();
return (data.organic_results || []).map(r => ({
title: r.title, link: r.link, snippet: r.snippet || ''
}));
}
search('python asyncio tutorial').then(r => r.slice(0, 5).forEach(x => console.log(x.title)));Expected Output
JSON
Full web: 10 results
docs.python.org: Python asyncio -- Asynchronous I/O
realpython.com: Async IO in Python: A Complete Walkt
stackoverflow.com: How to use asyncio in Python 3
Query: python asyncio
Restricted (3 results): docs.python.org
Full web (3 results): docs.python.org
Cost: $0.005/query, no domain cap