Overview
Takes AI-generated content as input, uses an LLM to extract verifiable factual claims, searches for each claim via SERP API, and classifies claims as verified, contradicted, or unverifiable.
Trigger
On-demand (triggered when AI-generated content is submitted for review)
Schedule
On-demand per content submission
Workflow Steps
Extract factual claims
Pass the content to an LLM with a prompt to extract all verifiable factual claims as a JSON list. Each claim should be a single sentence.
Generate search query per claim
For each claim, use the LLM or a template to generate an optimal search query (e.g., claim about a price -> '[product] price 2026 site:official or review site').
Search for each claim
POST to Scavio search API for each claim's search query. Retrieve top 5 results with snippets.
Classify claim against search results
Pass claim + search snippets to LLM. Classify as: verified (snippets corroborate claim), contradicted (snippets contradict), or unverifiable (snippets don't address claim).
Annotate content with flags
Tag each claim in the original content with its classification and the supporting or contradicting source URL.
Output verification report
Return JSON with: verified_count, contradicted_count, unverifiable_count, claims list with classification and source, and a recommended_revision for contradicted claims.
Python Implementation
import requests
import json
import openai
SCRAVIO_KEY = "YOUR_API_KEY"
client = openai.OpenAI(api_key="YOUR_OPENAI_KEY")
def extract_claims(content: str) -> list[str]:
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{
"role": "user",
"content": f"Extract all verifiable factual claims from this text as a JSON array of strings. Only include claims that can be checked against external sources (prices, dates, statistics, product features). Text:\n\n{content}"
}],
response_format={"type": "json_object"},
temperature=0
)
return json.loads(resp.choices[0].message.content).get("claims", [])
def search_claim(claim: str) -> list:
resp = requests.post(
"https://api.scavio.dev/api/v1/search",
headers={"x-api-key": SCRAVIO_KEY},
json={"query": claim, "platform": "google", "num": 5}
)
resp.raise_for_status()
return resp.json().get("results", [])
def verify_claim(claim: str, results: list) -> dict:
context = "\n".join(f"- {r.get('snippet', '')} ({r.get('url', '')})" for r in results[:5])
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{
"role": "user",
"content": f"Claim: {claim}\n\nSearch results:\n{context}\n\nClassify as verified/contradicted/unverifiable. Return JSON: {{\"status\": \"\", \"source_url\": \"\", \"note\": \"\"}}"
}],
response_format={"type": "json_object"},
temperature=0
)
return json.loads(resp.choices[0].message.content)
def fact_check(content: str) -> dict:
claims = extract_claims(content)
results_list = []
for claim in claims:
search_results = search_claim(claim)
verification = verify_claim(claim, search_results)
results_list.append({"claim": claim, **verification})
counts = {"verified": 0, "contradicted": 0, "unverifiable": 0}
for r in results_list:
status = r.get("status", "unverifiable")
counts[status] = counts.get(status, 0) + 1
return {**counts, "claims": results_list, "total_credits_used": len(claims)}
if __name__ == "__main__":
sample = "Scavio costs $99/month and has been used by over 10,000 companies. It supports 15 search platforms."
report = fact_check(sample)
print(json.dumps(report, indent=2))
JavaScript Implementation
const fetch = require('node-fetch');
const OpenAI = require('openai');
const SCRAVIO_KEY = 'YOUR_API_KEY';
const client = new OpenAI({ apiKey: 'YOUR_OPENAI_KEY' });
async function extractClaims(content) {
const resp = await client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: `Extract verifiable factual claims as JSON array. Text: ${content}` }],
response_format: { type: 'json_object' }, temperature: 0
});
return JSON.parse(resp.choices[0].message.content).claims || [];
}
async function searchClaim(claim) {
const res = await fetch('https://api.scavio.dev/api/v1/search', {
method: 'POST',
headers: { 'x-api-key': SCRAVIO_KEY, 'Content-Type': 'application/json' },
body: JSON.stringify({ query: claim, platform: 'google', num: 5 })
});
return (await res.json()).results || [];
}
async function verifyClaim(claim, results) {
const context = results.slice(0, 5).map(r => `- ${r.snippet} (${r.url})`).join('\n');
const resp = await client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: `Claim: ${claim}\nResults:\n${context}\nClassify: verified/contradicted/unverifiable. Return JSON: {"status":"","source_url":"","note":""}` }],
response_format: { type: 'json_object' }, temperature: 0
});
return JSON.parse(resp.choices[0].message.content);
}
async function factCheck(content) {
const claims = await extractClaims(content);
const results = [];
for (const claim of claims) {
const searchResults = await searchClaim(claim);
const verification = await verifyClaim(claim, searchResults);
results.push({ claim, ...verification });
}
const counts = results.reduce((acc, r) => { acc[r.status] = (acc[r.status]||0)+1; return acc; }, {});
return { ...counts, claims: results };
}
factCheck('Scavio costs $99/month and supports 15 platforms.').then(r => console.log(JSON.stringify(r, null, 2)));
Platforms Used
Web search with knowledge graph, PAA, and AI overviews