ScavioScavio
ProductPricingDocs
Sign InGet Started
  1. Home
  2. Tutorials
  3. How to Optimize D2C Site for AI Agent Discovery
Tutorial

How to Optimize D2C Site for AI Agent Discovery

Make your D2C brand visible to AI shopping agents. Structured data, llms.txt, and product feed optimization for agent traffic.

Get Free API KeyAPI Docs

AI shopping agents from ChatGPT, Perplexity, and Claude now recommend products directly. If your D2C site is not optimized for agent crawling, you lose these referrals to Amazon. This tutorial shows how to audit your current agent visibility, add structured data, create llms.txt, and verify agents can find your products.

Prerequisites

  • Python 3.8+
  • requests library
  • A Scavio API key from scavio.dev
  • D2C site URL with product pages

Walkthrough

Step 1: Audit current AI agent visibility

Check if AI agents can find and recommend your products today.

Python
import os, requests

API_KEY = os.environ['SCAVIO_API_KEY']
SH = {'x-api-key': API_KEY, 'Content-Type': 'application/json'}

def check_visibility(brand, product):
    queries = [
        f'best {product} 2026',
        f'{product} recommendations',
        f'buy {product} online direct',
        f'{brand} {product} review',
    ]
    results = {'found': 0, 'total': 0, 'positions': []}
    for q in queries:
        data = requests.post('https://api.scavio.dev/api/v1/search',
            headers=SH, json={'query': q, 'country_code': 'us'}).json()
        organic = data.get('organic_results', [])
        results['total'] += 1
        for i, r in enumerate(organic):
            if brand.lower() in r.get('link', '').lower() or brand.lower() in r.get('title', '').lower():
                results['found'] += 1
                results['positions'].append(i + 1)
                break
    visibility = results['found'] / results['total'] * 100 if results['total'] else 0
    avg_pos = sum(results['positions']) / len(results['positions']) if results['positions'] else 0
    print(f'Brand: {brand}')
    print(f'Visibility: {visibility:.0f}% ({results["found"]}/{results["total"]} queries)')
    print(f'Avg position when found: {avg_pos:.1f}')
    print(f'Cost: ${results["total"] * 0.005:.3f}')
    return results

check_visibility('YourBrand', 'organic face cream')

Step 2: Check structured data and llms.txt

Verify that your site has the right signals for AI agent crawling.

Python
def audit_agent_readiness(domain):
    """Check if a D2C site is ready for AI agent discovery."""
    checks = []
    # Check llms.txt
    data = requests.post('https://api.scavio.dev/api/v1/search',
        headers=SH, json={'query': f'site:{domain} llms.txt', 'country_code': 'us'}).json()
    has_llms = len(data.get('organic_results', [])) > 0
    checks.append(('llms.txt exists', has_llms))
    # Check structured data
    data = requests.post('https://api.scavio.dev/api/v1/search',
        headers=SH, json={'query': f'site:{domain} product', 'country_code': 'us'}).json()
    results = data.get('organic_results', [])
    has_rich = any(r.get('rich_snippet') for r in results)
    checks.append(('Rich snippets on product pages', has_rich))
    # Check sitemap
    data = requests.post('https://api.scavio.dev/api/v1/search',
        headers=SH, json={'query': f'site:{domain} sitemap.xml', 'country_code': 'us'}).json()
    has_sitemap = len(data.get('organic_results', [])) > 0
    checks.append(('Sitemap indexed', has_sitemap))
    print(f'\nAgent Readiness Audit: {domain}')
    for check, passed in checks:
        status = 'PASS' if passed else 'FAIL'
        print(f'  [{status}] {check}')
    score = sum(1 for _, p in checks if p)
    print(f'\nScore: {score}/{len(checks)}')
    return checks

audit_agent_readiness('example-d2c-brand.com')

Step 3: Generate llms.txt for your D2C site

Create an llms.txt file that helps AI agents understand your product catalog.

Python
def generate_llms_txt(brand, domain, products):
    """Generate llms.txt for a D2C brand."""
    lines = [
        f'# {brand}',
        f'',
        f'> {brand} sells directly to consumers at {domain}.',
        f'> All products ship from our warehouse. Free returns within 30 days.',
        f'',
        f'## Products',
    ]
    for p in products:
        lines.append(f'- [{p["name"]}](https://{domain}/products/{p["slug"]}): {p["description"]}')
    lines.extend([
        f'',
        f'## Ordering',
        f'- [Shop All](https://{domain}/shop)',
        f'- [FAQ](https://{domain}/faq)',
        f'- [Shipping Policy](https://{domain}/shipping)',
        f'- [Returns](https://{domain}/returns)',
    ])
    content = '\n'.join(lines)
    print(content)
    return content

# Example product catalog
products = [
    {'name': 'Organic Face Cream', 'slug': 'organic-face-cream', 'description': '50ml, $28, for sensitive skin'},
    {'name': 'Vitamin C Serum', 'slug': 'vitamin-c-serum', 'description': '30ml, $22, brightening formula'},
]

llms_txt = generate_llms_txt('GlowCo', 'glowco.com', products)
print(f'\nSave this as /llms.txt on your domain root')

Step 4: Monitor agent discovery over time

Track whether AI agents start finding your products after optimization.

Python
import json

def track_agent_visibility(brand, domain, products, history_file='visibility_log.json'):
    try:
        with open(history_file) as f:
            history = json.load(f)
    except FileNotFoundError:
        history = []
    today = {'date': '2026-05-20', 'queries': []}
    for product in products:
        query = f'best {product} 2026'
        data = requests.post('https://api.scavio.dev/api/v1/search',
            headers=SH, json={'query': query, 'country_code': 'us'}).json()
        organic = data.get('organic_results', [])
        position = None
        for i, r in enumerate(organic):
            if domain in r.get('link', ''):
                position = i + 1
                break
        today['queries'].append({'query': query, 'position': position})
    history.append(today)
    found = sum(1 for q in today['queries'] if q['position'])
    print(f'Visibility check: {found}/{len(products)} products found in SERP')
    for q in today['queries']:
        pos = f'#{q["position"]}' if q['position'] else 'Not found'
        print(f'  {q["query"][:40]:40} | {pos}')
    if len(history) >= 2:
        prev = history[-2]
        prev_found = sum(1 for q in prev['queries'] if q['position'])
        delta = found - prev_found
        print(f'\nChange vs last check: {delta:+d} products visible')
    print(f'Cost: ${len(products) * 0.005:.3f}')

track_agent_visibility('GlowCo', 'glowco.com', ['organic face cream', 'vitamin c serum'])

Python Example

Python
import os, requests
SH = {'x-api-key': os.environ['SCAVIO_API_KEY'], 'Content-Type': 'application/json'}

def check_d2c(brand, product):
    data = requests.post('https://api.scavio.dev/api/v1/search',
        headers=SH, json={'query': f'best {product} 2026', 'country_code': 'us'}).json()
    for i, r in enumerate(data.get('organic_results', [])):
        if brand.lower() in r.get('title', '').lower():
            print(f'{brand} found at position {i+1}')
            return
    print(f'{brand} not found in top results')

check_d2c('GlowCo', 'organic face cream')
print('Cost: $0.005')

JavaScript Example

JavaScript
const SH = { 'x-api-key': process.env.SCAVIO_API_KEY, 'Content-Type': 'application/json' };
async function checkD2C(brand, product) {
  const data = await fetch('https://api.scavio.dev/api/v1/search', {
    method: 'POST', headers: SH,
    body: JSON.stringify({ query: `best ${product} 2026`, country_code: 'us' })
  }).then(r => r.json());
  const pos = (data.organic_results || []).findIndex(r => r.title.toLowerCase().includes(brand.toLowerCase()));
  console.log(pos >= 0 ? `${brand} at #${pos+1}` : `${brand} not found`);
}
await checkD2C('GlowCo', 'organic face cream');

Expected Output

JSON
Brand: GlowCo
Visibility: 50% (2/4 queries)
Avg position when found: 6.5
Cost: $0.020

Agent Readiness Audit: glowco.com
  [FAIL] llms.txt exists
  [PASS] Rich snippets on product pages
  [PASS] Sitemap indexed

Score: 2/3

Visibility check: 2/2 products found in SERP
  best organic face cream 2026             | #5
  best vitamin c serum 2026                | #8
Cost: $0.010

Related Tutorials

  • How to Build an AI Visibility Audit Pipeline
  • How to Track Whether Your SaaS Shows Up in AI Search
  • How to Audit Your Site for LLM Readability

Frequently Asked Questions

Most developers complete this tutorial in 15 to 30 minutes. You will need a Scavio API key (free tier works) and a working Python or JavaScript environment.

Python 3.8+. requests library. A Scavio API key from scavio.dev. D2C site URL with product pages. A Scavio API key gives you 50 free credits on signup.

Yes. The free tier includes 50 credits on signup, which is more than enough to complete this tutorial and prototype a working solution.

Scavio has a native LangChain package (langchain-scavio), an MCP server, and a plain REST API that works with any HTTP client. This tutorial uses the raw REST API, but you can adapt to your framework of choice.

Related Resources

Use Case

D2C AI Agent Discovery Optimization

Read more
Best Of

Best D2C AI Readiness and llms.txt Tools (2026)

Read more
Workflow

D2C AI Readiness Audit

Read more
Best Of

Best API for Autonomous AI Agents in 2026

Read more
Solution

D2C Product Discovery for AI Agents

Read more
Use Case

YouTube Agent Data

Read more

Start Building

Make your D2C brand visible to AI shopping agents. Structured data, llms.txt, and product feed optimization for agent traffic.

Get Free API KeyRead the Docs
ScavioScavio

Real-time search API for AI agents. Search every platform, not just Google.

Product

  • Features
  • Pricing
  • Dashboard
  • Affiliates

Developers

  • Documentation
  • API Reference
  • Quickstart
  • MCP Integration
  • Python SDK

Alternatives

  • Tavily Alternative
  • SerpAPI Alternative
  • Firecrawl Alternative
  • Exa Alternative

Tools

  • JSON Formatter
  • cURL to Code
  • Token Counter
  • All Tools

© 2026 Scavio. All rights reserved.

Featured on TAAFT
Terms of ServicePrivacy Policy