- Add react-helmet-async with SEO component for dynamic meta tags on all pages - Add JSON-LD structured data (Organization, WebSite, ProfessionalService) to index.html - Add fallback OG/Twitter Card meta tags in index.html for non-JS crawlers - Add robots.txt allowing all crawlers including AI bots (GPTBot, Claude-Web, etc.) - Add sitemap.xml with all routes - Add llms.txt for AI crawler discovery - Add X-Robots-Tag header and reviews.json cache rule to nginx.conf - Replace fake testimonials with Google Reviews (fetch /reviews.json) - Add star ratings, Google badge, and "Leave us a review" button to testimonials - Add server/sync-reviews.mjs for daily Google Places API review sync Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
22 lines
290 B
Text
22 lines
290 B
Text
User-agent: *
|
|
Allow: /
|
|
|
|
User-agent: GPTBot
|
|
Allow: /
|
|
|
|
User-agent: ChatGPT-User
|
|
Allow: /
|
|
|
|
User-agent: Claude-Web
|
|
Allow: /
|
|
|
|
User-agent: PerplexityBot
|
|
Allow: /
|
|
|
|
User-agent: Amazonbot
|
|
Allow: /
|
|
|
|
User-agent: anthropic-ai
|
|
Allow: /
|
|
|
|
Sitemap: https://ai-impress.com/sitemap.xml
|