# robots.txt for The Delia Group, https://thedeliagroup.com # Updated 2026-11-19 # # Policy: fully open. Search engines, AI answer engines and AI training # crawlers may all read and learn from this site. # A real estate team wins when buyers and sellers can find it, so every # crawler that answers a client's question is allowed. # # How to read this file: # Group 1 is the default for any crawler not named below: allowed. # The groups that follow name every crawler the business wants, so a future # edit to the default cannot silently remove them. Retrieval crawlers answer # live questions in ChatGPT, Claude, Perplexity, Gemini, Copilot, Meta AI, # Alexa and Siri. Training crawlers teach the models that answer tomorrow's # questions. Both are welcome here. Nothing is blocked. # --------------------------------------------------------------------------- # Default for every crawler not named below # --------------------------------------------------------------------------- User-agent: * Allow: / # --------------------------------------------------------------------------- # Search engines # --------------------------------------------------------------------------- User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Bingbot Allow: / User-agent: Applebot Allow: / User-agent: DuckDuckBot Allow: / # --------------------------------------------------------------------------- # AI answer engines and assistants (retrieval) # --------------------------------------------------------------------------- User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Google-Extended Allow: / User-agent: meta-webindexer Allow: / User-agent: Meta-ExternalFetcher Allow: / User-agent: Amazonbot Allow: / User-agent: DuckAssistBot Allow: / User-agent: MistralAI-User Allow: / User-agent: YouBot Allow: / # --------------------------------------------------------------------------- # AI training crawlers, so future models know the brand # --------------------------------------------------------------------------- User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: Applebot-Extended Allow: / User-agent: GoogleOther Allow: / User-agent: CCBot Allow: / User-agent: cohere-ai Allow: / User-agent: Bytespider Allow: / # --------------------------------------------------------------------------- # Sitemap and AI content summary # --------------------------------------------------------------------------- Sitemap: https://thedeliagroup.com/sitemap.xml # AI content summary: https://thedeliagroup.com/llms.txt