User-agent: ia_archiver Disallow: / User-agent: CCBot Disallow: / User-agent: GPTBot Disallow: / User-agent: archive.org_bot Disallow: / User-agent: Riddler Disallow: / User-agent: BLEXBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: sogou spider Disallow: / User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User agent: MTRobot Disallow: / User-agent: Baiduspider Disallow: / User-agent: dotbot Disallow: / User-agent: SurveyBot Disallow: / User-agent: OSZKbot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: * Disallow: /404.php