User-agent: * Disallow: /fedora/ Disallow: /centos/ Disallow: /ubuntu/ubuntu/* # The SemRush Bot is going nuts in the Ubuntu tree User-agent: SemrushBot Disallow: /ubuntu/*