The Brave Bot crawler cannot access all websites. Some domains need to whitelist the BraveBot.
Example :
Can we have IP addresses and user agent to whitelist Brave Bot?
Many thanks
1 Like
I totally agree, Brave should communicate its IP addresses or ASN, so we can whitelist Brave bot.
Also, Brave staff should contact Cloudflare and ask them to add Brave bot to the Cloudflare “known bots” whitelist https://developers.cloudflare.com/firewall/known-issues-and-faq#bots-currently-detected otherwise Brave will be automatically blocked by one of the largest CDN in the world.
Just want to add my vote for this. We specifically block most bots but would obviously trust a Brave search bot.
This is having a negative effect already as we’re not visible for a key term that we have a position for on Google.
I’m starting to think Brave doesn’t have a search bot…
Recently I noticed large numbers of “unique visitors” hitting my hobby website and I have blocked more than 99% of the ASNs advertising IPs hitting my site on Cloudflare.
I too would like some distinguishing information about Brave’s search crawler to allow them because the documentation about this only says if Google’s crawler is allowed so to will Brave’s but I doubt that’d be right because Cloudflare easily allows you to block any fake Google crawlers.