Ok just in case anyone has a similar problem this one worked for me.
viewtopic.php?p=16018150#p16018150Add this to your .htaccess fileCode:
BrowserMatchNoCase "libwww-perl" bad_botBrowserMatchNoCase "wget" bad_botBrowserMatchNoCase "LieBaoFast" bad_botBrowserMatchNoCase "Mb2345Browser" bad_botBrowserMatchNoCase "zh-CN" bad_botBrowserMatchNoCase "MicroMessenger" bad_botBrowserMatchNoCase "zh_CN" bad_botBrowserMatchNoCase "Kinza" bad_botBrowserMatchNoCase "Bytespider" bad_botBrowserMatchNoCase "Baiduspider" bad_botBrowserMatchNoCase "Sogou" bad_botBrowserMatchNoCase "Datanyze" bad_botBrowserMatchNoCase "AspiegelBot" bad_botBrowserMatchNoCase "adscanner" bad_botBrowserMatchNoCase "serpstatbot" bad_botBrowserMatchNoCase "spaziodat" bad_botBrowserMatchNoCase "undefined" bad_botBrowserMatchNoCase "claudebot" bad_botBrowserMatchNoCase "facebook" bad_botBrowserMatchNoCase "Petalbot" bad_botBrowserMatchNoCase "YandexBot" bad_botBrowserMatchNoCase "Applebot" bad_botBrowserMatchNoCase "aiohttp" bad_botOrder Deny,AllowDeny from env=bad_botDeny from 47.76.0.0/16
won't this have negative impact on SEO if the crawlers are blocked?
Statistics: Posted by fuolo — Tue Aug 06, 2024 9:21 am