The google bot is suddenly going haywire on my site using up a ton of bandwidth because it's trying to index a bunch of session/redirect/login URLs that look similar to:
I tried modifying robots.txt.
... and variations of that disallow, but nothing seems to be working. Sorry if this question is beyond the scope of this forum, but figured I would ask and see if anyone has any ideas.
EDIT: I just realized it could take up to 24 hours for it to take effect.
Code:
/ucp.php?mode=login&redirect=viewtopic.php?p=XXXXX&sid=XXXXXXX
Code:
User-agent: *crawl-delay: 5Disallow: /ucp
EDIT: I just realized it could take up to 24 hours for it to take effect.
Statistics: Posted by Jx80 — Tue Jun 04, 2024 12:48 am