Quantcast
Channel: phpBB.com
Viewing all articles
Browse latest Browse all 1732

[3.3.x] Support Forum • Help with phpbb and robots.txt

$
0
0
The google bot is suddenly going haywire on my site using up a ton of bandwidth because it's trying to index a bunch of session/redirect/login URLs that look similar to:

Code:

/ucp.php?mode=login&redirect=viewtopic.php?p=XXXXX&sid=XXXXXXX
I tried modifying robots.txt.

Code:

User-agent: *crawl-delay: 5Disallow: /ucp
... and variations of that disallow, but nothing seems to be working. Sorry if this question is beyond the scope of this forum, but figured I would ask and see if anyone has any ideas.

EDIT: I just realized it could take up to 24 hours for it to take effect.

Statistics: Posted by Jx80 — Tue Jun 04, 2024 12:48 am



Viewing all articles
Browse latest Browse all 1732

Trending Articles