mirrors/catstodon
1
0
Fork 0
mirror of https://git.kescher.at/CatCatNya/catstodon.git synced 2024-11-22 14:08:07 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
catstodon/public/robots.txt
ThibG c4f2433300 Disallow robots from indexing /interact/ (#10666) 
This does not provide any new information and may just triple the number
of crawled pages
2019-05-02 00:10:19 +02:00

5 lines
158 B
Text
Raw Blame History

# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
User-agent: *
Disallow: /media_proxy/
Disallow: /interact/
Reference in a new issue View git blame Copy permalink