blitzwing/bintzwing
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
292 commits 9 branches 30 tags 15 MiB
Commit graph

3 commits

Author SHA1 Message Date
ic0nic
c918a9fe2e Incorrect structure 
The structure for robots.txt is incorrect for some/most search engines.
 2013-11-01 01:22:16 +01:00
ic0nic
50d4fb9bf3 Allow discovery 
To allow search engines to at least find the website.
 2013-01-29 19:33:31 +00:00
Greg Knaddison
f37303d858 For server-load and privacy reasons, disallow robots. 2012-08-10 18:21:26 +02:00