Minnocent
|
  |
| Joined: 26 Dec 2011 |
| Total Posts: 134 |
|
|
| 15 Jul 2012 07:30 PM |
go to www.roblox.com/robots.txt lol |
|
|
| Report Abuse |
|
|
Ryancp1
|
  |
| Joined: 05 Oct 2010 |
| Total Posts: 10396 |
|
| |
|
| |
|
| |
|
| |
|
Garnished
|
  |
| Joined: 09 Apr 2012 |
| Total Posts: 12695 |
|
|
| 15 Jul 2012 07:35 PM |
>Disallow: /JavaScript/
Javascript is a fine scripting language.
Please Define 'Noob'. ~ Garnished |
|
|
| Report Abuse |
|
|
balle
|
  |
| Joined: 22 Feb 2008 |
| Total Posts: 1203 |
|
|
| 15 Jul 2012 07:35 PM |
| Google uses robots to index sites. (So when you search ROBLOX, you get ROBLOX) |
|
|
| Report Abuse |
|
|
| |
|
GamerIMax
|
  |
| Joined: 12 Mar 2012 |
| Total Posts: 11146 |
|
| |
|
flame7787
|
  |
| Joined: 17 Dec 2011 |
| Total Posts: 150 |
|
|
| 05 Jan 2015 04:00 PM |
Disallow: /AbuseReport/
Wow
Disallow: /Data/ Disallow: /Reports/ Disallow: /JavaScript/
And they should change this Disallow: /Error/ To this Allow: /Error/ Naw jking |
|
|
| Report Abuse |
|
|
flame7787
|
  |
| Joined: 17 Dec 2011 |
| Total Posts: 150 |
|
|
| 05 Jan 2015 04:03 PM |
Oh and all of this is to index searches from the ROBLOX site.
Like when you take Google and search ROBLOX.
Under the link is text in the website. They can allow or disallow for info down there to be shown. |
|
|
| Report Abuse |
|
|
|
| 05 Jan 2015 04:03 PM |
Disallow: /AbuseReport/ haha but i think this is so when you search up roblox,you dont end up on the abuse report page, any of the scripts on the site,et cetera. |
|
|
| Report Abuse |
|
|
|
| 05 Jan 2015 04:04 PM |
User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /User-agent: * Disallow: /Admi/ Disallow: /AbuseReport/ Disallow: /Data/ Disallow: /Error/ Disallow: /JavaScript/ Disallow: /Reports/ Disallow: /CombineScriptsHandler.ashx Disallow: /Forum/AddPost.aspx Disallow: /asset/ Disallow: /Ads/
User-agent: Mediapartners-Google Allow: /
User-agent: AdsBot-Google Allow: /v |
|
|
| Report Abuse |
|
|
|
| 10 Jun 2017 08:13 PM |
| Anyone can access it, which is why roblox has security for those situations. It isn't a new thing that roblox has robots.txt |
|
|
| Report Abuse |
|
|
xvbones
|
  |
| Joined: 09 Nov 2006 |
| Total Posts: 6029 |
|
|
| 10 Jun 2017 08:18 PM |
| batman are you just bumping everything that says "robots.txt" |
|
|
| Report Abuse |
|
|
Probation
|
  |
| Joined: 12 Aug 2011 |
| Total Posts: 5392 |
|
| |
|
DR2D2
|
  |
| Joined: 10 Jan 2012 |
| Total Posts: 2592 |
|
| |
|
| |
|
IVIeal
|
  |
| Joined: 09 Apr 2013 |
| Total Posts: 123456 |
|
| |
|
|
| 25 Jun 2017 07:26 PM |
robots.txt is used 2 tell web crawlies and web bots places they shouldnt be lookin on a website
idk why roblox has one tho |
|
|
| Report Abuse |
|
|
IVIeal
|
  |
| Joined: 09 Apr 2013 |
| Total Posts: 123456 |
|
| |
|
Versalty
|
  |
| Joined: 14 Feb 2017 |
| Total Posts: 5521 |
|
| |
|
xvbones
|
  |
| Joined: 09 Nov 2006 |
| Total Posts: 6029 |
|
|
| 26 Jun 2017 09:59 AM |
"idk why roblox has one tho"
most sites have one, there may be spiders that can unintentionally disrupt a site's data/analytics
i left a work-in-progress site with almost nothing on it other than a visitor counter and after about a week i'd had 16,000 hits from polish crawlers. upon adding robots.txt they stopped, so yeah it's basically a case of getting rid of fake traffic |
|
|
| Report Abuse |
|
|
DR2D2
|
  |
| Joined: 10 Jan 2012 |
| Total Posts: 2592 |
|
|
| 26 Jun 2017 10:25 AM |
| user-agent: * ######### ############# ######### ###### ######### /ads/ ######### ############################### ######### ########################## ######### /catalog/contents ######### /catalog/html ######### /chat/ ######### /client-status ######### /data/ ######### /error/ ######### /forum/addpost.aspx ######### ############################## ######### /game/report-event ######### /game/report-stats ######### ############################## ######### ################## ######### ############ ######### /outfits/fetch ######### /reports/ ######### ############################ ######### #################### ######### ##################### ######### /thumbnail/remove-asset-media ######### /thumbnail/resolve-hash ######### /thumbnail_holder/g ######### /tracker/conversion/googleconversion.aspx ######### /viewapp/ insert signature here |
|
|
| Report Abuse |
|
|