Countdown to Judgement Day

MyCult

SEO to the FACE
it may also be a countdown to the New Year



... at which time SKYNET IS GOING TO START IT'S WAR AGAINST THE HUMANS
 

db

#chaplife
you really need to stop worrying so much about google lol





p5afln
(or do you?)
 

gdavies

Well-Known Forumite
well i agree why not have a look at what a google search i did brought up for me http://lmgtfy.com/?q=inurl%3Awp-admin+%2Bsite%3A.us

is this really what google should be revealing, some of you may ask is that the type of thing i should search well i only searched a little bit of code that would display admin section of wordpress blogs in the us and google is just so happy to help
 

Alan B'Stard

Well-Known Forumite
gdavies said:
well i agree why not have a look at what a google search i did brought up for me http://lmgtfy.com/?q=inurl%3Awp-admin+%2Bsite%3A.us

is this really what google should be revealing, some of you may ask is that the type of thing i should search well i only searched a little bit of code that would display admin section of wordpress blogs in the us and google is just so happy to help
Surely that's just an example of a poorly setup website that has not used a robots.txt file correctly rather than any fault on Skynet, sorry I mean Google.
 

shoes

Well-Known Forumite
The "I'm Feeling LuckyTM" button automatically takes you to the first web page returned for your query.
An "I'm Feeling Lucky" search means less time searching for web pages and more time looking at them.
 

gdavies

Well-Known Forumite
Colin Grigson said:
gdavies said:
well i agree why not have a look at what a google search i did brought up for me http://lmgtfy.com/?q=inurl%3Awp-admin+%2Bsite%3A.us

is this really what google should be revealing, some of you may ask is that the type of thing i should search well i only searched a little bit of code that would display admin section of wordpress blogs in the us and google is just so happy to help
Surely that's just an example of a poorly setup website that has not used a robots.txt file correctly rather than any fault on Skynet, sorry I mean Google.
yes it is there fault but google or skynet is quick on the mark to pick them up
 

Alan B'Stard

Well-Known Forumite
gdavies said:
Colin Grigson said:
gdavies said:
well i agree why not have a look at what a google search i did brought up for me http://lmgtfy.com/?q=inurl%3Awp-admin+%2Bsite%3A.us

is this really what google should be revealing, some of you may ask is that the type of thing i should search well i only searched a little bit of code that would display admin section of wordpress blogs in the us and google is just so happy to help
Surely that's just an example of a poorly setup website that has not used a robots.txt file correctly rather than any fault on Skynet, sorry I mean Google.
yes it is there fault but google or skynet is quick on the mark to pick them up
Because it automatically picks up everything a robots file allows it too, it's not up to Skynet to compensate for poorly constructed websites.
 

MyCult

SEO to the FACE
It's called a footprint. All CMS's have a footprint (most have more than one) you just need to know what your looking for. You can use search engines and footprints to scrape large lists of sites based on a common CMS e.g. Wordpress, PunBB, Drupal, CubeCart, OScommerce the list goes on.

Why you would want to do that? ... I have no idea :sneaky:

Skynet does play by the rules set in robot.txt most of the time and their crawler is definitely one of the better ones. :up:

That being said I have personal experience of Gbot looking at a robots.txt then ignoring it completely and going on and doing wtf ever it wants anyway.
 
Top