4images Forum & Community

General / Allgemeines => Chit Chat => Topic started by: V@no on February 24, 2003, 05:11:43 AM

Title: How does internet search engines works?
Post by: V@no on February 24, 2003, 05:11:43 AM

The reason I started this topic, because I've never subscribe my site to any search angines. But, recently, I got two hits from google.com ( http://www.google.com/search?hl=en&ie=Windows-1252&oe=Windows-1252&q=4images%20v@no )
so, the question is - HOW???

Title: How does internet search engines works?
Post by: Chris on February 24, 2003, 05:24:48 AM

That's easy.  

Google crawls the web and follows links.  You know there are many links to your web site from the forums here at 4homepages.de

http://www.google.com/webmasters/faq.html

Read all about it, including http://www.robotstxt.org/wc/norobots.html
for info on how to tell search engines not to crawl your site.

Title: How does internet search engines works?
Post by: V@no on February 24, 2003, 05:28:24 AM

thank you, Chris.
this is an answer for another question I had.
in the apache error log I noticed time-to-time 404 error, after requests for robots.txt file. getting clear now  :D