What is crawlers, spiders, robots?

A Crawler also known as Robot, Bot, Spider is a program that crawls Web site and reads
their pages and other information in order to create entries for a search engine index.As most of the websites contain links to other pages, a crawler can start crawling from
anywhere in the pages. As soon as it sees a link to another page, it can start crawling
that page leaving behind the current page that it was spidering.
Name of some search engine robots
Search Engines Robot
Google --> googlebot
MSN --> msnbot
Yahoo --> yahoo slurp
excite.com ---> architextspider
entireweb ---> speedy spider
look.com ---> lookbot
scrubtheweb.com --> scrubby

How search engines work?

Everyone wants those good listings. Unfortunately, many Web sites appear poorly in search engine rankings or may not be listed at all because they fail to consider how search engines work.When you query a search engine for information, you are actually searching through the index that the search engine has created - you are not actually searching the web. These indices are giant databases of information that is collected and stored and subsequently searched.

Search Engine Optimization And Its Importance

Let me give you a brief details about search engine optimisation and its importance. Search engine optimization or SEO is the technique of optimizing the web pages in such a way so that they rank higher in the search engine result pages. The higher a website ranks in the results of a search, the greater the chance that the site will be visited by a user. It is a common practise for Internet users that they don't click through pages and pages of search results, so where a site ranks in search is essential for directing more towards the site.

What Is Search Engine?

Let me first tell you about search engines before going into the details of search engine optimization.
Search Engines are a database for a large number of websites. It is a tool for finding information on the Internet.
Google, Yahoo & MSN are the three major search engines among all the hundereds & thousands of search engines that are present in the world of inernet.