How Search Engines Work
Each search engine uses a crawler or spider with a set of guidelines to find Web pages to bring back to you. Some spiders follow every link on every page to bring back your results. Others look for the most popular pages as a high priority. A spider can search through a site in seconds to several minutes based on its complexity.
As a spider finds URLs and documents, software agents take these findings and send information about them to indexing software. The indexing software then takes the information from the documents and puts it in a database. Each search engine indexes different information, such as every word in the document, the key 100 words, the number of words, the size of the page, the page’s headings, etc. The type of index determines how the information will be displayed when you search.
When you want to search the Internet using a search engine, you must type keywords that you would like the engine to search for. Some search engines may allow you to search for more specific information such as the date it was last updated. The search engine then searches its database for the information you have specified. The results are displayed in an HTML page, however, every search engine displays its results differently. Some search engines will sort them by relevancy or by popularity, and some may show you a few sentences from the page and the page’s URL.
When you click on one of the links, you will be taken to that page because no documents are stored in the database – only information and links.