Search engines have two major functions: crawling and building an index, and providing search users with a ranked list of the websites they've determined are the most relevant. A search engine crawler is a program or automated script that browses the World Wide Web in a methodical manner in order to provide up to date data to the particular search engine. While search engine crawlers go by many different names, such as web spiders and automatic indexers, the job of the search engine crawler is still the same. 

Crawling- This fetches all the web pages linked to a particular website. A software known as a crawler or a spider or Googlebot does crawling.

Indexing- It creates an index for all the fetched web pages and keeps them into a giant database from where retrieval is also possible. It can be understood as identifying the words and expressions that accurately describe your page.

Processing- It involves the processing of search request by comparing search string in the search request with the indexed pages in the search engine database.

Calculating Relevancy- As there could be more than one page contains the same search string; relevancy calculation helps in checking the relevancy of each of the pages.

Retrieving Results- Retrieving the best-matched results is the last search engine activity.

Search Engine Rank

For any keyword search on a search engine, thousands of results may occur. Ranking of your page is measured by the position of your web pages displayed in SERPs (search engine result pages).

How do I write a good title tag?

Because title tags are such an important part of both search engine optimization and the search user experience, writing them effectively is a terrific low-effort, high-impact SEO task. Here are critical recommendations for optimizing title tags for search engine and usability goals:

1. Watch your title length

If your title is too long, search engines may cut it off by adding an ellipsis and could end up omitting important words. While we generally recommend keeping your titles under 60 characters long, the exact limit is a bit more complicated and is based on a 600-pixel container. Try to avoid ALL CAPS titles. They may be hard for search visitors to read, and may severely limit the number of characters Google will display. Keep in mind that, even within a reasonable length limit, search engines may choose to display a different title than what you provide in your title tag.

Keep in mind that longer titles may work better for social sharing in some cases, and some titles are just naturally long. Its good to be mindful of how your titles appear in search results, but there are no penalties for using a long title. Use your judgment, and think like a search visitor.

2. Don't overdo SEO keywords

While there is no penalty built into Googles algorithm for long titles, you can run into trouble if you start stuffing your title full of keywords in a way that creates a bad user experience, such as:

Buy Widgets, Best Widgets, Cheap Widgets, Widgets for Sale

Avoid titles that are just a list of keywords or repeat variations of the same keyword over and over. These titles are bad for search users and could get you into trouble with search engines. Search engines understand variations of keywords, and it's unnecessary and counterproductive to stuff every version of your keyword into a title.

3. Give every page a unique title

Unique titles help search engines understand that your content is unique and valuable, and also drive higher click-through rates. On the scale of hundreds or thousands of pages, it may seem impossible to craft a unique title for every page, but modern CMS and code-based templates should allow you to at least create data-driven, unique titles for almost every an important page of your site.

For example, if you have thousands of product pages with a database of product names and categories, you could use that data to easily generate titles like:

[Product Name] - [Product Category] | [Brand Name]

4. Put important keywords first

Keywords closer to the beginning of your title tag may have more impact on search rankings. In addition, user experience research shows that people may scan as few as the first two words of a headline. This is why we recommend titles were the most unique aspect of the page (e.g. the product name) appears first. Avoid titles like:

Brand Name | Major Product Category - Minor Product Category - Name of Product

5. Take advantage of your brand

If you have a strong, well-known brand, then adding it to your titles may help boost click-through rates. We generally still recommend putting your brand at the end of the title, but there are cases (such as your home page or about page) where you may want to be more brand-focused. As mentioned earlier, Google may also append your brand automatically to your display titles, so be mindful of how your search results are currently displayed.

6. Write for your customers

While title tags are very important to SEO, remember that your first job is to attract clicks from well-targeted visitors who are likely to find your content valuable. It's vital to think about the entire user experience when you're creating your title tags, in addition to optimization and keyword usage. The title tag is a new visitor's first interaction with your brand when they find it in a search result— it should convey the most positive and accurate message possible.