There are several types of indexing signals sites can implement to control the way their site is indexed. These are often used to manipulate search engines and block certain pages from being crawled and indexed by Google.
In this guide, we list the key indexing signals, their purpose and what happens when you mix them.
What are the different types of Indexing Signals, their purpose and use?
|
Type of
Signal:
|
Purpose: |
Info |
When to use them:
|
|
Noindex
|
Index control
|
- Google states that if a page is noindexed it will become crawled less frequently over time.
- This is important for eCommerce pagination as if your pagination is noindex, followed, they may become crawled less frequently.
- Avoid using these with canonicals.
- Noindex tags don’t provide any information about passing the information on from one page to another (like canonical etc.)
|
- Marketing/social media pages that you don’t want users to find in Google.
- Pages with thin content that you want to keep, but don’t want users to find.
- You should remove all internal links to noindex pages.
- Parameter urls that you do not want in Google’s index
|
|
Nofollow
|
Crawl control:
rel=“nofollow” tag.
|
- Google does not transfer pagerank across nofollow tags. This means if an external site provides a nofollow link to your site, no SEO value will be passed on.
- There are two ways nofollow links are used – page level and link level nofollows.
- At a page level the page still gets crawled (but the links on that page do not).
- Link level, the link does not get crawled but does pass some PageRan).
|
- Links to user-generated content/forums.
- Can be used on parameter pages again to stop Google from crawling the deeper internal links
|
|
Robots.txt
|
Crawl control
|
- Don’t block pages or links that you want to be crawled. However, do block stuff you don’t want to be crawled.
- Remember that blocking e-commerce pages also blocks Google Shopping so take caution.
- You can check a sites robots.txt file by typing /robots.txt after a domain in the search bar.
|
- Blocking parameter URLs from being crawled in e-commerce sites e.g.
- ?search=
- ?sorting=
- ?requiredIds=&requiredIds=* (tbc based on other tickets)
- Block any performance marketing landing pages e.g. if you put your PPC landing pages in a /PPC/ folder you can block these for SEO and avoid duplication
|
|
Canonical
|
Index control
rel=” canonical”
|
- Not a directive so Google can (and will) ignore it. So don’t solely rely on them if there is a page you don’t want to be indexed.
- Make sure to use canonicals correctly.
|
- Often used on news sites which have multiple versions or colours of one product.
|
|
JavaScript
|
Crawl & index control
|
- Using some JS that Google can’t crawl can technically mean areas of a site aren’t crawled or indexed but can be used by visitors.
|
- Filters or parameters that you know have 0 value to Google so there’s no need to use a static URL e.g. a price slider would work as front end JavaScript which google can not see or reference
|
What happens when you mix indexing signals?
Canonical and noindex
This will give conflicting signals to Google. Google states that it will most likely pick rel=canonical over noindex if both are used in conjunction with each other. However, this is not recommended.
The canonical suggests the pages have the same content on (canonical), but one should be noindexed.
Robots.txt and noindex
If robots.txt is set up correctly, Google can still index a page but not crawl it. This means that if a page has a noindex tag, Google won’t be able to. Google Search Console usually reports these images with an error message.
Canonical and robots block
This will give mixed signals to Google and we’d avoid using this combination.
A correct understanding of indexing signals and its various purposes are essential to design an appropriate tech SEO strategy for your eCommerce site so that Google can bring the customers to your site smoothly. We understand that this topic can be confusing, so if you still have any questions, tweet me (Laura) at @lauraabrady8.
