Speed up indexing of website data with IndexNow tools of Microsoft Bing and Yandex.
Website owners often have to find every method to get their content indexed as quickly as possible. Unless you have a large enough website, chances are search engines only crawl your site to index new content every few days. To solve this problem and improve crawl efficiency, Microsoft Bing and Yandex teamed up to create an API-based protocol called IndexNow.
Explore more:
Table of Contents
What is IndexNow?
IndexNow is a protocol that allows you to notify search engines of changes to your website with an HTTP request. With each API call, you can provide search engines using IndexNow with a list of up to 10,000 changed URLs on your site.
IndexNow aims to make the internet more efficient. It’s an effort to help search engines improve crawling performance by eliminating unnecessary resource-consuming bulk crawling.
How are CDN services supporting IndexNow?
CDNs are often the first services to receive a signal when users update content on their site. Furthermore, they have crawler behavior insights to leverage IndexNow effectively.
CloudFlare, the world’s largest CDN service (with over 80% market share), has powered IndexNow. They integrated it into the Crawler Hints feature and made it available to users for free.
Similarly, Akamai, another leading CDN service provider, also announced support for IndexNow so their customers can more easily send notifications to search engines when content is cached in memory. Their cache changes.
With the help of industry-leading CDN services, it is likely that search engine crawling will soon transition from being proactive to reactive to requests.
Difference between IndexNow and Google’s Indexing API
Google launched its indexing API in 2018. Although IndexNow took a similar approach, Microsoft made it available to the masses instead of a small segment of the internet.
Google’s API is only for sites with job postings and live streams. In contrast, IndexNow supports all content types.
IndexNow accepts up to 10,000 URLs per API call, while Google only supports 100. Moreover, IndexNow’s implementation process is very simple, while Google’s process is significantly more complicated.
IndexNow vs XML Sitemaps
You update the sitemaps (XML sitemaps), ping the search engines, and wait for them to crawl the sitemap to discover the changes before crawling and indexing. This traditional approach seems to have begun to go out of fashion.
IndexNow solves this problem by showing search engines the information they need in advance. Search engine crawlers do not need to re-check sitemaps for up-to-date information.
And unlike XML sitemaps, which only help you to redirect crawlers to pages on your website, IndexNow allows you to notify search engines of page status codes other than 200, such as such as 403, 301 and 302.
But that doesn’t mean sitemaps no longer work. Bing and Yandex represent only a small portion of the internet. The search engine with the largest market share, Google, is still testing IndexNow and has not announced any plans to adopt it.
Google has the infrastructure needed to build an alternative to IndexNow. They just need to modify and upgrade it. Regardless of their decision, however, it is clear that a future where website indexing moves to an API-based approach is not far away.
Speed up website indexing with IndexNow
There are 2 ways to do this: manual and automatic.
Manual methods
1. Access the link The following, scroll down to Generate API Key, click the download icon to download the TXT file containing the API Key to your computer. You can also click the . button Generate to change another API Key. For example here I have API Key as d4170665a96444438e7c3533dbd04735.
2. Upload the TXT file containing the API Key to the root directory (where your website source code is installed). For the WordPress source code, this file must be on the same level as the file wp-config.php. Try accessing the file link to make sure it has been uploaded to the correct location.
Remember instead www.example.com by website link and d4170665a96444438e7c3533dbd04735 with your API Key. If done correctly, you will get the same result as the API Key created in step 1.
3. To submit a link of a page on your website to Bing, use a structure similar to the one below:
Here is the structure for Yandex:
Remember instead www.example.com/product.html and d4170665a96444438e7c3533dbd04735 with your site URL and API Key.
Automatic method
If your website is using CloudFlare CDN (with proxy enabled), all you need to do is access your CloudFlare account => select the correct domain name of the website => select the tab Caching => choose next tab Configuration => find and enable the feature Crawler Hints go up.
It’s simple, right? Good luck!
Are you using IndexNow tool in general and CloudFlare’s Crawler Hints feature in particular to speed up indexing data for your website? How do you rate their effectiveness and future? Let us know your opinion through the comment box below.
If you liked this article, follow my blog to regularly update the best and latest articles. Thanks very much. 🙂
