Bing Revamps Crawl System to Enhance Search Engine Efficiency. In the ongoing quest to deliver the most relevant and accurate search results, Bing has announced a revamp of its crawl system. This new system will allow for more efficient and effective indexing of the vast amount of content available on the web, providing users with even more accurate and relevant results.
To address this issue, Bing is revamping its crawl scheduling stack to better utilize the information provided by the “lastmod” tag in sitemaps. This will improve crawl efficiency by reducing unnecessary crawling of unchanged content and prioritizing recently updated content.
According to a recent study by Bing, most websites have XML sitemaps, with the “lastmod” tag being the most critical component of these sitemaps.
The “lastmod” tag indicates the last time the webpages linked by the sitemap were modified and is used by search engines to determine how often to crawl a site and which pages to index.
However, the study also revealed that a significant number of “lastmod” values in XML sitemaps were set incorrectly, with the most prevalent issue being identical dates on all sitemaps.
How To Use The Lastmod Tag Correctly
To correctly set the “lastmod” tag in a sitemap, you should include it in the tag for each page in the sitemap.
The date should be in W3C Datetime format, with the most commonly used formats being YYYY-MM-DD or YYYY-MM-DDThh:mm:ssTZD.
The date should reflect the last time the page was modified and should be updated regularly to ensure that search engines understand the relevance and frequency of updates.
Here’s an example code snippet:
<?xml version=”1.0″ encoding=”UTF-8″?>
<urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>
<url>
<loc>http://www.example.com/</loc>
<lastmod>2023-01-23</lastmod>
</url>Google’s Advice: Use Lastmod Tag After Significant Changes Only
Google’s crawlers also use the “lastmod” tag, and the suggestions on using it by both major search engines are similar.
Google Search Advocate John Mueller recently discussed the lastmod tag in the January edition of Google’s office-hours Q&A sessions.
It’s worth noting that Google recommends only using the “lastmod” tag for substantial modifications, which was not mentioned in Microsoft’s blog post.
Changing the date in the lastmod tag after minor edits can be viewed as an attempt to manipulate search snippets.
The Importance of a Good Crawl System
A search engine’s crawl system is the backbone of its ability to index and rank websites. The crawl system is responsible for discovering and collecting information about new pages and updates to existing pages, so that this information can be added to the search engine’s index and used to rank websites in search results.
In order to deliver the best possible search results, a search engine must have a crawl system that is both efficient and effective. This requires the crawl system to be able to discover new and updated pages quickly, collect all relevant information about those pages, and add that information to the search engine’s index in a timely manner.
Bing’s Revamped Crawl System
Bing’s new crawl system is designed to do just that, delivering even more accurate and relevant results to users. The new system will allow Bing to index an even greater number of pages, while also collecting more information about each page.
One of the key improvements to the new crawl system is the ability to better handle JavaScript and other dynamic web technologies. This will allow Bing to crawl and index more pages that are built using these technologies, providing users with even more relevant and accurate results.
Another improvement to the new crawl system is the ability to prioritize pages based on their importance. This means that the most important pages on the web will be crawled and indexed first, ensuring that users are presented with the most relevant and accurate results.
The Benefits of the New Crawl System
The new crawl system will provide users with a number of benefits, including:
- More accurate and relevant search results
- Faster indexing of new and updated pages
- More comprehensive indexing of dynamic web pages
- Better handling of large and complex websites
Conclusion
The revamp of Bing’s crawl system is an important step forward in the ongoing quest to deliver the most relevant and accurate search results to users. By allowing for more efficient and effective indexing of the web, Bing is positioning itself to stay ahead of the competition in the constantly evolving world of search engine technology.
