eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Google is advising Web developers to double-check the performance of their Web pages after Google made some recent changes to its page indexing and page rendering systems involving JavaScript.
The issue was raised by Google software engineers Erik Hendriks and Michael Xu and Google Webmaster trends analyst Kazushi Nagayama in a May 23 post on the Google Webmaster Central Blog.
“In 1998 when our servers were running in Susan Wojcicki‘s garage, we didn’t really have to worry about JavaScript or CSS,” wrote Hendriks, Xu and Nagayama. “They weren’t used much, or, JavaScript was used to make page elements … blink! A lot has changed since then. The web is full of rich, dynamic, amazing websites that make heavy use of JavaScript.”
In the past, Google Search has been “only looking at the raw textual content that we’d get in the HTTP response body and didn’t really interpret what a typical browser running JavaScript would see,” they wrote. “When pages that have valuable content rendered by JavaScript started showing up, we weren’t able to let searchers know about it, which is a sad outcome for both searchers and webmasters.”
In response, Google Search began to try to understand pages by executing JavaScript, they wrote. “It’s hard to do that at the scale of the current Web, but we decided that it’s worth it. We have been gradually improving how we do this for some time. In the past few months, our indexing system has been rendering a substantial number of web pages more like an average user’s browser with JavaScript turned on.”
That step, however, can affect how pages are then seen by Google’s search engine, they wrote.
“Sometimes things don’t go perfectly during rendering, which may negatively impact search results for your site,” they wrote. Several issues have come up that need to be analyzed by Webmasters so that their page rankings are not harmed.
One of the problems, wrote Hendriks, Xu and Nagayama, is that if resources like JavaScript or CSS in separate files are blocked (with robots.txt or similar means) so that Googlebot can’t retrieve them, Google’s indexing systems won’t be able to see the site like an average user would see it.
“We recommend allowing Googlebot to retrieve JavaScript and CSS so that your content can be indexed better,” they wrote. “This is especially important for mobile websites, where external resources like CSS and JavaScript help our algorithms understand that the pages are optimized for mobile.”
In addition, if a business Web server “is unable to handle the volume of crawl requests for resources, it may have a negative impact on our capability to render your pages,” they wrote. “If you’d like to ensure that your pages can be rendered by Google, make sure your servers are able to handle crawl requests for resources.”
Google Advising Web Developers on JavaScript Page Indexing Issues
Another important step to consider is giving your site the ability to work so “users enjoy your content even if their browser doesn’t have compatible JavaScript implementations,” they wrote. “It will also help visitors with JavaScript disabled or off, as well as search engines that can’t execute JavaScript yet.”
Other JavaScript issues can include code that “may be too complex or arcane for us to execute, in which case we can’t render the page fully and accurately,” wrote Hendriks, Xu and Nagayama. “Some JavaScript removes content from the page rather than adding, which prevents us from indexing the content.”
Google is working on a tool for developers that will help them better understand how Google renders their sites, according to Hendriks, Xu and Nagayama. “We look forward to making it available for you in the coming days in Webmaster Tools.”
Google is often working to help Webmasters and companies improve their websites for more interactions with users and customers.
Earlier in May, Google advised companies that they should tailor their websites for customers who use other languages if they want to increase their international sales and footprints, according to an earlier eWEEK report. To do that, Google created a helpful list of suggestions that business can use to add foreign language capabilities for their sites, especially their home pages, as well targeting specific content for users in other countries.
In April 2014, Google refreshed its free online Webmaster Academy course to provide even more tips and information to beginning webmasters to help them improve their website skills. The new Webmaster Academy includes expanded information on how beginning webmasters can build great websites that are valuable and attractive to users, as well as details about how to build sites that will be seen by Google’s search engine before displaying them to users in search. The course was started two years ago.
In March 2014, Google unveiled a new Google Maps Embed API that aims to make it easier for website designers to place detailed Google Maps into Web pages so that customers can locate their physical locations more easily. The new API improves on a previous move by Google in December 2013, when the search giant began allowing Website owners and bloggers to embed and use Google Maps images for free.
In December 2013, Google provided a checklist for website developers to help them ensure that the sites that they build and maintain are always optimized to work well on mobile devices so that sales can be closed and revenue can be generated. The checklist includes relevant studies and business cases as well as videos and slides explaining how to use Google Analytics and Webmaster Tools to understand mobile visitors’ experiences and intent, according to an eWEEK report.