“URL Parameters helps you control which URLs on your site should be crawled by Googlebot, depending on the parameters that appear in these URLs,” explains Kamila Primke, Software Engineer with the Google Webmaster Tools Team. “This functionality provides a simple way to prevent crawling duplicate content on your site. Now, your site can be crawled more effectively, reducing your bandwidth usage and likely allowing more unique content from your site to be indexed. If you suspect that Googlebot’s crawl coverage of the content on your site could be improved, using this feature can be a good idea. But with great power comes great responsibility! You should only use this feature if you’re sure about the behavior of URL parameters on your site. Otherwise you might mistakenly prevent some URLs from being crawled, making their content no longer accessible to Googlebot.”

Google is now letting users describe the behavior of parameters. For example, you can let Google know if a parameter changes the actual content of the page.

“If the parameter doesn’t affect the page’s content then your work is done; Googlebot will choose URLs with a representative value of this parameter and will crawl the URLs with this value,” says Primke. “Since the parameter doesn’t change the content, any value chosen is equally good. However, if the parameter does change the content of a page, you can now assign one of four possible ways for Google to crawl URLs with this parameter.”

Those would be: let Googlebot decide, every URL, only crawl URLS with value or no URLs.

Users can tell Google if a parameter sorts, paginates, determines content, or other things that it might do. For each parameter, Google will also “try” to show you a sample of example URLs from your site that it has already crawled that contain a given parameter.

To bring up the use of caution again, Primke warns about the responsibilities that come with using the No URLs option. “This option is the most restrictive and, for any given URL, takes precedence over settings of other parameters in that URL. This means that if the URL contains a parameter that is set to the ‘No URLs’ option, this URL will never be crawled, even if other parameters in the URL are set to ‘Every URL.’ You should be careful when using this option. The second most restrictive setting is ‘Only URLs with value=x.’”

She runs through some examples in this blog post, and there is more related information in Google’s Webmaster Help forum.

Be Careful About Selling the Same Stuff From Multiple Domains

As long as we’re discussing webmaster issues for Google, I’ll also point to the latest Webmaster Help video from Matt Cutts, who discusses selling products on multiple domains. The user question he sought to answer was:

“I manage 3 websites that sell the same products across 3 domains. Each site has a different selling approach, price structure, target audience, etc. Does Google see this as spumy or black hat?”

Cutts says, “On one hand, if the domains are radically different lay-out, different selling approach, different structure – like, essentially completely different, and especially the fact that you said it’s only 3 domains, that might not be so bad. Clearly if it were 300 domains or 3,000 domains – you can quickly get to a fairly large number of domains that can be crowding up the search results and creating a bad user experience…by the time you get to a relatively medium-sized number of sites.”

“The thing that was interesting about the question is that you said it’s the same products, as in identical. So it’s a little weird if you’re selling identical products across 3 domains. If you were selling like men’s sweaters on one, and women’s sweaters on another, and shoes on a third….I’ve said before, there’s no problem with having different domains for each product, and a small number of domains (2, 3, or 4) for very normally separable reasons can make perfect sense, but it is a little strange to sell the same products, so if they’re really identical, that starts to look a little bit strange – especially if you start to get more than 3 domains.”

“Definitely, I have found that if you have one domain, you’ve got the time to build it up – to build the reputation for that domain…in my experience, when someone has 50 or 100 domains, they tend not to put as much work – as much love into each individual domain, and whether they intend to or not, that tends to show after a while. People have the temptation to auto-generate content or they just try to syndicate a bunch of feeds, and then you land on one domain vs. another domain, and it really looks incredibly cookie cutter – comparing the two domains, and that’s when users start to complain.

The company said they would only be used in place of some URLs, mainly ones that don’t give the added context of a link the way that breadcrumbs do. Interestingly, there seems to be an incentive for those who go the breadcrumb route because of the multiple links that you just don’t get with regular search results.

Google’s move was generally well received. For example, a commenter going by the handle Stupidscript said, “It’s definitely a good time to start wrapping your head around the notion of ‘providing context’, because the web is heading into its “semantic” period … where each link will be more or less valuable based on its relationships with and context to information found behind other links.”

Google’s use of breadcrumbs in search results is the focus of a recently submitted question to the Google Webmaster Central team. The question was, “Google is showing breadcrumb URLs in SERPs now. Does the kind of delimiter matter? Is there any best practice? What character to use is best? > or | or / or???” Google’s Matt Cutts responded:

Matt says you should have a set of delimited links on your site that accurately reflect your site’s hierarchy. He also notes, however, that it is still in the “early days” for breadcrumbs.

“Think about the situation with sitelinks,” he says. “Whenever we started out with sitelinks, it took a while before…for example, we added the ability in Google Webmaster Tools where you could remove a sitelink that you didn’t like or that you thought was bad. So we started out, and we did a lot of experiments, and we’ve changed the way that sitelinks look several times. And we have different types of sitelinks (within a page, and the standard ones you’re familiar with). So we’ve iterated over time.”

In this same way, he says, Google is in the early stage with breadcrumbs and he has seen different experiments with them. For example, there have been prototypes where the breadcrumbs were in the rich snippet gray line, above the regular snippet. “Having it in the URL is kind of nice, but it could still change over time,” he says.

He says the best advice he can give is to make sure you have a set of delimited links that accurately reflect your site’s hierarchy, and that will give you the best chance of getting breadcrumbs to show up in Google, but Google will continue to work on ways to improve breadcrumbs. He says any new announcements about it will likely be made on the Google Webmaster blog.

While Matt doesn’t exactly lean toward one way or another with regards to which character to use as asked about in the submitted question, all of the examples I have seen highlighted show the “>” used. That includes examples from Google’s original announcement on the inclusion of breadcrumbs (if you see other ways, please point them out in the comments). Based on that, if I were going to choose one, I’d go with that.

“One of the great things about working at Google is that we get to take advantage of an enormous amount of computing power to do some really cool things,” says Patrick Chapman of Google’s Search Quality Team. “One idea we tried out was to let webmasters know about their potentially hackable websites. The initial effort was successful enough that we thought we would take it one step further by expanding our efforts to cover other types of web applications—for example, more content management systems (CMSs), forum/bulletin-board applications, stat-trackers, and so on.”

“This time, however, our goal is not just to isolate vulnerable or hackable software packages, but to also notify webmasters about newer versions of the software packages or plugins they’re running on their website,” he continues. “For example, there might be a Drupal module or Joomla extension update available but some folks might not have upgraded.”

One way Google identifies sites that need to be notified is by parsing source code of web pages that it crawls. Content management systems usually include generator meta tags that specify the version number. This helps Google figure out when sites aren’t running the latest version.

The company is encouraging developers to use such generator meta tags in their software if they are not already doing so. This, along with Google’s efforts, should theoretically keep their users using the most recent versions. That is Google’s goal, anyway.

Google says they will begin sending out notifications soon, but doesn’t give a more specific time. Do you think this is a good idea?