Curious why pagination was set to noindex, nofollow in 1.6.2 when Google seems to say otherwise?

In the latest 1.6.2, I’m confused by the default changes to html_meta_tags and sitemap_extended supposedly for SEO best practices, when they seem to go against Google’s advice.

Paginated pages are set to noindex and nofollow, when Google does use them.

html_meta_tags:

  • Set robots to noindex, nofollow for paginated pages by default to follow SEO best practices and add options to allow otherwise [acrylian]

Google says it indexes paginated pages, as long as they are setup using <a href> tags.

For example, you can implement pagination using links to new pages on your ecommerce site, or using JavaScript to update the current page. Load more and infinite scroll are generally implemented using JavaScript. When crawling a site to find pages to index, Google only follows page links marked up in HTML with <a href> tags.

Support is also added for <link rel="prev/next"> when Google no longer uses it.

Add suppprt and options for <link rel="prev/next"> for gallery and news pagination and for single image pages [acrylian]

Google-

In the past, Google used <link rel="next" href="..."> and <link rel="prev" href="..."> to identify next page and previous page relationships. Google no longer uses these tags, although these links may still be used by other search engines.

Also for sitemap_extended:

Paginated pages are excluded by default from sitemap to follow general SEO best practice but options have been added just in case it makes sense for individual usages and fix variable type issue [acrylian]

Again, I don’t see any clear guidance on this. I see some old SEO advice saying not to do it, but plenty of other posts and SEO sites saying it is fine to include paginated pages.

It looks like it started with this bug report. I know that the defaults can be changed, and they seem to work, so this is not a real issue. More just curious on why it was done?

Comments

  • acrylian Administrator, Developer

    The exclusion of paginated pages by default was indeed a mistake. It was sometimes said due to the changing contents of paginated pages this can cause trouble. Google has a hard time (in my observation) to properly index sites that change a lot or even daily.
    It may indeed make or not make sense to exclude them depending on the site itself. I will change the default for the next version.

    prev/next is not supported by google but it may still be an indicator. And I don't know perhaps other search engines - unbelieveable but there is more than google ;-) - do or even assistive tools may use it).

    Regarding sitemaps I actually followed what a very widely used SEO plugin for a very widely used CMS does.

    But for different needs options to enable/disable were added.

  • kuzzzma Member
    edited April 19

    Finally got to this topic & checking html_meta_tags settings and code.
    Currently (ver. 1.6.3), $indexing_allowed = false creates generation of

    meta name="robots" content="noindex,nofollow"

    For SEO it may make sense to limit indexing and serving paginated pages in search results in favour of actual albums/images/pages/news items, as they will be available on permanent basis (for example, news item will eventually move to a new page in news category as you add more stuff) + it helps with a problem of duplicate content.

    The problem is including nofollow too - it prevents robots from using those paginated pages to discover new links to actual photos/albums/news/pages, which we actually want to happen ALWAYS.

    Also description for this setting doesn't mention nofollow, which is misleading.

  • acrylian Administrator, Developer

    We changed the default option to enabled already in 1.6.3a which of course does not change anything for existing usages.

    Good point about the nofollow actually. Will remove that.

  • acrylian Administrator, Developer

    I have reworked html_meta_tags once again regarding this in 1.6.3a. The options are now renamed and now are opt-in for disallowing indexing and also set a proper "noindex, follow" in that case. Also I added - inspired by the Paradigm theme - some options for search and some common static custom pages.

Sign In or Register to comment.