I solved (at least partly) my problem maintaining rankings over time. Old pages that used to get traffic are once again seeing inbound search visits. I did it by tweaking my blog’s category pages, but I think the lessons are also helpful for ecommerce sites.

By default, Wordpress category pages show 10 posts. Similarly, most ecommerce category pages show 9 products per page (in a 3×3 grid).

This sucks for SEO, because your individual blog posts or category products get less and less internal linklove over time as they get pushed to the secondary category pages.

There are two types of secondary category pages - the extremely deep ones that no one will ever get to because it takes 10 “next page” clicks to get there. For this problem, Egghead created Wordpress pagination. Adding “1,” “2,” “3,” … links is just a partial solution, though. And as Rand highlights in this fantastic Whiteboard Friday (prompted by a question of mine, I believe ), pagination is still not great.

Search engines don’t have a particular reason to crawl deeper than the main category page, as a general rule.

This is more true for ecommerce sites where the products tend to resemble each other (e.g. diamond rings) and a deeper crawl is unlikely to make the results returned to searchers significantly better. Search engines also have that incentive in a Wordpress blog context since a percentage of the newer posts likely rehash older ones, though the rehash is less pronounced relative to ecommerce sites.

Further compounding the problem is the fact, as I just recently found out, that All In One SEO - probably the single most popular Wordpress SEO plugin around - adds a meta robots noindex tag to your category pages! (Or perhaps it writes to robots.txt; I didn’t pay exact attention but that’s a technical detail that is beside the point.) If I were a search engineer, I would treat links from noindexed pages as less valuable than those from pages meant to be in my index. No, I haven’t run this test scientifically - this part is just a hunch.

The Takeaway Tips on Optimizing Archival Material:

1) Go to Settings (aka Options in older versions of Wordpress) - All In One SEO - and remove the checkmark next to ‘noindex category pages’.

2) Go to Settings - Reading. Set “Blog pages show at most: ” to 100. Rand suggests (in that Whiteboard Friday) that this won’t solve all your problems, because it doesn’t scale. So here’s how to scale it…

3) Ironically, the answer is in another SEOmoz post. Consolidate the repetitive long tail post selection by moving all your content on a narrow subject to one particular post. This will reduce the number of posts (and hence the depth to which spiders need to crawl) as well as make for a single, more comprehensive post.

That makes it more linkable, incidentally. And it should help bring in more longtail traffic too, due to the mix of previously absent words with those already present! While we’re getting all Wikipedia style, why not throw in a table of contents for good measure, since your post is going to be getting longer?

4) Take Michael Gray’s tip on putting posts in just one category (see the vid below). This will enable you to maximize the value of each category page in getting posts indexed (or each subcategory’s power, as Rand cleverly advises in the Whiteboard Friday video).

For example, I recently wrote that my idea for measuring social media results could be coded using the Friendfeed API. Well, the Friendfeed API blurb is going to be integrated into the main post as soon as I find time. Similarly, a number of my link buying posts can be simplified and consolidated.
The Ideal Category Archive SEO Solution:
Ajax - 100 posts can be intimidating, especially when that scroll button gets all tiny. Instead, why not have 100 posts loaded, but only 10 or 20 visible? To get to the others you click an AJAX link. Only when you get to say, the 100th or 200th post does a new page need to be loaded. This makes for decent usability while keeping things search engine friendly.
Value-add tagging and sorting system - Essentially, you’d be able to tag something on the backend on a scale of 1-5 on how useful a post is. Alternatively, you could base this on user ratings of how good a post was, but this depends on having a large enough user base to sustain this.
Then, users viewing your category archives could sort them by how valuable or popular an individual post is. This ’sort-by-importance’ view would be the default for search engines, ensuring your best content gets the most crawler attention.

What do you make of these ideas? Would you do any of this differently? Have additional tips on this? Why does All In One SEO noindex category pages by default, and likewise Joost’s robots plugin?

Gab Goldenberg writes on SEO at his SEO blog. Check out his services if you’re in the market for something professional.

I solved (at least partly) my problem maintaining rankings over time. Old pages that used to get traffic are once again seeing inbound search visits. I did it by tweaking my blog’s category pages, but I think the lessons are also helpful for ecommerce sites.

By default, Wordpress category pages show 10 posts. Similarly, most ecommerce category pages show 9 products per page (in a 3×3 grid).

This sucks for SEO, because your individual blog posts or category products get less and less internal linklove over time as they get pushed to the secondary category pages.

There are two types of secondary category pages - the extremely deep ones that no one will ever get to because it takes 10 “next page” clicks to get there. For this problem, Egghead created Wordpress pagination. Adding “1,” “2,” “3,” … links is just a partial solution, though. And as Rand highlights in this fantastic Whiteboard Friday (prompted by a question of mine, I believe ), pagination is still not great.

Search engines don’t have a particular reason to crawl deeper than the main category page, as a general rule.

This is more true for ecommerce sites where the products tend to resemble each other (e.g. diamond rings) and a deeper crawl is unlikely to make the results returned to searchers significantly better. Search engines also have that incentive in a Wordpress blog context since a percentage of the newer posts likely rehash older ones, though the rehash is less pronounced relative to ecommerce sites.

Further compounding the problem is the fact, as I just recently found out, that All In One SEO - probably the single most popular Wordpress SEO plugin around - adds a meta robots noindex tag to your category pages! (Or perhaps it writes to robots.txt; I didn’t pay exact attention but that’s a technical detail that is beside the point.) If I were a search engineer, I would treat links from noindexed pages as less valuable than those from pages meant to be in my index. No, I haven’t run this test scientifically - this part is just a hunch.

The Takeaway Tips on Optimizing Archival Material:

1) Go to Settings (aka Options in older versions of Wordpress) - All In One SEO - and remove the checkmark next to ‘noindex category pages’.

2) Go to Settings - Reading. Set “Blog pages show at most: ” to 100. Rand suggests (in that Whiteboard Friday) that this won’t solve all your problems, because it doesn’t scale. So here’s how to scale it…

3) Ironically, the answer is in another SEOmoz post. Consolidate the repetitive long tail post selection by moving all your content on a narrow subject to one particular post. This will reduce the number of posts (and hence the depth to which spiders need to crawl) as well as make for a single, more comprehensive post.

That makes it more linkable, incidentally. And it should help bring in more longtail traffic too, due to the mix of previously absent words with those already present! While we’re getting all Wikipedia style, why not throw in a table of contents for good measure, since your post is going to be getting longer?

4) Take Michael Gray’s tip on putting posts in just one category (see the vid below). This will enable you to maximize the value of each category page in getting posts indexed (or each subcategory’s power, as Rand cleverly advises in the Whiteboard Friday video).

For example, I recently wrote that my idea for measuring social media results could be coded using the Friendfeed API. Well, the Friendfeed API blurb is going to be integrated into the main post as soon as I find time. Similarly, a number of my link buying posts can be simplified and consolidated.
The Ideal Category Archive SEO Solution:
Ajax - 100 posts can be intimidating, especially when that scroll button gets all tiny. Instead, why not have 100 posts loaded, but only 10 or 20 visible? To get to the others you click an AJAX link. Only when you get to say, the 100th or 200th post does a new page need to be loaded. This makes for decent usability while keeping things search engine friendly.
Value-add tagging and sorting system - Essentially, you’d be able to tag something on the backend on a scale of 1-5 on how useful a post is. Alternatively, you could base this on user ratings of how good a post was, but this depends on having a large enough user base to sustain this.
Then, users viewing your category archives could sort them by how valuable or popular an individual post is. This ’sort-by-importance’ view would be the default for search engines, ensuring your best content gets the most crawler attention.

What do you make of these ideas? Would you do any of this differently? Have additional tips on this? Why does All In One SEO noindex category pages by default, and likewise Joost’s robots plugin?

Gab Goldenberg writes on SEO at his SEO blog. Check out his services if you’re in the market for something professional.