Duplicate content means that the same content is published on different URLs. This can hurt your search rankings because Google doesn't know which page is correct. In this article, we tell you all about duplicate content, show some of its most common causes, and tell you how you can fix it.
What is duplicate content?
Duplicate content means the same content is published on different URLs, either on the same website or on a different site. For example, www.example.com/t-shirts and www.example.com/t-shirts?sort=price may have different URLs, but their content is the same.
Why does duplicate content matter?
You don't want to see the same result twice if you search on Google. Therefore, Google will only show one result with similar content on the results page. Google is forced to make a choice: which content am I going to include in the results? Their choice may not be the one you would have liked to rank.
It gets worse: other websites may link to different versions of your content. Since links are an essential ranking signal for Google and other search engines, they won't know which content to prioritize. As a result, Google might distribute the PageRank (the authority of that page in Google's index) among the different URLs, lowering the overall rank.
Additionally, duplicate content can impact your crawl budget — the time and resources Google is willing to spend on indexing your website. Duplicate content makes it harder for them to crawl and index your website, and some other pages may not rank.
The result: confused search engines and lower rankings for your content. That's a shame because duplicate content is easy to avoid.
It’s often not easy to manage large quantities of duplicate content, especially on eCommerce sites. Products often have similar descriptions, leading to a high volume of repetition.
You can learn more about how Google handles duplicate content in this article by Neil Patel.
When does Google see content as duplicate?
Google has described duplicate content as "substantive blocks of content within or across domains that either completely match other content or are appreciably similar." Note that the content doesn't have to be the same to be considered duplicate.
What is a duplicate content penalty?
The good news is that there’s no duplicate content penalty. Google stated this in a blog post in 2008, and it’s still true, as Google's John Mueller confirmed in a 2014 Hangout session.
Google may issue penalties when content is intentionally copied (plagiarism), but that's not the same as duplicate content. Duplicate content is often unintentional and won't result in your website being removed, and you won’t receive a notification or alert from Google Search Console indicating that your site has been penalized for this reason.
When Google finds the same content on a site – or multiple sites – the search engine’s algorithm decides which content to rank. In some cases, Google seems to rank the wrong content due to a number of reasons, from mismatched intent to keyword cannibalization. This can lead to duplicate pages being filtered from the results, giving you less control over which pages are ranking. Google may also filter out redundant results from the search listings, prioritizing pages that provide unique and original information.
That’s enough reason to avoid duplicate content!
Is there an acceptable percentage of duplicate content?
You should aim for a duplication percentage that falls below 30%. So, at least 70% of your content should be original. Several online tools, such as duplicate content checkers and keyword density analyzers, can help you measure this. We’ll talk about them in just a moment.
Causes and solutions
There are many reasons why the same content may live on different URLs. We’ll walk you through the most common causes and how to fix them.
URL parameters
Often, you'll see parameters in URLs used for sorting, filtering, pagination, or recognizing where traffic comes from. For example, www.example.com/products?sort=price and www.example.com/products may be the exact same page but have different URLs. The same is true for tracking parameters: www.example.com/blog-post?utm_source=email may not differ from www.example.com/blog-post.
You probably can't remove these parameters because they are there for a reason. There's an easy fix for this: use canonical URLs. A canonical URL tells search engines that although various URLs may go to the same content, only one canonical URL is the original one. Generally, Google will use that URL in their results.
At the head of your page, add the following:
<link rel="canonical" href="http://www.example.com/blogs/my-blog-post" />
This tells Google that http://www.example.com/blogs/my-blog-post should be indexed, even if the URL shown is:
- http://www.example.com/blogs/my-blog-post?utm_source=email
- http://www.example.com/blogs/my-blog-post?show-comments=true&page=5
It's similar to a 301 redirect but without changing the URL.
Content in different categories
Some Content Management Systems (CMS) allow you to place products or blog posts in different categories. For example, a gardening webshop might list apple trees under www.example.com/trees/apple-tree and www.example.com/fruit/apple-tree. As a result, the product page is available on two different URLs – duplicate content!
Product pages with duplicate content might rank initially but risk losing their ranking if search engines can't determine which version to prioritize. Handling this issue is crucial, especially for larger sites where product similarity can worsen things.
But here’s some good news: Optimized category pages rank better and can drive conversions, even if some product pages still have duplicate content.
Here’s how you can manage it:
Optimizing Category Pages
When dealing with multiple URLs for the same product, prioritizing high-quality, unique content on your category pages is key. These pages often serve as a first point of contact for visitors and are crucial for search engine rankings. Instead of relying on duplicated content across various product pages, ensure your category pages stand out with well-optimized content. This can both mitigate the effects of duplicate content and improve conversion rates.
If resources are limited, rewriting duplicate content on category pages is even more important since properly optimized category pages can improve your chances of ranking and converting visitors.
Address CMS-generated duplicates
Now, Content Management Systems (CMS) often contribute to duplicate content issues by generating multiple URLs for the same product when it's listed in different categories. For instance, a product like an apple tree could be found under both www.example.com/trees/apple-tree and www.example.com/fruit/apple-tree, which confuses search engines and dilutes ranking signals.
To solve CMS-generated duplicates, here are two possible solutions
- Ensure the product page uses the category of the most important page: Always assign the product to the most relevant category, even if it could fit in multiple. This prevents search engines from indexing duplicate pages.
- Use a canonical URL: If a product must exist in multiple categories, use canonical URLs to indicate the primary version to Google to maintain proper SEO rankings.
Avoid automated solutions
While automated tools might seem like a quick fix to handle duplicate content, they often create poorly optimized, hard-to-read pages that can damage your site's rankings.
Instead, invest in human writers to craft unique, engaging content, starting with your most critical pages, like the category pages mentioned above.
Avoid content scraping issues
Scraping, where other websites copy your content and present it as their own, is more common than you might think. This dilutes your brand and competes with your original content for search engine placement. Again, regular checks can help identify and address these issues right away.
Implement incrementally
If your site has thousands of pages, rectifying all duplicate content might feel overwhelming. Start with the most important pages—often the category pages—and gradually work through the rest. Despite limited resources, focusing on quality content in priority areas will yield noticeable improvements.
Pagination may result in duplicate content
You might run into duplicate content issues if your site has multiple blog or product listing pages. For example:
- www.example.com/blog/ "Awesome SEO blog"
- www.example.com/blog/page/2 "Awesome SEO blog"
In some CMS platforms like WordPress, titles and meta descriptions are often auto-generated, leading to identical titles and descriptions across different pages.
To avoid this, simply add a page number to the title:
- www.example.com/blog/ "Awesome SEO blog"
- www.example.com/blog/page/2 "Awesome SEO blog - page 2"
Unoriginal content
Say you're selling products, and you use the description provided by the manufacturer on the product item page. There's a good chance that many of your competitors are doing the same. As a result, the content on your product page is hard to distinguish from your competitors.
We can’t stress enough that you should write your own content or at least adjust the provided texts so that they speak to your audience. That way, you not only avoid duplicate content but also make sure that your audience is targeted with text written just for them instead of generic descriptions everyone uses.
Pay close attention to guest posts
Imagine you can write a guest post on a big blog in your industry. That's pretty cool! But what if you wanted to post that same article on your own blog? Now, you have two different URLs with the same content, even in other domains.
Canonical URLs, again, are the solution. If you can, ask the blog owner to include a canonical URL to your page's blog post. That's a strong signal that yours is the original.
A note on canonical URLs
Keep in mind that if you have page A with a canonical URL pointing to page B, page A will probably not be indexed. Often, that is what you want, but make sure you take good care when placing canonical URLs because the effects can be serious.
Got country-specific domains?
Say you have www.example.com targeting the US and www.example.co.uk targeting the UK. Both websites sell kitchen apparel and have identical product descriptions. But because the pricing and delivery costs differ, you want to make sure you're sending the right people to the right website, and of course, you want to avoid duplicate content.
Href lang attributes are the answer here. They tell Google which page targets which country so that Google can display the .com website to US searchers and the .co.uk website to people from the UK.
www and non-www URLs
Some websites have www in front of their domain, like www.google.com. Others don't, like dribbble.com. If your website works on the domain with and without www, you have two identical websites on different URLs. Google will consider that duplicate.
There's an easy fix: redirect all your traffic to www. If you have an Apache server, add this to your .htaccess file:
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]
https and http URLs
So, you secure your website with an SSL certificate? That's great! Just don't forget to redirect all traffic to that secure URL, otherwise, your content will live on 2 URLs: one with and one without SSL.
If you're on an Apache server, you can do this by adding the following lines to your .htaccess file:
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
Trailing slashes
You see this often: www.example.com/products and www.example.com/products/ (note the trailing slash at the end) show the same page. Google is getting smart enough to realize this is probably the same page, but most SEOs agree it's not worth the risk. It's much better to redirect all traffic to the URL without the trailing slash.
If your website is running on an Apache server, add the following line to your .htaccess to redirect all traffic to the variant without the trailing slash:
RewriteRule ^/?(.+)/$ /$1 [R=301,L]
Boilerplate content
When we talk about content, we normally refer to the text in your blog post, news article, or product description. But there's more content on your page: you have a menu, a header, a footer, and maybe even a sidebar that you show on every page across your site. That's what we call boilerplate content.
If you have a lot of boilerplate content on your page compared to the specific content of that page, Google may view these pages as duplicate. The result is pretty serious: it may not show your individual product pages in the search results. Therefore, Google recommends keeping your boilerplate content to a minimum.
Of course, you'll need a menu and a footer. In the footer, don't include your entire privacy statement. Instead, add a link to a specific page.
How to find and monitor duplicate content
Checking for duplicate content regularly is essential because new pages, updates, or site changes can accidentally create duplications over time. Regular audits ensure you maintain control over your indexed pages and avoid unintentional penalties or ranking issues caused by duplicate content.
Here’s how you can easily keep everything in check:
- Use tools like Google Alerts to notify you when your content appears elsewhere on the web.
- Tools like SiteGuru can help you identify duplicate content quickly with a recently added AI-powered feature called Content Similarity Report. By analyzing the actual meaning behind your pages, this report detects when content overlaps too much, leading to potential internal competition. Unlike so many similar tools, this one doesn’t just look for identical meta descriptions or keywords, but rather it digs into the content itself. Whether it suggests merging similar pages or revamping certain sections, it empowers you to optimize your site structure, so that each page stands out and delivers the best possible results for your SEO strategy.
- Last but not least, schedule regular content audits to review your website and ensure originality across all pages.
Conclusion
Duplicate content can negatively impact how your website is indexed and ranked. Thankfully, it's easy to find and fix, so there’s no reason to have duplicate content on your website!
By tackling duplicate content methodically and focusing on quality, you can improve your SEO rankings and ensure your content stands out to search engines and visitors.
Understand that fixing thousands of pages of duplicate content is a time-consuming process. However, creating unique, authoritative content for each page is crucial for long-term success. And it can be a lot quicker with the right tools!
