But even if you create a stellar page that perfectly matches your visitor's search intent, it needs to be indexed correctly to rank in the SERPs.
If you're wondering, "How do I index my website on Google?” this article will go over the potential issues that might be getting in the way, as well as how to improve your SEO indexation.
Let's dive in!
Why Do Problems With Indexation Occur?
Just because a page is indexable doesn’t mean it’s indexed.
Certain issues can prevent Google from crawling and indexing pages properly, including:
- Meta tags are blocking the crawling and indexing process.
- No internal or external links point to the page, making it an orphan page.
- The page is considered a duplicate.
- Google doesn't think the content is relevant.
Fortunately, you can fix all of these issues. Go through the handy list below to troubleshoot and solve the most frustrating indexation-blocking errors!
How to Index Your Website on Google
Check the Indexation Status of Your Pages
First things first, you should assess whether your website or page is indexed or not.
I’ll show you two methods: using SiteGuru and using Google Search Console.
Checking Indexation with SiteGuru
Head to the “Technical” tab ? Indexation.
You’ll see a brief overview of all the issues, as well as a detailed breakdown of pages which haven’t been indexed (with reasons).
You can click through on each page to see a detailed SEO report:
How to Check Indexation in GSC
You can use Google Search Console's Pages report to learn the index status of your website or individual pages by navigating to Google Search Console ? Index ? Pages (previously "Coverage").
You'll see an overview page showing indexing errors on your website (if any), indexed pages, and indexed pages with warnings.
Or, if you'd like to quickly check whether a specific page is indexed, GSC can also help. You'll use the URL Inspection Tool (below "Overview").
Paste the URL into the Inspect search box and press enter.
If that page is indexed, it’ll say: “URL is on Google.” If the page is on Google but has issues, it'll tell you. If the page isn’t indexed, it'll say: “URL is not on Google.”
Prevent Crawl Blocks by robot.txt
Not all pages are meant to be crawled and indexed, such as login pages and staging environments. It's okay to use robots.txt directives to block those pages from search engine crawlers.
However, there’s a problem if a page is supposed to be indexed, but it's still blocked by robots.txt.
Before crawlers can crawl a page, they will read through a robots.txt tag. If the tag doesn’t contain any "disallow" rules, the crawler will go on its merry way.
However, if it does contain a "disallow" rule, you're instructing it not to crawl this URL.
SiteGuru will notify you of this type of indexation issue, so recheck your robots.txt file for any “disallow” rules.
Fix NoFollow Internal Links
NoFollow links tell search engines that a link shouldn’t be crawled. And when a link isn't crawled, it won’t pass on its authority or influence search engine rankings.
That's why you should ensure that all internal links to indexable pages are set to "DoFollow.”
To check if a link is NoFollow, right-click anywhere on a page and navigate to “View page source.”
In the HTML version of the page, look for a rel=”nofollow” attribute to the link you want to check. If you find it, that means the link is NoFollow.
If you don't see a "rel" attribute, it’s a Followed link, and you're good to go!
To solve this specific issue, remove the NoFollow tag from these links.
Identify Orphan Pages
Pages without any internal links pointing to them are called orphan pages. These pages are less likely to rank high in search engines because, as you know, strong backlinks pointing to a particular page indicate high authority.
Then, when you connect your pages internally, the “link juice” is distributed evenly.
Not to mention, orphan pages are hard to find (to both searchers and Google), so ensure that every page on your website has at least one link pointing to it.
You can also use internal links to signal to Google that a page is important. Simply point more links to it.
The “Internal links” report in SiteGuru helps you find any linkless pages at a glance. You can find it at Links ? Internal links.
Increase Page Speed
By this point, you're already familiar with the adage: the faster your page is, the better your user experience and rankings.
You can check your URLs on PageSpeed insights to view their performance.
Or, you can save tons of time using SiteGuru’s Performance Report, which uses PageSpeed Insights data to scan hundreds of pages simultaneously.
The report, located at Technical ? Pagespeed, will give you the Pagespeed Index, Largest Contentful Paint, and the Cumulative Layout Shift score for each page.
But it’ll also show you immediate improvements that will help increase the speed of each page.
You can also view your PageSpeed Insights directly by clicking the Pagespeed Report link.
Remove Rogue noindex Tags
The noindex metatags can block search engines from indexing your page. While they allow Google to crawl noindex pages, they won't be indexed and will no longer get traffic from search engines.
An example of a noindex tag includes the X-Robots-Tag.
For instance, if you include the following HTTP header in your webpage:
X-Robots-Tag: noindex
It indicates to search engines that the page's content shouldn’t be indexed in search results. Accidentally noindexing a page will yield the same effect, so remove any rogue noindex tags from any pages you want to rank.
SiteGuru's Indexation Report will highlight which of your pages are showing as "noindex," so you can fix the problem immediately.
Remove Rogue Canonical Tags
Adding self-referencing canonical tags to URLs helps you avoid duplicate content by specifying the preferred URL version for search engines to index.
They come in handy when you have different versions of similar pages.
For example, here are two pages considered similar:
http://techwebsite.com/smartphones?page=2
http://techwebsite.com/smartphones?page=3
So, how would you prioritize one of the above URLs by adding a canonical tag?
It's simple. In the header of your page, add the following:
<link rel="canonical" href="http://techwebsite.com/smartphones?page=2">
The URL above would be the preferred version of your page – the one you want Google to index and display in the search results.
You can use SiteGuru's canonical URLs report. Navigate to Technical ? Canonical URLs to spot the rogue canonicals across your website.
Add the Page to Your Sitemap
Since sitemaps tell Google which pages on your site are important to you and which aren’t, their lack could slow down your crawling and indexing.
(That's also why you should make sure every URL in the sitemap is worth indexing.)
By submitting your sitemap, the search engine will know which pages you need it to crawl ASAP.
You can easily do it by visiting the Technical ? Sitemaps section in SiteGuru:
The sitemap you submit through SiteGuru will sync with the Google Search Console data.
And if you’re unsure how to do it, simply click the “Generate full sitemap” button, and SiteGuru will get your sitemap ready for you!
You’ll also see if there are any pages missing from your sitemap.
In GSC, you can check the status of your pages with the following process:
- First, navigate to “Sitemaps.”
- Next, select the submitted sitemap.
- Then, click “See Index Coverage.”
As a result, GSC will show you if some pages haven’t been indexed.
Don't Waste Your Crawl Budget!
Smaller websites that only feature a handful of pages shouldn't worry about their crawl budget. On the other hand, huge websites with thousands of pages can exhaust their budget quickly if they aren't careful.
Google's crawling isn't a one-and-done job. It often re-crawls it to refresh the page with any updates and changes.
That can be a problem for big websites: the search engine could be wasting too much time on low-priority pages while your top pages are left on the back burner. And if you’re in eCommerce, implement the AJAX navigation so you don’t create duplicate pages whenever someone runs a filtered search.
Use the tips in this article to ensure your top-priority pages get indexed and ranked faster. For example, exclude the low-priority pages from the sitemap, or add robots.txt directives that omit the pages you don’t want to be indexed.
The Crawl Stats report in Google Search Console is the unsung hero of this section. It lurks in the Settings tab and provides plenty of crawling information.
Remove Broken Links
Broken or "dead" links (especially internal) lead to a poor user experience, disrupt navigation???????, and make it hard for Google to crawl and index your pages.
You can spot them in the “404 - Not found” section of your Indexation Report in Google Search Console or get a notification from SiteGuru.
SiteGuru’s broken links report lives in the Links ? Broken links section:
Build Strong Backlinks
Backlinks (also called inbound links or incoming links) are like a "vote of confidence" from one website to another.
Because backlink-earning sites have so much potential to provide valuable and authoritative content, web crawlers tend to index those sites more often.
The more (high-quality) backlinks pointing to your website, the more importance Google will give it. I've emphasized "high-quality" because this doesn't apply to spammy backlinks.
Quality over quantity, always!
More about backlinks and how to build backlinks
You've Done Everything In Your Power, and Your Page Still Isn't Indexed?
You've checked all the boxes, and there are no technical issues.
In this case, ask yourself if your content satisfies the user intent and brings real value to its audience.
Sometimes you'll know the answer, and other times you won't. That's when you turn to data.
The best and quickest way to find potentially low-quality pages is by conducting a thorough SEO content audit on your website.
With SiteGuru, you can perform a content audit in a few simple steps and monitor all other on-page issues.
Get Indexed So You Can Get Ranking
That's a massive checklist, I know. So if you’re strapped for time, consider getting automated, weekly SEO audits through SiteGuru. This way, you can jump in only when it spots SEO issues or opportunities.
Only by optimizing SEO indexation can website owners enter the race for top-ranking spots.
Don't delay your SEO efforts. Start ranking right away!
