Technical SEO

Crawler

Search engines crawl the internet to find meaningful sites to display in search results. One example is the GoogleBot.

A crawler follows the links on your site to discover new pages, but also used the sitemap, redirects and canonical URLs to get the full picture. SEO tools like SiteGuru also use crawlers to find pages on a website.

Back