vortialbum.blogg.se - Screaming frog seo spider linux

SCREAMING FROG SEO SPIDER LINUX FULL
SCREAMING FROG SEO SPIDER LINUX SOFTWARE
SCREAMING FROG SEO SPIDER LINUX PASSWORD

Meta Description and length: shows the meta-description content and its lenght.

Title and length: displays URL's title and the number of characters it contains.

Thanks to that, we can see what page are not found (404), or which have server error (500). Status code: reports HTTP status code.This list includes the following information:

SCREAMING FROG SEO SPIDER LINUX SOFTWARE

Once the spider has finished to crawl all webpages, the software generates a list of each of the pages inspected. Perhaps our website generate tens of thousands of pages, but if we set up correctly noindex and rel="canonical" metatags, the number of indexed pages will be reduced considerably, thus contributing to increase internal PR of important pages.

SCREAMING FROG SEO SPIDER LINUX FULL

Respect Canonical: if we check this option, the spider will not crawl those pages containing rel=”canonical” meta tag.Īctivating the option of Respect noindex and Respect Canonical, we will have a full report of pages that will be indexed by search engines, since our spider will behave according to the rules that follow the majority of Internet search engines.

It's advisable to use this option if we want to know the total number of pages indexed in search engines like Google.

Respect noindex: if we mark this option, the spider will not crawl those pages with the “noindex” meta tag.

SCREAMING FROG SEO SPIDER LINUX PASSWORD

Request authentication: if any of the web pages that are part of the site is password-protected, by checking this option, the program will ask us to enter username and password to access and analyze protected page.Limit depth search: with this option we can set the spider to only crawl few clicks away from home page.Limit total number of pages to crawl: with this option we will limit the number of pages that will be crawled by the “ spider”.If we want our spider ignore that file and inspect all areas of the website we need to check this option. Ignore robots.txt file: if we have blocked certain areas of our site by using the file robots.txt.Crawl subdomains: if our site has multiple subdomains, and we want to “ spider” it, we need to check this option.This option is very useful if we want to know the total number of “ dofollow” pages that our site contains. Follow “nofollow” internal or external links: with this option the spider will follow “ nofollow” links or ignore them.Check External Links: if our site has links to other sites, this option will check that these links are not broken.Check Images, CSS, Javascript, SWF: this option enables the program to check files CSS, JS, etc, on the web pages, reporting a any broken link found.Some of the options that we can set are the following: The first thing we must do is to set the “ spider” (this is how the name for the program that will visit and collect information from the web pages that make up the site). After a minute or hours, depending on the size and depth of the site, we will get a report with useful information that we will be able to filter and rearrange, so we can look at possible failures or errors in the website. The program is easy to use, we only need to set some basic settings and input site's URL that we want to analyze. Screaming Frog SEO Spider is an ideal tool to analyze and report website's problems.