HTTP Status Codes, 2xx, 3xx, 4xx, 5xx

A HTTP status code is returned by your server in response when a request is made to your server for a web page on your website. Examples are:

  • When a user accesses the web page in a browser.
  • When a search engine spider (e.g. Googlebot) crawls the web page.

HTTP status codes are of 5 different categories:

  • 1xx Provisional/Informational Response
  • 2xx Successful
  • 3xx Redirected
  • 4xx Client/Request Error
  • 5xx Server Error

The first digit of the status code represents one of five classes of response.

HTTP Status Codes

Google provides a list of HTTP status codes to webmasters, in which the most common and imporant status codes are below:

200 (Successful) – The server successfully processed the request. Generally, this means that the server provided the requested page. If you see this status for your robots.txt file, it means that Googlebot retrieved it successfully.

301 (Moved permanently) – The requested page has been permanently moved to a new location. When the server returns this response (as a response to a GET or HEAD request), it automatically forwards the requestor to the new location. You should use this code to let Googlebot know that a page or site has permanently moved to a new location.

302 (Moved temporarily) – The server is currently responding to the request with a page from a different location, but the requestor should continue to use the original location for future requests. This code is similar to a 301 in that for a GET or HEAD request, it automatically forwards the requestor to a different location, but you shouldn’t use it to tell the Googlebot that a page or site has moved because Googlebot will continue to crawl and index the original location.

400 (Bad request) – The server didn’t understand the syntax of the request.

403 (Forbidden) – The server is refusing the request. If you see that Googlebot received this status code when trying to crawl valid pages of your site (you can see this on the Web crawl page under Diagnostics in Google Webmaster Tools), it’s possible that your server or host is blocking Googlebot’s access.

404 (Not found) – The server can’t find the requested page. For instance, the server often returns this code if the request is for a page that doesn’t exist on the server. If you don’t have a robots.txt file on your site and see this status on the robots.txt page of the Diagnostic tab in Google Webmaster Tools, this is the correct status. However, if you do have a robots.txt file and you see this status, then your robots.txt file may be named incorrectly or in the wrong location. It should be at the top-level of the domain and named robots.txt. If you see this status for URLs that Googlebot tried to crawl (on the HTTP errors page of the Diagnostic tab), then Googlebot likely followed an invalid link from another page (either an old link or a mistyped one).

500 (Internal server error) – The server encountered an error and can’t fulfill the request.

503 (Service unavailable) – The server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state.

HTTP Status Codes for SEO

Webmasters can regularly review the HTTP status codes and search engine spiders in IIS log files (if sites are running on Internet Information Services). Other sources on HTTP status codes:

  • SEOmoz.org – Regarding Google SEO, webmasters’ main objective is to increase Google PageRank (link juice) or to maintain Google PageRank values of their web pages. Only web pages returning either HTTP status codes 200 and 301 are Google SEO friendly and are able to pass link juice.
  • SEO glossary includes the definitions of 301, 302, and 404 HTTP status codes as well as many other definitions related to search engine optimization.
  • W3.org has a complete list of HTTP status codes.

HTTP Status Code 301

This test shows that Google recognizes 301 redirects and passes link juice from the redirecting web page to the redirected web page, as Google SEO traffic is not lost.

To set up 301 redirects on Microsoft’s Internet Information Server ( MS IIS ):

  • You can redirect either a domain or an individual web page.
  • You will need one source web page or site, and one destination web page or site.
  • You can use the IIS control panel to create the redirects.

To set up 301 redirects on Apache:

  • You will need one source web page or site, and one destination web page or site.
  • When you have the mod_rewrite extension installed by default and need to redirect .htm files from an old server to the equivalent .php files on a new server using a 301 redirect, use a combination of mod_rewrite and the redirect directive to do the URL change and redirection.
  • Use the redirect directive syntax in the .htaccess file: Redirect permanent /yourdirectory http://www.newdomain.com/newdirectory or Redirect 301 /yourdirectory http://www.newdomain.com/newdirectory

A example from SEOmoz shows it previously did 301 redirections involving redirecting canonical hostnames and redirecting specific files and folders from one domain to another.

HTTP Status Code 404

How Google prefers webmasters to handle 404 errors:

  • Google defines the 404 response code should be returned for a file “Not Found” request.
  • Google explains “hard 404s” and “soft 404s” and suggests avoid returning “soft 404s”. A soft 404 occurs when a user requests a non-existent URL on your site, but the server returns a web page with an error message and a 200 HTTP status code.
  • “Soft 404s” may be confusing to both users and search engine spiders, and cause the Googlebot to spend unnecessary time crawling and indexing non-existent and duplicate URLs on your website.
  • Google offers a soft 404s reporting feature under the Crawl errors section in Google Webmaster Tools.

To improve user-friendliness, webmasters can set up the servers to return a beautifully/friendly designed 404 error pages with messages.

Comments

Leave your comments

  • Your first comment will be reviewed before getting posted.
  • Your subsequent comments will be posted without review.
  • All spammy comments will be deleted.