Semalt Recommendations On How To Deal With 404 Errors And Deleted Products
A 404 error is basically a URL that has no content. Technically, this is an HTTP response error code indicating that the client (web browser, search engine robot) can connect to the server and then, the server is unable to find the requested file.
404 error is also named as 404 error page, 404 error, 404 not found, 404 error site, 404 page, or page not found.
Error 404 - when can it occur?
Probably each of us came to a page with information about a 404 error.
There are many possibilities through which a 404 error can occur, but in this article, we will focus on those which, in our experience, are responsible for most of the cases in which information about a page that has not been found appears on the page.
Removal of a product, article or any subpage
We can most often come across 404 errors in online stores, which are characterized by a large number of subpages (e.g. categories and products). Due to the specificity of e-commerce, we often deal with products that "disappear" from the store's offer.
There can be many reasons for this, but the most common are:
- the product has been sold out;
- the product has been withdrawn from the store/manufacturer's offer.
In such cases, the store administrator often turns off the display of the product in the store or removes the product, so that the product ceases to exist at the current URL for users and search engines - this way a page with error 404 is created.
A similar situation may apply to offer pages or categories in online stores, which are also disabled/removed due to the recall of a given product group.
Providing an incorrect URL
As we mentioned earlier, also the so-called typos in URLs can cause 404 errors on our site. Imagine a situation in which users of a certain online forum share their opinions about a given online store - it is enough for one of the users to accidentally make a typo in the URL that they want to put in their comment. In this way, subsequent forum users and search engine robots (e.g. Google Bot) "clicking" on this link with a typo go to the store's website with a 404 error.
Also, we, as website owners, are often able to accidentally provide an incorrect internal link in our category descriptions/blog articles to direct the user to another subpage of our website - so the effect is the same as the above-mentioned, i.e. a page with error 404 will be displayed.
Change the structure of URLs
An equally common reason for large-scale 404 errors is a change in the URL structure that occurs most often when migrating a website from one CMS system to another store management system.
So, if during the migration of the website/change of the URL structure, we do not perform the appropriate 301 redirects (permanent redirection of the old URL to the new one), then users and search engine robots may still want to use the old URLs, which in such a situation will display the 404 error although the product exists, but at a "slightly different" URL.
How does the 404 error affect SEO and the user?
Many website owners do not realize how many of their 404 pages are displayed to users and search engine robots every day. We will write about how to check which and how many of our subpages are displaying a 404 error later in the article. For now, let's focus on the impact of pages with the 404 code on SEO and users and how to deal with these pages.
Customer impact and bounce rate
Often, users who come to our website with the 404 code come from the Google search engine or from clicks on a wrongly provided link (e.g. due to a typo). In such cases, we can assume that the user is already aware of what he/she expects - coming e.g. from a Google search engine, he/she had to enter a query beforehand, after which our website was displayed to him/her. In such a situation, when our potential customer sees a page with a 404 error instead of the expected content, he/she will probably leave the page within a few seconds and use another one on the list of Google search results.
This may result in a high bounce rate of our website and a loss of trust in our website.
Effect of 404 errors on SEO
In the case of individual 404 errors, our entire website should not significantly lose its visibility in the Google search engine - only the loss of organic traffic/position in such cases will apply to those subpages that ceased to exist on our website overnight. This is because the Google search engine is primarily concerned with the fact that the pages with high positions meet the intentions of Internet users. If our website has so far fully met the intentions of searches (e.g. the user was looking for a specific product model and we had this product), then we had high positions in Google. However, if our product ceased to exist overnight, our website no longer meets these intentions and therefore sooner or later it will lose its high position for a given keyword. The consequence will be a decrease in organic traffic on our website.
However, if the subpages with a 404 error constitute a significant% of all our subpages - they may significantly lose their visibility in the Google search engine. This is for many reasons:
- The result of a large number of 404 errors can be a high bounce rate - this way Google can conclude that your website does not meet the intentions of users, which may result in lower positions in Google's organic search results.
- Google, visiting and indexing a large number of your pages with a 404 error every day, may have a problem with indexing the remaining full-fledged subpages of your website due to the limitation of the crawl budget (resources that the search engine robot can spend on "scanning" your site at the moment). As a result, a large number of your subpages may not be indexed in the Google search engine.
How to stop users despite 404 errors?
Pages with 404 code are and will continue to be an issue that will require periodic analysis and optimization (at least once every few weeks, depending on the size of the website). However, it is also worth thinking about solutions that will to some extent help solve the problem of the growing number of pages with the 404 message, and at the same time have a positive impact on the experience of users and search engines.
How do I fix the 404 errors?
Since the most common 404 pages are product pages, we will focus in this article on how to deal with them. The methods presented below will also be used for other subpages with the 404 code, e.g. blog articles, offer pages, etc.
The product is temporarily unavailable
If your product is only temporarily unavailable - do not remove it or hide it from the user on the page! If you know that a given product will return to the store's offer and it is only a matter of time - inform the user about it.
You can inform the user about the unavailability of a product, e.g. information about the possibility of contacting the store to verify whether a given product can be imported specifically for him/her or support in finding a similar product.
This solution encourages the user to stay on our website and continue to contact the store staff, thanks to which there is a chance to maintain a "bond" with a potential customer. If you see a regular page with a 404 error, the user will likely leave your site in a few seconds.
Similar products
If your product will not return to the store's offer, but you can offer similar products that perform the same functions - do not delete the unavailable product page, and use it to inform about the unavailability of the product and offer similar products. In this way, there is a chance that you will be able to use the traffic on the website with an unavailable product to advertise another similar product that a potential customer would not have reached in your store otherwise.
Redirect 301
If your product has been permanently withdrawn from the store's offer and you do not have products similar to it, and you do not want to mislead users that a given product is temporarily unavailable - you will probably disable the ability to display this product in your store or remove it (while creating a page with error 404). In this case, it is worth analyzing with the Dedicated SEO Dashboard whether a given product (or more specifically - its URL address) still generates traffic and whether it has links from external domains (e.g. you once bought a sponsored article with a link to the product or users recommended your product on various websites).
If a given product still generates traffic on your website and has strong links from external domains - redirect it using a 301 redirect to its parent category. This way, you will convey to some extent the "SEO power" of your product for the category it has been in so far.
It is a common procedure, that in such cases, is applied by SEO specialists to use the potential of non-existent products to strengthen a given category in terms of its positioning.
Optimize the 404 error page
When none of the above solutions can be implemented on your website - and the removed products will display the page with a 404 error, don't forget to optimize it for SEO and UX.
The mere information that a given subpage has not been found makes it practically certain that the user will leave the website and will not return to it for a given moment. Take advantage of this moment of attention from a potential customer and encourage them to continue browsing your website, and you can do it in at least several ways:
- take care of an interesting and original way of presenting a page with a 404 error in the form of a dedicated graphic design;
- suggest going to the most popular products/categories in your store;
- suggest using the store's internal search engine to find an interesting product;
- propose a special discount code for purchases due to problems encountered by the customer (i.e. a page with a 404 error).
How to detect 404 errors?
There are many more and less effective ways to find 404 errors on our site. We will focus on proven and relatively easy methods for novice users that will provide us with the necessary information about our subpages generating a 404 error.
Google Search Console
If our website is connected to the free Google Search Console tool, it is enough to go to the "Status -> Excluded" tab:
Here we will find information about pages that have recently been visited by Google and classified as pages with a 404 error.
Google Analytics
If all pages with a 404 error on your website have a text fragment with a value such as "Error 404" automatically added in the title (meta title), then just log in to your Google Analytics account, and then go to the "Behaviours Site Content" section, and then The title of the page and enter "404" in the search engine and add "Entry page" as an additional dimension.
Page crawl
An effective way to analyze our website for broken links (404) is to crawl the page using SEO tools such as Semalt's Dedicated SEO Dashboard. These programs allow you to scan all URLs of our website and create, among others, a list of subpages with a 404 error.
Server log analysis
The analysis of the server logs allows, among others learn about the behaviour of Google robots on our website. By analyzing the server logs, we can check when and for what the search engine robot visited our website, what subpage was visited and what was the status of this subpage (e.g. error 404). For such analysis, we will need server logs (they are usually located on the server in the/log/folder) and tools for their analysis.
Error 404 - Summary
As you can see, detecting pages with 404 errors on the website is not a major problem if we have specialized SEO tools for this like the DSD. We recommend that you analyze 404 errors on your website at least once every few weeks to ensure a positive experience for both users and search engine robots.