How to Avoid a Disastrous Website Migration: A Comprehensive Guide for Marketing Directors


December 15, 2023
How to Avoid a Disastrous Website Migration

Website migration can be a daunting process for any marketing director. It involves moving your website from one location to another, be it a new domain, a new content management system, or even just a redesign. While site migrations are sometimes necessary to keep up with changing technology and design trends, they can also be fraught with pitfalls that can negatively impact your online presence and customer experience.

Simple mistakes that are often overlooked could lead to a catastrophic site launch. In this blog, we’ll provide you with a comprehensive guide on how to avoid a disastrous website migration and give you the tools that you need to migrate your website with confidence. But first, let’s define the term “site migration”.

What is a Site Migration, Exactly?

The term “site migration” often refers to any change or update that is significant enough to have an impact on your website’s SEO and visibility in the search engines. This can include any of the following:

  • Moving to a new domain
  • Changing the content management system (CMS)
  • Updating the website’s design and structure
  • Merging multiple websites into one
  • Altering the URL structure of the website

Essentially, a site migration involves a substantial change in the way your website is structured, and this can have significant implications for your search engine rankings and user experience.

Is My Site Ready for a Migration?

Before embarking on a website migration, it’s essential to assess whether your site is ready for the change. Here are some factors to consider:

  • Site Age and Size: Older, larger websites tend to have more complex structures. Consider the size and age of your site to gauge the complexity of the migration.
  • Current Performance: Is your website currently performing well in terms of SEO, traffic, and user engagement? All site migrations come with a risk of a decline in search engine visibility. If your site relies on organic traffic for performance, you must weigh the risks of a migration against potential declines in performance.
  • Domain Authority: If your current domain has high domain authority and you’re considering migrating to a new domain, it’s important to understand the impact this might have on your visibility in the search engines.
  • Reason for Migration: Clearly define the objectives and reasons for your migration. Is it for a rebranding, improved user experience, or technical optimization? Understanding your goals is crucial to planning an effective migration.

It’s important to factor in all of these elements to decide whether or not it’s the right time for a site migration. If you’re unsure and need some professional advice, our team is here to help!

Pre-Launch Checklist

You’ve decided to move forward with a site migration, and you’re ready to launch the new website. Before you hit that metaphorical launch button, there are a few important items you need to check off your list to ensure a smooth and successful migration. 

Crawl Your Live Site

The first thing you should do is crawl your old (live) website. Use a tool like Screaming Frog SEO Spider or Google Search Console to generate a list of all the URLs on your current website. The crawl will also reveal any 404 errors or 301 redirects that currently exist on your live site. It’s not a bad idea to clean these up before compiling your full list of URLs: you want to avoid creating too many redirect chains on your new website, which could eat up your crawl budget or overload the server with too many requests.

It’s important to keep in mind that a crawl might not reveal every single page on your website. A site crawler like Screaming Frog will only show pages that are linked internally on your website, which means that if you have any orphan pages that are not linked to internally from any other page, the crawler will not pick those up. These could include important landing pages used for ad campaigns that users cannot normally navigate to from your website’s linking structure.

You can use your CMS or your database to find these orphan pages, or you can find them through your Google Analytics data or Google Search Console. If you do have orphan pages, make sure to include these URLs in your list so that we can map them to your new URLs.

Map Your Old URLs to Your New URLs

The site structure and URLs on your new website have probably changed. Since your old website is already indexed in the search engines, you need to ensure those pages are pointing to the new version of the page (also called a redirect).

Since there are likely multiple external sources linking to your website, you don’t want visitors following those links to be met with a 404 page after the migration. To avoid this, you need to map all of your old URLs to the new URLs on the new website.

This can easily be done in a spreadsheet with a column for the old URL and a column for the new URL. Once you’ve mapped out the URLs, your developer can set up an .htaccess file containing all of the redirects.

Set Up an .htaccess File with Redirect Rules

An .htaccess file is a configuration file used on Apache web servers that lets you configure certain rules and settings at the directory level, before a page is loaded. It’s in this file that we will insert rules to redirect your old URLs to your new URLs. 

Here is the standard 301 redirect structure for an .htaccess file: 

It’s important to note here that the source URL (the URL from the old site) does not contain the top-level domain name, only the “slug”, or everything after the first slash. However, the target URL (the new URL) must contain the full URL, including the protocol (https) and the domain name. 

You can also use regex expressions to bulk redirect certain URLs and reduce load on your server. For example, if you need to bulk redirect an entire subdirectory from your old site to a single page on your new site, you could do the following:  

This would redirect every single child URL under /a-sub-folder/ to the new URL. Here’s a useful resource for common .htaccess redirects that can help you with setting up your .htaccess file.

Note: Redirects using .htaccess files only apply to servers running Apache. For servers running Nginx (pronounced “Engine X”) or other web software like Squarespace, you will need a different approach, since it will not support the .htaccess file. For more information, contact your hosting provider on how you can implement redirects on your server. 

Run a Crawl on the New Site and Check for 404s and 301s

In the first step, we crawled the old website to fetch a list of URLs. Now, it’s time to crawl the new website and double-check that there are no issues, 404 errors, broken images, or any remaining internal 301 redirects. If there are, these should be fixed before launching the site. This ensures a positive user experience, helps with SEO, improves indexing efficiency, and contributes to accurate analytics and reporting.

Create a Custom 404 Page

Creating a custom 404 page is often overlooked by developers but is an important step to ensure you don’t lose any potential site visitors that land on a broken page, or an old URL that was missed in the redirects from the previous steps. 

A good 404 page will let the user know that the page they are looking for is no longer available, and then guide the user towards relevant content like a blog article or a main service page. It can also prompt the user to perform a search on the website for specific content they might be looking for. 

Your server must also be configured to recognize this page as an actual 404 page and return the 404-error code in the browser. This can be achieved by defining your custom 404 page in the .htaccess file, like this:  

Doing so will redirect any traffic to a non-existent page to the custom 404 page defined on this line and will return the “404 error” response code, letting the search engines know that this page should not be indexed. Failing to properly define your 404 page could result in Search Console issues. 

A content management system like WordPress or Drupal will usually handle the 404 page automatically, but if you’re building a custom website with PHP or React, it is imperative that this step does not get missed.

Implement Google Analytics and Google Tag Manager on the New Site

A simple and easy task to forget, launching your new site without your Google Analytics or Google Tag Manager scripts could cause loss of visitor data, especially during those crucial first days after the migration. Don’t forget to include these scripts on your site!

Keep a Backup of Your Old Site

This should go without saying, but please make a copy of your old site’s files and database before launching or migrating. You could also keep a password-protected copy of this site on a staging or development server to be accessed at any time. This will allow you to reference the old website and compare site structure and data, if needed. If your new site is a complete rebuild with brand new content and a reorganized site structure, being able to refer to the old website can be especially helpful since you’ll be able to easily retrieve content, data, or code snippets as needed. 

Keep in mind that if you do choose to keep a copy of your website behind a password-protected page, this will use up additional resources on the server and may incur additional costs. There are also some potential security concerns if your old website contains sensitive information, or old web technology that may become vulnerable. In this case, additional security measures could be put in place to properly secure the website.  

Post-Launch Checklist

Congratulations on launching your new website. Hopefully, by following the steps above, you managed to pull off a relatively smooth website migration. However, the fun isn’t over yet! There are still a few more things to do to ensure a seamless transition.  

Make Sure Redirects Are in Place and Working

Now that your new website is launched, you should test all of your old URLs and make sure they are redirecting. If your old website only had several pages, you could go ahead and test these manually, but if you have hundreds of URLs, it’s best to do this with a tool like Screaming Frog. 

In Screaming Frog, you can run a crawl from a predefined list of URLs. Simply change the mode from “Spider” to “List”, then click the Upload button and select your list of URLs from a file or enter them manually. Using the URL mapping spreadsheet you created before launch, you can grab the list of old URLs and just replace the domain name with your new domain (If your domain name hasn’t changed, you can leave it as is). Then, run this list of URLs through Screaming Frog to crawl them and check whether they are redirecting properly. If all is well, the crawl should return a list of your URLs with the “200” response code (if the URL on your new site hasn’t changed), or “301” (if the URL is redirecting to the new one). If you see any 404 errors, that means some redirects were missed. Find all the 404s, and create new redirect rules in your .htaccess file to redirect these pages to the appropriate page.

Check the robots.txt File

The robots.txt file is a text file that lives at the root directory of your website. Its purpose is to indicate to crawl bots which files and directories they are allowed to crawl.   

When your website is in the development and staging phases, developers will sometimes add a line in the robots.txt file instructing the search engine robots not to crawl the website, to ensure the search engines don’t crawl a website in development. Here’s what it looks like:

Here, “User-agent” is used to define the specific robots we are targeting for the disallow rule below. In this case, the “*” is a wildcard, meaning we are targeting all robots. If you wanted to target only the Google bot, you would enter “User-agent: Googlebot”. The “Disallow” line below it instructs the robots not to crawl anything starting from “/”, which is the root folder of the website.  

When the staging site gets deployed to the live production server ahead of the launch, depending on the development workflow, there is a chance that this robots.txt file will get carried over to the live site with these instructions still in place. That would mean that your new, live site would be blocked from being accessed by all search engine robots. To fix this, open the robots.txt file that sits in the root directory of your website, and make sure to remove the slash in the disallow line, like this:   

This tells the robots that nothing is disallowed, and that they are “allowed” to freely crawl the entire website. You can test the indexability of your website by running it through a crawling tool like Screaming Frog. These tools will usually indicate if a website cannot be crawled due to a disallow rule in the robots.txt file. 

Update Your PPC Campaigns

After you’ve launched your new website, you should update your PPC campaigns so that they point to the new site. Even though we’ve implemented redirects (which will redirect users who click on the ad to the correct page), if your PPC campaigns are pointing to the old site, attribution will be lost in Analytics because of the redirect.

Fix Duplicate Content Issues and Redirect Any Additional Domains

If the “Ensure Your Canonical Links Are Pointing To the New Site” step was done correctly in the pre-launch checklist, there should be no duplicate content issues on your website. If there are any duplicate content issues, it’s likely due to repeating content on separate pages that could be consolidated at a later time. For now, let’s just make sure that there are no canonical issues. The www version of your site (https://www.example.com) should be redirecting to the non-www version of your site (https://example.com), or vice versa. If both these versions are accessible without a redirect, and the canonicals aren’t correctly set, Google will flag your site as having duplicate content, making this step an important one not to miss.

Additionally, if you’ve purchased any other domains that you’d like to point to your new site, you must correctly implement redirects for these domains to point to your main domain. For instance, if your main domain is example.com, but you’ve also purchased example.net and example.org, you should redirect both example.net and example.org to example.com. In some cases, some servers will simply point your add-on domains like example.net and example.org to the website folder, which would make your entire site completely accessible via these add-on domains. The page example.org/about would be visible, instead of redirecting to example.com/about. Google would treat these as 2 separate websites and flag it as duplicate content, penalizing your rankings and hurting your SEO. This is why it’s imperative that you handle your additional domains correctly and implement redirects to your main domain. 

Submit Your New Sitemap

After you’ve checked everything and cleaned up any issues resulting from the site migration, like internal 301 redirects and 404 errors, you should submit the new sitemap to Search Console to signal to Google to crawl your entire website again, so it can index your new links and site structure.

If you’re using WordPress, we highly recommend using the Yoast SEO plugin. Yoast is free and comes with a built-in dynamic XML sitemap, which will generate and update automatically when you make changes to your pages or create new ones. To access your Yoast XML sitemap, go to www.example.com/sitemap_index.xml (replace example.com with your domain name). Yoast breaks up the sitemap into separate individual sitemaps for each post type on your website (pages, posts, etc.). This will differ based on the post types created in your theme. A sitemap will also be created for your post type categories and tags, as well as author pages. To remove these pages from the sitemap, go to Yoast Settings -> Search Appearance. The Content Types, Taxonomies, and Archives tabs will allow you to remove specific types from the sitemap. To do so, set “Show in search results” to off.

Conclusion

A successful website migration requires careful planning and execution. By following this comprehensive checklist, you can significantly reduce the risk of a disastrous migration and ensure that your new site maintains or improves its SEO rankings, user experience, and overall performance. Each migration scenario is different and may require a unique approach, but this list is a great starting point and will give you the confidence you need to ensure a smooth and successful migration. Remember, you can never be too careful to check, double-check, and triple-check everything before moving forward. A well-executed migration can lead to a stronger online presence and better results for your business. 

If you need any assistance with your site migration, our team is here to help! 

Recent Posts