Terminal 101: Mirror Websites for Offline Viewing
Posted 03/26/2012 at 12:54pm
| by Cory Bohon
Every Monday, we'll show you how to do something new and simple with Apple's built-in command line application. You don't need any fancy software, or a knowledge of coding to do any of these. All you need is a keyboard to type 'em out!
There are a variety of rather costly tools that can mirror websites locally for you. However, there's also a command line tool called wget that does the exact same thing, and it's free! While wget isn't installed by default in Mac OS X, this free tool can easily be downloaded and installed to your Mac using MacPorts.
This week, we'll show you how to use Terminal to download and install wget, and create a complete website mirror for backup or offline viewing purposes.
What You Need:
>> MacPorts (free)
>> wget (free, install guide below)
Installing MacPorts
MacPorts is a free Mac OS X installer download from MacPorts.org. After downloading the correct version of MacPorts for your version of Mac OS X, you can install the software easily by double-clicking on the installer file.

Follow the prompts to install MacPorts on your system. MacPorts will not appear in your Applications folder because it's a command line utility that is accessed through the Terminal.
Installing wget

After installing MacPorts, you can install the wget command line application by opening up Terminal and typing in the following command:
sudo port install wget
This command tells the MacPort package manger to go out to the Internet and download the latest version of wget, and then install it on your system.

If you have not used MacPorts before, you may need to update its database by issuing the following command before attempting to install wget on your Mac:
sudo port selfupdate
Mirroring a website with wget

After you have installed wget, we can now use the command line utility to mirror websites. To mirror a website, type the following command into the Terminal:
wget -mk domain.com
Replace “domain.com” in the above command with the URL of the website that you wish to mirror. The “-m” puts wget into mirror mode, which follows links on webpages and downloads subsequent pages on the site. The “-k” ensures that wget re-writes links so that they link to your local copy of the website instead of the original destination (the remote web server).

All of the wget-downloaded files can be found in the directory in which you originated the command (usually your User directory).
Cory Bohon is a freelance technology writer, indie Mac and iOS developer, and amateur photographer. Follow this article's author, Cory Bohon on Twitter.