Running a website is hard. In addition to creating compelling and useful content that people want to read, you also need to be aware of dead links, misspellings, and grammar. But luckily, there are tools available to help make this easier.
SiteInspector is a free and open source application that you can use to scrape your website and check for issues that are turning off readers and causing you to drop in the search rankings. Here’s how to use it to automatically scan your site for errors.
Why use SiteInspector to scan your website?
It’s easy and cheap to build your own website, and you can run a website from a Raspberry Pi or an old laptop. But it can be much more difficult to keep up with your website maintenance.
Visitors come to your site to read what you’ve written, buy your product, or view your images. Unless you’re throwing a stream of consciousness onto the page in the style of Irvine Walsh or James Joyce, you want your writing to be clean and consistent, with standardized spelling.
Similarly, your grammar should be correct and there should be no broken internal or outbound links. If your website doesn’t live up to this, visitors won’t be as confident in your attention to detail and may leave.
Worse, you’ll take an SEO hit, which means your site will appear further down the page in search results and you may not have any visitors.
Going through your site page by page and manually checking links, spelling and grammar is time consuming and tedious. SiteInspector takes the hard work out of catching errors and presents you with a condensed list of possible improvements.
How to install SiteInspector
SiteInspector will run on Linux, macOS, or on Windows via Windows Subsystem for Linux (WSL).
The application is packaged as a Docker image and you will need to have Docker and Docker Compose installed. If you don’t already have them, read our guide on how to install Docker and Docker Compose.
The easiest way to install SiteInspector is to use the following command:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/siteinspector/siteinspector/master/scripts/docker-install.sh)"
This command will run a script which, in turn, will get a Docker Compose file and then issue the appropriate command to open Docker Compose. Docker Compose will set up containers and make the application accessible on port 808 of your local machine.
After running SiteInspector for the first time, you can start it by running:
docker-compose up -d
How to use SiteInspector to check your website for errors
Once SiteInspector is up and running, open a web browser and in the address bar, type: homeserver: 808.
You will need to create an administrator account with an email address and password. Don’t worry, these are for local authentication only and won’t be sent to developers unless you decide to subscribe to newsletters after hitting the Come on! button.
Then click add website, then enter the URL of the website you want to scan. don’t click Deliver still. Instead, press Set up to expand a menu where you can adjust the scan options.
Exclude Path allows you to specify paths that should not be scanned.
If you’re using a structure that separates content into years, and you already know that pre-2022 content is perfect and error-free, you’ll want to exclude paths that include “/2021”, “/2020”, etc.
Alternatively, you can set a value in start path. This tells SiteInspector to start its scan using a particular path as its root. You can also set up custom tests; you might want to check if you have left any “Lorem Ipsum” placeholder text, for example.
You can then toggle the switches to see if you want SiteInspector to check for spelling and grammar, broken links, images, or scripts.
To put SiteInspector to the test, we benchmarked it against MakeUseOf, checking for spelling and grammatical errors, along with broken links.
Starting our test around 6am EST, we found that SiteInspector traversed MUO at a rate of around 25,000 pages per hour for the first hour, before slowing down as the continental US woke up and went online.
The next 80,000+ pages took around 10 hours. For reference, MUO has over 85,000 articles and over 100,000 individual pages.
While the above stats don’t look great, MakeUseOf is a very technical site and includes a lot of jargon, code snippets, and other factors that can contribute to the perceived error rate. That being said, SiteInspector allowed us to catch multiple spelling issues that went through the editorial network.
Each page containing an issue has its own report section, with broken links displayed at the bottom. When you have fixed the problems on each page, you can click Resolved. The page report will collapse and you can move on to the next one.
If you need help correcting the errors, click the share report button on the report tab. This will generate a link that you can send to your collaborators, assigning them an editor or viewer role.
Note that if you are sharing with people on your local network, you will need to replace localserver with your local IP address. If your coworkers aren’t on the same network as you, consider hosting on a VPS.
SiteInspector makes it easy to fix errors on your website
With SiteInspector, you can quickly identify and fix errors on your website. Of course, it’s better if you don’t have any errors to begin with. While you can’t fix these issues through SiteInspector, you can take steps to improve your own spelling and grammar.
