How to Use Screaming Frog

Getting started with Screaming Frog

After reading this guide, you'll be able to crawl your website using Screaming Frog SEO Spider, identify technical SEO issues, and leverage its AI integrations for enhanced analysis.

Step 1: Sign up and set up

1. Go to the Screaming Frog website.
2. Click on the "Download" button on the homepage.
3. Choose the version for your operating system (Windows, macOS, or Ubuntu).
4. Install the application by following the prompts.
5. Once installed, open Screaming Frog. You can use the free version, which allows you to crawl up to 500 URLs.

Step 2: Your first crawl

1. In the Screaming Frog interface, locate the URL input bar at the top.
2. Enter the URL of the website you want to crawl (e.g., https://www.example.com).
3. Click the "Start" button (green play icon) next to the URL bar.
4. Wait for the crawl to complete. You’ll see progress in the status bar at the bottom.
5. Once finished, navigate through the tabs (like "Internal", "External", "Images", etc.) to view the results of your crawl.

Step 3: Get better results

1. Use the "Filters" dropdown to focus on specific issues, such as "Client Error (4xx)" or "Redirects".
2. Explore the "Audit" tab to analyse metadata and identify missing title tags or meta descriptions.
3. Access the "Integrations" menu to connect with AI tools like OpenAI for deeper insights. Set up your API key under "Configuration" > "API Access".

Pro tip

Use the "Custom Extraction" feature under "Configuration" > "Custom" to pull specific data from your pages. This can save time if you need to extract specific attributes, such as schema markup or other custom elements.

Common mistake to avoid

A common mistake is not checking the crawl settings before starting. Ensure that the settings under "Configuration" > "Crawl" are set to include all desired page types (like JavaScript or CSS) to avoid missing crucial data during your audit.