🪄Auto-Indexing Tool

With this Auto-indexing tool, you can connect your personal Google Service Account to automatically submit many URLs for indexing.

This tool has been deprecated. Please use the new version integrated into the Indexing Dashboard (Google Search Console integration).

Set up the Web Search Indexing API and Google Service Accounts.

Step 1: Enable the Web Search Indexing API

Visit the Google Cloud Console at https://console.cloud.google.com/ and create a Project.

Create a project on Google Cloud Console

In the top search bar, type "API" and select "Enabled APIs & services".

Select "Enabled APIs & services"

Click "Library" on the left sidebar.

Select "Library"

Search for "Web Search Indexing API" in the top search bar and enable it, so you can access it with your Google Service Account

Enable the Web Search Indexing API

Step 2: Create a Google Service Account

In your Google Cloud Console dashboard, search for "Service Accounts" and click on it.

Click on "Service Accounts"

Hit the "Create Service Account" button, and enter a service account name. Then, hit the "Done" button.

Enter service account name

Copy the email of the service account.

Copy the email

Click on the Service Account's email, select the "KEYS" tab, then click the "Create new key" button.

Create a key for your service account

A model will pop up. Select "JSON" and click the "Create" button.

Select JSON in Key type field.

After creating the key, Google will download it to your machine. Keep that file. We will need it in Step 3.

Step 3: Add the Google Search Console to SEO Utils

Access the Google Service Accounts menu from the App dropdown at the top-right corner.

Access the Google Service Accounts menu

Then, click the Add Account button, and select the JSON file that you downloaded in Step 2 to add a new account.

Google Indexing API Rate Limit

Google only allows you to submit 200 URLs per day via the Indexing API. There are 2 ways to bypass this.

1. Request more quota.

2. Create multiple projects and Google Service Accounts by repeating Step 1 to Step 3. Please note that you have to create a new project and then create a new Google Service Account in that new project because Google limits the API usage per project, not per Google Service Account.

After creating an account, please copy the Google Service Account email address.

Copy the Google Service Account email address

Visit your Google Search Console, select the site that you want to submit for indexing. Click on Settings > Users and Permissions.

Click the "Add User" button, then paste the Google Service Account that you copied previously. Select "Owner" permission for the new user.

Add a new user

If you have multiple Google Service Accounts, you will need to add multiple users.

Connect your Google Search Console Sites & Sitemaps

If you haven't connected a site on your Google Search Console, please follow this guide to do that.

After connecting a site, SEO Utils will pull all the sitemaps that you entered on your Google Search Console.

If you haven't added a sitemap to your Google Search Console account, you can add sitemaps manually on SEO Utils

Add a sitemap manually.

You can add as many sitemaps as you want.

After adding sitemaps, you can fetch all URLs from them by clicking the "Fetch URLs From All Sitemaps" action. SEO Utils will pull all URLs from all sitemaps and check the index of every single URL.

Fetch URLs From All Sitemaps

Checking Index

You can trigger a process to bulk-check the index for all URLs on your site by clicking the Check Index action.

Bulk-check index

You can also check index for a single URL by clicking the vertical dots icon.

Check index for a URL

Two Checking Index Methods

#1. Using Google Search Console API (default)

Since March 4, 2024, SEO Utils has supported checking index by using Search Console URL Inspection API. This method has some pros when compared to method #2 which uses the "site:url" operator.

• Do not require proxies, so no extra cost.

• Faster, can check 10 URLs per second.

• Easy to set up, just need to enable the Google Search Console API.

However, there are some quota limits. The quota is enforced per Search Console website property (calls querying the same site):

• 2,000 queries per day

• 600 queries per minute

So if you reach the limit, you might want to switch to the #2 method.

Moreover, the Google Search Console API method also gives you some extra information that the second method doesn't provide like

• Indexing State

• Coverage State

• Crawled Date

You can hover on the blue "info" icon to see that information.

Extra information that the first method gives you.

There is a filter for the Indexing State, so you can tell why the URL is not allowed to be indexed.

#2. Using the "site:url" Operator

When using this method, SEO Utils will search Google using the "site:url" operator to see if the URL is on the SERP.

This method doesn't have a quota limit but you want to consider using Proxies to prevent Google from blocking your IP when checking thousands of URLs.

Switching Method

You can click on the "cog icon" button to switch the method. The method is saved on the site scope, which means you can use method #1 for site A and method #2 for site B.

Switching checking index method.

Click the "Update" button to save the settings.

Submitting Index

Just like the "Check Index" action, you can submit either multiple URLs or just one URL for indexing.

When you bulk-submit URLs, SEO Utils will submit all URLs regardless of their indexed or submitted status.

Auto Index Feature

SEO Utils provides an auto-pilot mode so you don't need to pull URLs from sitemaps, check & submit index manually.

To enable the Auto Index feature of a site, please ensure that you toggle the Auto Index field on.

Enable Auto Index for sites

You can also enable the Auto Index feature for specific sitemaps by visiting the Sitemaps list page.

Enable Auto Index for all URLs of a specific sitemap

You can tell if a URL is enabled for auto-indexing if the URL is yellow like this.

URLs are disabled for auto-indexing

When URLs are disabled for auto-indexing, SEO Utils won't automatically check and submit them for indexing. However, you still can manually do that by using "Check Index" & "Submit Index" actions.

To use the Auto Index feature, you need to keep SEO Utils running on your machine or use a VPS, so you don't need to keep your computer on constantly.

Auto Pull URLs From Sitemaps

SEO Utils will pull URLs from all sitemaps daily at 5:00 AM (your local time) to get new URLs and check if the existing URLs have content updated by using the Last Mod field.

Using Last Mod field to detect if the content of a URL is updated.

If a URL has updated content, SEO Utils will resubmit it for indexing, regardless of the submission status.

Auto Check Index

SEO Utils will check index of all sites that enable the Auto Index feature at 6:00 AM (your local time).

Note: It will skip the URLs that belong to a sitemap with the Auto-Index setting turned off.

Please consider using Proxies if you have many URLs to check index.

Auto Submit Index

SEO Utils will submit for indexing all sites that enable the Auto Index feature every 10 minutes.

Note:

• It will skip the URLs that belong to a sitemap with the Auto-Index setting turned off.

• The auto-pilot mode only submits URLs that have already been checked for indexing. You don't have a setting to ignore this behavior like when running the "Submit Index" action manually.

Override the Auto-Mode Behavior

This is the default behavior of the Auto-mode

• Checking index: If a URL is already indexed, it won't re-check index for that URL in the subsequent runs.

• Submitting index: If a URL is already submitted or indexed, it won't re-submit that URL in the subsequent runs.

If you would like to override those behaviors, please open the Site setting modal.

Open the setting modal.

You will see 2 fields:

• Auto Recheck Index After field: You can specify the number of days to recheck the index even if the URL is already indexed.

• Auto Resubmit Index After field: You can specify the number of days to resubmit URLs even if URLs are already submitted but not indexed.

Override the Auto-mode behavior

Those two fields will only be shown if you turn on the Auto mode for the site.

Quick Tips

You can view all the logs of check and submit for indexing of each URL by clicking the log bar at the bottom.

View logs

You can quickly switch between pages on a site by using the dropdown menu in the header section.

Quickly move to a page

Updates on March 22, 2024

You can now set which Google service accounts you want to use on specific sites.

For example, if you set up a Google Search Account can be used on site A. SEO Utils will only use that account to check & submit index for that site only.

Set site scopes for each Google Service Account

Select a site where you want to use the Google Service Account.