🪄Auto-Indexing Tool
With this Auto-indexing tool, you can connect your personal Google Service Account to automatically submit many URLs for indexing.
Last updated
With this Auto-indexing tool, you can connect your personal Google Service Account to automatically submit many URLs for indexing.
Last updated
Visit the Google Cloud Console at https://console.cloud.google.com/ and create a Project.
In the top search bar, type "API" and select "Enabled APIs & services".
Click "Library" on the left sidebar.
Search for "Web Search Indexing API" in the top search bar and enable it, so you can access it with your Google Service Account
In your Google Cloud Console dashboard, search for "Service Accounts" and click on it.
Hit the "Create Service Account" button, and enter a service account name. Then, hit the "Done" button.
Copy the email of the service account.
Click on the Service Account's email, select the "KEYS" tab, then click the "Create new key" button.
A model will pop up. Select "JSON" and click the "Create" button.
After creating the key, Google will download it to your machine. Keep that file. We will need it in Step 3.
Access the Google Service Accounts menu from the App dropdown at the top-right corner.
Then, click the Add Account button, and select the JSON file that you downloaded in Step 2 to add a new account.
Google Indexing API Rate Limit
Google only allows you to submit 200 URLs per day via the Indexing API. There are 2 ways to bypass this.
Create multiple projects and Google Service Accounts by repeating Step 1 to Step 3. Please note that you have to create a new project and then create a new Google Service Account in that new project because Google limits the API usage per project, not per Google Service Account.
After creating an account, please copy the Google Service Account email address.
Visit your Google Search Console, select the site that you want to submit for indexing. Click on Settings > Users and Permissions.
Click the "Add User" button, then paste the Google Service Account that you copied previously. Select "Owner" permission for the new user.
If you have multiple Google Service Accounts, you will need to add multiple users.
If you haven't connected a site on your Google Search Console, please follow this guide to do that.
After connecting a site, SEO Utils will pull all the sitemaps that you entered on your Google Search Console.
If you haven't added a sitemap to your Google Search Console account, you can add sitemaps manually on SEO Utils
You can add as many sitemaps as you want.
After adding sitemaps, you can fetch all URLs from them by clicking the "Fetch URLs From All Sitemaps" action. SEO Utils will pull all URLs from all sitemaps and check the index of every single URL.
You can trigger a process to bulk-check the index for all URLs on your site by clicking the Check Index action.
You can also check index for a single URL by clicking the vertical dots icon.
#1. Using Google Search Console API (default)
Since March 4, 2024, SEO Utils has supported checking index by using Search Console URL Inspection API. This method has some pros when compared to method #2 which uses the "site:url" operator.
Do not require proxies, so no extra cost.
Faster, can check 10 URLs per second.
Easy to set up, just need to enable the Google Search Console API.
However, there are some quota limits. The quota is enforced per Search Console website property (calls querying the same site):
2,000 queries per day
600 queries per minute
So if you reach the limit, you might want to switch to the #2 method.
Moreover, the Google Search Console API method also gives you some extra information that the second method doesn't provide like
Indexing State
Coverage State
Crawled Date
You can hover on the blue "info" icon to see that information.
There is a filter for the Indexing State, so you can tell why the URL is not allowed to be indexed.
#2. Using the "site:url" Operator
When using this method, SEO Utils will search Google using the "site:url" operator to see if the URL is on the SERP.
This method doesn't have a quota limit but you want to consider using Proxies to prevent Google from blocking your IP when checking thousands of URLs.
You can click on the "cog icon" button to switch the method. The method is saved on the site scope, which means you can use method #1 for site A and method #2 for site B.
Just like the "Check Index" action, you can submit either multiple URLs or just one URL for indexing.
When you bulk-submit URLs, SEO Utils will submit all URLs regardless of their indexed or submitted status.
SEO Utils provides an auto-pilot mode so you don't need to pull URLs from sitemaps, check & submit index manually.
To enable the Auto Index feature of a site, please ensure that you toggle the Auto Index field on.
You can also enable the Auto Index feature for specific sitemaps by visiting the Sitemaps list page.
You can tell if a URL is enabled for auto-indexing if the URL is yellow like this.
When URLs are disabled for auto-indexing, SEO Utils won't automatically check and submit them for indexing. However, you still can manually do that by using "Check Index" & "Submit Index" actions.
To use the Auto Index feature, you need to keep SEO Utils running on your machine or use a VPS, so you don't need to keep your computer on constantly.
SEO Utils will pull URLs from all sitemaps daily at 5:00 AM (your local time) to get new URLs and check if the existing URLs have content updated by using the Last Mod field.
If a URL has updated content, SEO Utils will resubmit it for indexing, regardless of the submission status.
SEO Utils will check index of all sites that enable the Auto Index feature at 6:00 AM (your local time).
Note: It will skip the URLs that belong to a sitemap with the Auto-Index setting turned off.
Please consider using Proxies if you have many URLs to check index.
SEO Utils will submit for indexing all sites that enable the Auto Index feature every 10 minutes.
Note:
It will skip the URLs that belong to a sitemap with the Auto-Index setting turned off.
The auto-pilot mode only submits URLs that have already been checked for indexing. You don't have a setting to ignore this behavior like when running the "Submit Index" action manually.
This is the default behavior of the Auto-mode
Checking index: If a URL is already indexed, it won't re-check index for that URL in the subsequent runs.
Submitting index: If a URL is already submitted or indexed, it won't re-submit that URL in the subsequent runs.
If you would like to override those behaviors, please open the Site setting modal.
You will see 2 fields:
Auto Recheck Index After field: You can specify the number of days to recheck the index even if the URL is already indexed.
Auto Resubmit Index After field: You can specify the number of days to resubmit URLs even if URLs are already submitted but not indexed.
Those two fields will only be shown if you turn on the Auto mode for the site.
You can view all the logs of check and submit for indexing of each URL by clicking the log bar at the bottom.
You can quickly switch between pages on a site by using the dropdown menu in the header section.
You can now set which Google service accounts you want to use on specific sites.
For example, if you set up a Google Search Account can be used on site A. SEO Utils will only use that account to check & submit index for that site only.
