Batch queries (Preview)

Perform multiple data source queries using a single batch request.

Please note that this functionality is in Preview and might be changed without prior notice. Please let us know of any issues you might encounter to help us improve this solution.

• Overview
• Start batch run
• Get batch information
• Get batch status
• Cancel batch run
• Restrictions
• Object reference
• Errors

Overview

Query batches are intended for situations when the number of queries is very high, and both requesting and polling each query individually become impractical.

Starting a query batch

Send one request containing multiple queries. They are processed in the order they appear in the list.

POST /enterprise/v2/query/batch
Authorization: Bearer <your api key>
Content-Type: application/json

{
    "queries": [
        {
            "ds_id": "AW",
            "ds_accounts": "123456789",
            "start_date": "2022-09-01",
            "end_date": "2022-09-03",
            "fields": "date,impressions,cost"
        },
        {
            "ds_id": "AW",
            "ds_accounts": "234567890",
            "start_date": "2022-09-01",
            "end_date": "2022-09-03",
            "fields": "date,clicks,views"
        }
    ]
}

You can use split_accounts_policy to duplicate the same query for each account ID mentioned in the query, to run them separately from each other.

Polling for status

Using the returned batch_id, you can then poll for the status of all the queries in the batch. They are returned in the same order they were created.

GET /enterprise/v2/query/batch/qrb_3feb5ff5c71e

Retrieving results

As queries finish, you can retrieve query results from the returned result URLs. Please remember to provide authentication credentials when accessing URLs.

"results_info": {
  "total_rows": 47,
  "results_url": "https://.../results?schedule_id=qrb_3feb5ff5c71e_query_1"
}

Handling errors

Any errors that might occur while running the query are reported for each query separately.

"error_info": {
  "status": 500,
  "code": "QUERY_ACCOUNT_UNAVAILABLE",
  "message": "Account is not available",
  "description": "Google Ads account \"123456789\" is no longer available."
}

Start batch run

• queries object[] | string[]
  Required. List of objects containing parameters for a get data request, or a list of query URL strings. Both Supermetrics API version 1 and version 2 URLs are accepted.
• description string
  Optional. Custom description for the batch run.
• split_accounts_policy string
  Optional. Whether each provided query should be split into separate queries based on account IDs. Supported values are "all" to split each account ID into its own query, and "none" to do no splitting at all. Defaults to "none". If queries refer to a team list, splitting uses account IDs inside the list.
• split_dates_policy string
  Optional. Whether each provided query should be split into separate queries based on query date range. Supported values are "day" to split each day into its own query, and "none" to do no splitting at all. Defaults to "none".
• timeout_minutes int
  Optional. Total number of minutes the batch is allowed to run. Minimum value of 5 minutes is enforced. Maximum allowed value is 60 minutes. Defaults to 30 minutes.
• query_concurrency int
  Optional. Number of queries to run concurrently inside the batch. Minimum value of 1 is enforced. Maximum allowed value is 50. Defaults to 5. Please note that actual query processing concurrency still follows your license limits.
• query_output_format string
  Optional. Output format for all query results in the batch. Defaults to json. See output formats for all supported formats.
• add_exclude_account_errors bool
  Optional. Add settings into every query to exclude account access related errors. Defaults to false.
• add_clean_data_types bool
  Optional. Add settings into every query to clean numeric data types and date values. Defaults to false.

Request

POST https://api.supermetrics.com/enterprise/v2/query/batch
Authorization: Bearer <your api key>
Content-Type: application/json

{
    "description": "My AW run at...",
    "queries": [
        {
            "ds_id": "AW",
            "ds_accounts": "123456789",
            "start_date": "2022-09-01",
            "end_date": "2022-09-03",
            "fields": "date,impressions,cost",
            "settings": {
                "no_headers": true
            }
        }
    ]
}

Response

Errors

BATCH_CREATE_FAILED, BATCH_QUERIES_LIMIT_EXCEEDED, BATCH_QUERY_CONCURRENCY_LIMIT_EXCEEDED, BATCH_QUERY_ITEM_SPLIT_FAILED, BATCH_QUERY_ITEM_TYPE_INVALID, BATCH_QUERY_ITEM_VALUE_INVALID, BATCH_QUERY_OUTPUT_FORMAT_INVALID, BATCH_SEARCH_FAILED, BATCH_SPLIT_FAILED, BATCH_TIMEOUT_MINUTES_LIMIT_EXCEEDED, BATCH_UPDATE_FAILED

Get batch information

• batch_id string
  Required. Supermetrics query batch ID.

Response

HTTP 200 OK
{
    "meta": {
        "request_id": "WnjTGeKqOXTnkZQMbZCfVMdmswv2wAzx"
    },
    "data": {
        "@type": "query_batch",
        "batch_id": "qrb_3feb5ff5c71e",
        "status_code": "RUNNING",
        "schedule_time": "2022-09-30T13:25:17+00:00",
        "start_time": "2022-09-30T13:25:20+00:00",
        "end_time": null,
        "cancel_time": null,
        "description": "My AW run at...",
        "split_accounts_policy": "none",
        "split_dates_policy": "none",
        "timeout_minutes": 60,
        "query_amount": 3,
        "query_concurrency": 50,
        "query_output_format": "json",
        "add_exclude_account_errors": false,
        "add_clean_data_types": false,
        "links": {
            "status": {
                "href": "https://.../query/batch/qrb_3feb5ff5c71e/status"
            }
        }
    }
}

Errors

BATCH_NOT_FOUND, BATCH_SEARCH_FAILED

Get batch status

• batch_id string
  Required. Supermetrics query batch ID.
• offset int
  Optional. Pagination offset for the number of queries to return. Defaults to 0.
• limit int
  Optional. Maximum number of queries to return. Defaults to 100, maximum of 1000 is enforced.

Response

HTTP 200 OK
{
    "meta": {
        "request_id": "WnjTGeKqOXTnkZQMbZCfVMdmswv2wAzx",
        "status_code": "RUNNING",
        "progress": {
            "queued": 0,
            "running": 1,
            "success": 1,
            "failure": 1
        },
        "paginate": {
            "offset": 0,
            "limit": 100,
            "total": 3
        }
    },
    "data": [
        {
            "@type": "batch_query",
            "schedule_id": "qrb_3feb5ff5c71e_query_1_acc_1",
            "status_code": "RUNNING",
            "query_start_time": "2022-09-30T13:25:17+00:00",
            "query_end_time": null,
            "split_info": {
                "ds_accounts": [
                    "account1"
                ],
                "start_date": null,
                "end_date": null
            },
            "error_info": null,
            "result_info": null
        },
        {
            "@type": "batch_query",
            "schedule_id": "qrb_3feb5ff5c71e_query_1_acc_2",
            "status_code": "SUCCESS",
            "query_start_time": "2022-09-30T13:25:17+00:00",
            "query_end_time": "2022-09-30T13:26:45+00:00",
            "split_info": {
                "ds_accounts": [
                    "account2"
                ],
                "start_date": null,
                "end_date": null
            },
            "error_info": null,
            "result_info": {
                "total_rows": 47,
                "results_url": "https://..."
            }
        },
        {
            "@type": "batch_query",
            "schedule_id": "qrb_3feb5ff5c71e_query_2_acc_1",
            "status_code": "FAILURE",
            "query_start_time": "2022-09-30T13:26:19+00:00",
            "query_end_time": "2022-09-30T13:27:45+00:00",
            "split_info": {
                "ds_accounts": [
                    "account3"
                ],
                "start_date": null,
                "end_date": null
            },
            "error_info": {
                "status": 500,
                "code": "QUERY_ACCOUNT_UNAVAILABLE",
                "message": "Account is not available",
                "description": "Google Ads account \"account3\" is no longer available."
            },
            "result_info": null
        }
    ]
}

Errors

BATCH_NOT_FOUND, BATCH_SEARCH_FAILED, BATCH_STATUS_NOT_FOUND

Cancel batch run

Cancelling a query batch run will stop it from starting new queries and from waiting responses of currently running queries. Already collected query results will remain available. For more details, see Restrictions.

• batch_id string
  Required. Supermetrics query batch ID.

Response

Errors

BATCH_CANCEL_FAILED, BATCH_NOT_FOUND, BATCH_STATUS_NOT_FOUND, BATCH_SEARCH_FAILED, BATCH_UPDATE_FAILED

Restrictions

Please contact us to raise limits or to address other restrictions.

Usage limits

• Maximum allowed runtime for a batch run is 60 minutes.

• Number of concurrently running batches is limited to two batches at-a-time. Any additional batch runs will be placed in a queue, and processed in the order they were received.

• Number of individual queries in a single batch, after splitting is applied, is limited to maximum of 100 queries.

• Each performed query is still affected by the same license limitations that would be used when running queries individually, such as query concurrency.

Splitting

• Splitting queries by date range does not support date range types. You can however use both fixed and relative start and end dates.

• Splitting queries by accounts does not support all accounts.

Data retention

• Query batch information is available for 90 days from the moment the batch was created.

• Query batch status is available up to one hour after the batch run has stopped.

Object reference

query_batch

batch_query

batch_query_split

batch_query_result

batch_progress

paginate

error

resource_url

Errors