Intelligence API Features

Rate limiting

The Intelligence API was not built to export RAW data on a large scale but to make the calculated results and insights from process mining available to 3rd party platforms and applications. That’s why the Celonis API enforces rate limiting. This means that only a certain number of requests are allowed per day and a certain number of records can be retrieved in each call. Celonis reserves the right to adjust the rate limits at any time to guarantee high-quality service for all clients.

If a client repeatedly exceeds the rate limits or engages in behavior that is deemed to be suspicious, Celonis reserves the right to temporarily or permanently limit or suspend access to the API for that client.

When a client exceeds the number of requests per day, the Celonis API will return a 429 response (too many requests) including an HTTP header (x-ratelimit-reset) which indicates the time (in seconds) that the client needs to wait before a new request can be processed.

The following HTTP headers are also returned as part of each call:

• x-ratelimit-limit : Represents the max number of requests that the client can perform in the current time window.
• x-ratelimit-remaining : Number of remaining requests in the current time window.

Currently, the API has the following default limits:

Table 1. Default rate limits

Pagination

All endpoints that return results in the form of lists provide the ability to paginate their results by leveraging page-based pagination. This can be configured via two parameters:

• page : The page number to be retrieved starting from 0 . As the maximum number of records that can be retrieved through the /data endpoint is 5000 records , the maximum page number is the one that contains the 5000th record based on the pageSize .
• pageSize : The number of items per page to be returned. The default value is 50 meaning that the maximum number of items that can be retrieved in each call is 50 records.

Sorting

Data Endpoint

The /data endpoint provides the ability to allow sorting by using the sort query parameter:

• sort : Comma separated list of fields that are being preceded by "+" to display in ascending order, or "-" for descending order. For example: sort=+opportunity_value,-opportunity_id

Note that if no +/- prefix is provided, the results will be sorted in ascending order (+) by default.

Note also, the fields used for sorting must exist for the requested record.

Schema Endpoints

Schema endpoints also provide the ability to sort their results by leveraging the sort query parameter as follows:

• Multi-field sorting is not allowed, i.e. it is not possible to sort by more than one field.
• Fields in the sort query param should be prefixed with a ‘+’ for ascending order or ‘-’ for descending order.
• If no +/- prefix is provided, the results will be sorted in ascending order (+).
• The results will be sorted as case insensitive.

Some examples are:

• Get all Knowledge Models: /knowledge-models?sort=+id
• Get all Filters for a Knowledge Model: /knowledge-models/{km_id}/filters?sort=-name
• Get all Records for a Knowledge Model: /knowledge-models/{km_id}/records?sort=id
• Get all Fields (attributes, augmented attributes and flags) for a Record of a Knowledge Model: /knowledge-models/{km_id}/records/{record_id}?sort=-id

Fields Selection

The /data endpoint allows API consumers to retrieve data for the following record objects:

• Attributes
• Augmented Attributes In order to do so, the ids of the objects to be retrieved must be listed as part of the fields query parameter:
• fields: A comma-separated list of attribute, augmented attribute or flag ids for a given knowledge model record.

Note that:

• fields is a mandatory query parameter.
• The maximum number of objects per record that can be retrieved through the API Data endpoint is 200.
• fields query parameter is case-sensitive. The object ids must match the object ids returned by the record. schema endpoint ( /knowledge-models/{km_id}/records/{record_id} ).
• Each object id must be separated by a comma or alternatively, use the fields query parameter multiple times.

Example:

fields=ACTIVE,Category,APPROVAL or fields=ACTIVE&fields=Category&fields=APPROVAL 

Filtering

Filtering allows API consumers to only retrieve the desired records and not the whole data set. There are two ways to filter the data returned by the /data endpoint.

Predefined Knowledge Model Filters

1. Apply filters pre-defined in the Knowledge Model by using the filters query parameter.
   Filters: A comma-separated list of filter ids.
   
   Note that:
   • The Celonis Platform Knowledge Model allows filters to be defined using the FILTER PQL operator as explained in the Celonis Platform documentation .
   • You can apply multiple filters by providing multiple filter ids separated by commas or use the filters parameter multiple times.
   • Remember this parameter is case sensitive. The filter ids must match the one returned by the filter schema endpoint ( /knowledge-models/{km_id}/filters ).

Examples:

filters=not_null_orders,delayed_orders or filters=not_null_orders&filters=delayed_orders

Ad-hoc Filter Expressions

2. Filter expression using the filterExpr query parameter.
   This parameter allows filtering of record data using a provided value of a field through an expression.

Legacy Filter Expressions

1. First pattern field operator value where: * field is a filterable attribute, augmented attribute or flag of a record. This value is case-sensitive and should be equal to the attribute id. * operator is a comparison operator. For this pattern, it is currently supported by following operators: * eq(=), gt(>), lt(<), ne(!=), ge(>=), le(<=), is(IS) and in(IN) * value is a constant, such as a Salesforce Opportunity ID or a number/text.
   2. Second pattern operator(field,value) where:
      • operator is a string search operator,similar to the LIKE in the SQL clause. For this pattern, it is currently supported by following operators:
        • startswith, endswith and contains

It is also possible to combine filters with logical operators and, or, not and parenthesis.

• Example of the first pattern
  Copy

  Copied

  filterExpr=field1 eq value,field2 ne value

• Example of the second pattern
  Copy

  Copied

  filterExpr=startswith(field1,value),contains(field2,value)

• Example with first pattern, second pattern and or
  Copy

  Copied

  filterExpr=startswith(field1,value) or filed3 eq value,contains(field2,value) or field3 eq value

• Example with not
  Copy

  Copied

  filterExpr=not startswith(field1,value) or filed3 eq value,not (contains(field2,value) or field3 eq value)

Example:

{
    "page": 0,
    "pageSize": 100,
    "total": 1,
    "sort": [],
    "content": {
        "headers": [
            {
                "id": "description",
                "name": "Description",
                "type": "string",
                "aggregation": false,
        ===>    "filterable": true,  
                "sortable": true
            }
            {
                "id": "count",
                "name": "Count",
                "type": "integer",
                "aggregation": true,
        ===>    "filterable": false,
                "sortable": true
            }
        ],
        "data": [
            {
                "description": "total"
                "count": 50
            }
        ]
    }
}

You can apply multiple filters separated by commas or use the filterExpr parameter multiple times separated by “&”.

Examples:

filterExpr=opportunity_id eq AA00022475H, opportunity_value gt 50000

filterExpr=opportunity_id eq AA00022475H& 
filterExpr=opportunity_value gt 50000

New strongly-typed filter expression syntax

Described in detail in OData Filtering Syntax

Differences between legacy and new filter expression syntax

Selecting distinct data rows

If your request returns duplicate rows, then the way to deduplicate is by using the parameter options=distinct

⚠️ If a date field is included in the request: Please make sure to use ROUND_DAY in the PQL definition of the attribute in the Knowledge Model record !

Subscription to Trigger API Features

Team Rate limiting

Subscription to Trigger is enforcing rate limiting. This means for a team, only a certain number of subscriptions can be created. In addition, no matter how many subscriptions are created, there is a maximum number of events that can be emitted from the API to the third parties consumers.

If you reach the maximum number of subscriptions, you'll need to delete an existing subscription in order to create a new one.

If you reach the maximum number of events emitted from the API, the rest of data produced by the Celonis Platform will be discarded.

If you are approaching your daily quota, you'll be informed via emails to your admin account. The first email will be sent when 80% of the quota is exceeded, letting your admin account know that you're over that percentage. The second email will be sent as soon as you exceed 100% of the quota.

Currently, the API has the following default limits:

Table 1. Default rate limits:

Creating a new Subscription

To create a new subscription for an Celonis Platform Trigger you need to consume the next endpoint.

POST
/knowledge-models/{{km-id}}/triggers/{{trigger-id}}/subscriptions

{
    "name": "new subscription with headers",
    "callbackInfo": {
        "uri": "https://my-awesome-webhook.com",
        "protocol":"HTTPS",
        "headers": {
            "Authorization" : "my bearer token"
        }
    }
}

km-id refers to the knowledge model that is existing at the Celonis Platform. trigger-id refers to the trigger that is existing at the Celonis Platform.

All the attributes (with the exception of the headers) of the payload are mandatory.

name must be shorter than 1024. callbackInfo.uri uri used for the API to forward events emitted from Celonis Platform. callbackInfo.protocol the protocol used to forward events emitted from the Celonis Platform. It must fit with the callbackInfo.uri attribute. Otherwise it will fail into a Bad Request. callbackInfo.headers map of key/value regarding all the headers that will be attached from the API when the event is forwarded to the consumer.

Configure the Subscription with SKIP_ERRORS

You can include the configurations: ["SKIP_ERRORS"] in the payload, to apply extra capabilities to the subscription.

By now, the only value allowed is "SKIP_ERRORS", that means, even if the subscription started to fail, it will always retry sending the next event.

Bear in mind, a Subscription with "SKIP_ERRORS", will never get suspended, but you can't perform the manual reconcile.

This is an example about how to add the "SKIP_ERRORS" when creating a subscription.

POST
/knowledge-models/{{km-id}}/triggers/{{trigger-id}}/subscriptions

{
    "name": "new subscription with headers",
    "configurations" ["SKIP_ERRORS"],
    "callbackInfo": {
        "uri": "https://my-awesome-webhook.com",
        "protocol":"HTTPS",
        "headers": {
            "Authorization" : "my bearer token"
        }
    }
}

Updating an existing Subscription

To update an existing subscription, you need to consume the next endpoint.

PUT
/subscriptions/{{subscription-id}}

{
    "name": "updated subscription with headers",
    "callbackInfo": {
        "uri": "https://my-awesome-webhook.com",
        "protocol":"HTTPS",
        "headers": {
            "Authorization" : "my bearer token"
        }
    }
}

subscription-id refers to the subscription to update. It must exist in the API.

All the attributes of the payload (with the exception of the headers) are mandatory.

• name the subscription name, which must be must be shorter than 1024 characters.
• callbackInfo.uri uri used for the API to forward events emitted from Celonis Platform.
• callbackInfo.protocol the protocol used to forward events emitted from the Celonis Platform. It must fit with the callbackInfo.uri attribute. Otherwise it will fail into a Bad Request.
• callbackInfo.headers map of key/value regarding all the headers that will be attached from the API when the event is forwarded to the consumer.

In case of updating a Subscription which was failing, it will be set back to ACTIVE status.

Update the Subscription with SKIP_ERRORS

You can include the configurations: ["SKIP_ERRORS"] in the payload, to apply extra capabilities to the subscription.

By now, the only value allowed is "SKIP_ERRORS", that means, even if the subscription started to fail, it will always retry sending the next event.

Bear in mind, a Subscription with "SKIP_ERRORS", will never get suspended, but you can't perform the manual reconcile.

This is an example about how to add the "SKIP_ERRORS" when updating a subscription.

PUT
/subscriptions/{{subscription-id}}

{
    "name": "updated subscription with headers",
    "configurations" ["SKIP_ERRORS"],
    "callbackInfo": {
        "uri": "https://my-awesome-webhook.com",
        "protocol":"HTTPS",
        "headers": {
            "Authorization" : "my bearer token"
        }
    }
}

Pausing a Subscription

To pause an active subscription, you need to consume the next endpoint.

PATCH
/subscriptions/{{subscription-id}}/pause

subscription-id refers to the subscription to pause. It must exist in the API.

Once the subscription is paused, it will not forward more events from Celonis Platform, those events aren't lost, they will be send to the consumer once the subscription is resumed.

Resuming a Subscription

To resume a paused subscription, you need to consume the next endpoint.

PATCH
/subscriptions/{{subscription-id}}/resume

subscription-id refers to the subscription to resume. It must exist in the API.

Once the subscription is resumed, it will forward new events emitted from Celonis Platform, as well as the data that was pending when the subscription was paused.

Replaying a Subscription

To replay a non suspended subscription, you need to consume the next endpoint.

PATCH
/subscriptions/{{subscription-id}}/replay?fromOffset=1

Once the subscription is replayed, it will forward events from the given offset.

The fromOffset url parameter is optional, in case of not adding it to the url, the replay will start from the beginning of the Subscription. In case of adding a "from", it must be a value between 0 and the latest received from the Celonis Platform.

In case of having a Paused subscription, it will go back to Active status.

In case of having a Failed subscription, it will go back to Active in case of being able to forward events to the Subscription's Callback. If the Callback is still failing, it will go back to Failed automatically.

This operation can't be executed for Suspended subscriptions.

Unsubscribing

To remove an existing subscription you need to consume the next endpoint.

DELETE
/subscriptions/{{subscription-id}}

subscription-id refers to the subscription to unsubscribe. It must exist in the API.

Once this action is executed, the subscription will be no longer visible at Celonis Platform, and all its data is removed from the API.

Manual extraction (reconcile endpoint)

If the subscription is failing, in the meanwhile the subscription is fixed, is possible to manually extract the data from Celonis Platform in short batches. This is what is called the Reconciliation.

To manually extract data, you need to consume the next endpoint.

PATCH
/subscriptions/{{subscription-id}}/events

subscription-id refers to the subscription to reconcile. It must exist in the API.

Once this action is executed, the next 50 events will be taken. Bear in mind this action will remove those events from the system.

{
    "pageSize": 2,
    "total": 0,
    "content": [
        {
            "subscriptionId": "XXX-YYY-ZZZZ",
            "triggerId": "97185b79-d6e9-4425-bbee-225627533149",
            "signal": "{\"id\":\"a8ffef1d-aba0-4520-ae1a-02cecc7b7e4e\",\"recordIdentifier\":\"800A3000531000\",\"typedRecordIdentifier\":\"800A3000531000\",\"attributes\":[{\"attributeId\":\"ATTRIBUTE_1\",\"signalId\":\"a8ffef1d-aba0-4520-ae1a-02cecc7b7e4e\",\"value\":null,\"columnType\":\"STRING\"}],\"creationDate\":1699010395821}"
        },
        {
            "subscriptionId": "XXX-YYY-ZZZZ",
            "triggerId": "97185b79-d6e9-4425-bbee-225627533149",
            "signal": "{\"id\":\"8cf03551-471c-41f0-a85c-e88646eef354\",\"recordIdentifier\":\"80000000130071000\",\"typedRecordIdentifier\":\"80000000130071000\",\"attributes\":[{\"attributeId\":\"ATTRIBUTE_1\",\"signalId\":\"8cf03551-471c-41f0-a85c-e88646eef354\",\"value\":\"D1\",\"columnType\":\"STRING\"}],\"creationDate\":1699010395821}"
        }
    ]
}

pageSize refers to the events returned in the page. total refers to the total events still at the API. content events as raw JSON.

Untransferred data

Untransferred data means the data unable to forward to the consumer but saved at the API to retry later. The data can be saved at the API under two circunstances:

• The subscription started to fail and is still within the time window to sort it out (after 8 hours the service is suspended).
• The subscription is paused.

Untransferred data can be manually extracted using the reconcile endpoint. But for active subscriptions with data remaining in the API, every 10 minutes our API will retry to forward the data to the consumer.

Finally, the data is kept at the API for 7 days, after the period of time all the pending data will be discarded.