Monitoring and Replaying the Errors
Overview
When pushing data to Celonis via the Standard Data Ingestion API, the file transfer can be successful but the ingestion still fails because of data related issues. These errors will be displayed on the Monitoring page with details to guide the user on how to fix them. After fixing any problems, the user can replay the ingestion.
The Monitoring tab can be accessed by going to the three-dot menu of the Data Connection and selecting "Monitoring".
The errors, if any, are shown grouped by Table and Error type. In the screenshot below, there are five tables (corrupted, flatfile, flatfileagecolumn, flatfileschema_error, unnesting). Expanding the table shows all the errors for the given table grouped by the Types.
Click on the type to view the detailed errors.
If the error message is not enough to resolve the problem, the user can download a sample file from the failed batch to troubleshoot the issue further.
The sections below explain the different Error types, possible solutions, and how to replay the failed files.
Error Types and Explanations
PK NOT FOUND IN INPUT FILE ERROR
The error occurs when the PK column(s) defined in the Table Configuration is missing in the pushed file. If the PK definition in the Table Configuration is wrong, you can remove the irrelevant column and replay the file.
AGE COLUMN NOT FOUND IN INPUT FILE ERROR
The error occurs when the Age Column(s) defined in the Table Configuration is missing in the pushed file. You can fix the Table Configuration and replay the file.
DATAPOOL SCHEMA MISMATCH ERROR
The error occurs when the database table in Celonis and the pushed file have different schemas. For example, the column XYZ is of timestamp type in Celonis database, but the pushed file has floats in the column XYZ. The fix will depend on the root cause of the problem. For example, if the database table in Celonis has an incorrect column type, the user can drop the problematic column and replay the file. Make sure that no data is lost in this process.
UNNESTING ERROR
This is a generic category for all kinds of errors that can happen during the unnesting process. For example, the schema of the ingested file can be different from the schema defined in the table configuration. Or, the PK defined in the nested table configuration might be missing in the file. If the issue can be fixed via a change in the configuration, the user can make the change and replay the file.
INTERNAL ERROR
This is a generic category for all unknown and unexpected errors. For further troubleshooting, contact customer support and share the support ID associated with the error.
Replay the Failed File
The errors related to the Table Configuration, i.e. wrong PK or Age Column, can typically be fixed and replayed.
The replay can be initiated at a table level and includes ALL the errors, irrespective of the applied time filters. So even if the error is not visible in the Monitoring due to the applied time filters, it will still be replayed. If the replay is successful, the error(s) will disappear. However, if the root cause issue has not been fixed, the error will appear again.
