Fully Resolved - Intermittent Lengthy Transaction Times
Incident Report for Cardknox
Postmortem

Resolved - A small subset of merchants experienced lengthy transaction and/or reporting times intermittently during the affected period. While transaction processing was not completely disrupted, the lengthy transaction times may have caused merchant systems to time out before a transaction response was returned, therefore there may be approvals which were not accounted for by a merchant.

The cause was identified to be faulty hardware on one of the back-end processing nodes. The faulty hardware did not cause a complete failure, which delayed detection of the issue.

The identified hardware was replaced, causing a momentary disruption and returning service to normal.

Next Steps

We understand that even a small disruption to normal processing can be extremely disruptive to our merchants and commit to ensuring such events do not happen again. To this end we will be implementing improved monitoring to detect system slowdowns. Additionally, we will be performing emergency maintenance to migrate the affected node. The status of all transactions during the affected period are available on the Portal and via the Reporting API. Upon request we can also send a customized transaction report and assist with any reconciliation efforts.

Posted Dec 12, 2018 - 12:01 EST

Resolved
A small subset of merchants experienced lengthy transaction and/or reporting times intermittently during the affected period. While transaction processing was not completely disrupted, the lengthy transaction times may have caused merchant systems to time out before a transaction response was returned, therefore there may be approvals which were not accounted for by a merchant. The root cause was identified and the issue is completely resolved.
Posted Dec 07, 2018 - 17:00 EST