Is MindTickle, Inc. Down?

# **Incident Summary** On June 12, 2026, Mindtickle experienced a service disruption caused by an unexpected surge in processing activity within one of our backend workflows. The resulting increase in downstream traffic exceeded the capacity of a critical platform dependency, leading to service degradation and eventual unavailability. The issue was mitigated by reducing processing throughput and allowing affected systems to recover. **Impact Duration:** **3 hours and 15 minutes**. ## **Impact** During the incident window, customers may have experienced: * Delays in invitation-related workflows * Elevated latency and intermittent errors across affected services * Temporary degradation of platform functionality dependent on the impacted backend systems ## **Incident Timeline** * ‌**Jun 12, 03:00 PM**: A critical backend dependency becomes saturated, resulting in service degradation and elevated error rates. * **Jun 12, 03:15 PM**: The issue is detected through automated monitoring and customer reports. Investigation begins. * **Jun 12, 04:50 PM**: Mitigation measures are implemented to reduce processing throughput and stabilize affected systems. * **Jun 12, 05:46 PM**: Platform functionality is restored fully ## **Root Cause** A backend workflow generated a significantly higher volume of processing activity than historical norms. As the system worked through the resulting backlog, requests propagated through multiple downstream services, creating sustained load across the platform. While each component behaved as designed, the combined volume exposed capacity and protection gaps within the processing chain. The resulting amplification effect overwhelmed a critical backend dependency, causing service degradation and eventual unavailability. ## **Resolution** The team mitigated the incident by reducing the rate at which backlog processing occurred, allowing downstream systems to stabilize and recover. Once traffic levels returned to normal operating ranges, affected services resumed normal operation and platform functionality was fully restored. ## **Corrective and Preventive Actions** * Introduce rate limiting and additional safeguards for high-volume processing workflows to prevent a single workflow from generating excessive downstream load. * Implement backpressure, circuit-breaking, and load-shedding mechanisms across critical processing paths to better protect downstream dependencies during traffic spikes. * Review platform capacity limits, concurrency settings, and monitoring to improve resilience and provide earlier visibility into abnormal processing patterns. --- [2026-06-14T20:36:34.837-07:00] (resolved) The incident has been resolved and services are operating normally. We will continue to monitor the platform to ensure stability. A detailed Root Cause Analysis will be shared shortly. --- [2026-06-12T18:01:52.825-07:00] (monitoring) Mitigation has been deployed and services are operating normally. We are continuing to monitor the platform closely to ensure stability. --- [2026-06-12T17:51:35.292-07:00] (identified) We have identified the cause of the elevated error rates and are deploying mitigation measures. We are monitoring the platform closely and working to restore normal service. --- [2026-06-12T17:35:28.172-07:00] (investigating) We are currently experiencing a partial service disruption on Mindtickle platform. Users may encounter errors, intermittent failures, or degraded performance across the platform. Our engineering team is actively investigating and mitigating the issue while closely monitoring recovery efforts.

On June 2nd, between 7:19 AM PDT and 9:21 AM PDT, our third-party recording provider experienced a disruption that prevented the Call AI bot from joining meetings. A limited number of customers were impacted, and unfortunately, the affected recordings are non-recoverable. We sincerely apologize for the disruption and any impact to your workflows. The issue is fully resolved, and recordings are functioning normally. If your account was affected, your account team will reach out with specific details.

All impacted systems are now operating normally, and Two-Way Roleplay functionality has been fully restored. We will continue to monitor the systems closely. The incident is now resolved. --- [2026-05-21T10:49:59.152-07:00] (monitoring) The impacted systems are now operational again, and roleplay workflows are functioning successfully. We are continuing to monitor the systems closely to ensure stability. --- [2026-05-21T10:18:25.286-07:00] (identified) We are seeing partial recovery in the impacted service, and some roleplay attempts are now completing successfully. We continue to work with the third-party provider toward full restoration. --- [2026-05-21T10:07:24.000-07:00] (identified) Recovery efforts are still ongoing, and we continue to work closely with our third-party partners to restore the impacted service. Our teams remain actively engaged on the issue. --- [2026-05-21T08:59:11.421-07:00] (identified) We have identified the root cause as a service disruption at a third-party provider that powers parts of the Two-Way Roleplay workflow. We are actively working with the vendor to restore the service and recover impacted functionality as quickly as possible. --- [2026-05-21T08:58:59.686-07:00] (investigating) Users may experience issues starting Two-Way Roleplay sessions. In addition admins may be unable to preview roleplays. The team is actively investigating the issue and working toward resolution.