The issue is now resolved. We're working on a limiting process for the API calls that were putting strain on our databases and should have something soon to avoid this from happening again in the future.
If you have any specific questions please contact support@iron.io or your dedicated support channel.
Posted Feb 02, 2017 - 11:44 PST
Update
We've discovered the source of the database slowdown and are taking corrective actions to build a permanent fix into the system so this does not happen again.
Thanks for your patience while we work through slower than usual queue times. The system did not lose jobs, but rather saw slowdowns in queue times.
Posted Feb 02, 2017 - 11:23 PST
Monitoring
The database has stabilized and we are sweeping projects for queued tasks. You may notice some tasks that ran long. Many of them probably finished just did not exit properly and mark as "complete". You can verify by viewing the task logs.
We'll update again as we monitor the system.
Posted Feb 02, 2017 - 10:47 PST
Update
The cause of the slowdowns is in a backend database under high load. We're searching for the source of the load and as soon as that's found we'll update this page. That said we've restarted the DB and it's mostly recovered so tasks should start flowing again.
Stand by.
Posted Feb 02, 2017 - 10:36 PST
Investigating
We have identified a system slow down with IronWorker. This is a high priority issue, and our Operations team is actively investigating. We will post an update as soon as information becomes available.
Posted Feb 02, 2017 - 10:04 PST
This incident affected: IronWorker Dedicated and IronWorker Public.