Some systems are experiencing issues

Past Incidents

Tuesday 24th January 2023

Database Site and API performance issues

We're investigating database issues, the site may be slow or unresponsive until this is resolved.

  • The rebuild has completed and performance has returned to normal.

  • The rebuild is going faster than expected, we're currently at 75%.

  • Performance has been stable over the last 10 hours, the RAID array is 45% rebuilt.

  • Performance seems to be stabilising, however it may be erratic over the next 96 hours or so. The initial issue was caused by some corruption on the database due to a degraded RAID array on our primary database server. While in a degraded state replication lagged and we started seeing some strange data replication. The faulty disk has been replaced and we've failed over to your secondary database server for most queries while the array rebuilds. Site and API performance may be slow until this is complete.

    Apologies for the issues.

  • The issue has been identified, we're working on a fix.

  • We've taken the API offline while we investigate.

  • Monday 23rd January 2023

    No incidents reported

    Sunday 22nd January 2023

    No incidents reported

    Saturday 21st January 2023

    No incidents reported

    Friday 20th January 2023

    No incidents reported

    Thursday 19th January 2023

    No incidents reported

    Wednesday 18th January 2023

    No incidents reported