Some systems are experiencing issues

Past Incidents

Tuesday 31st January 2023

No incidents reported

Monday 30th January 2023

No incidents reported

Sunday 29th January 2023

No incidents reported

Saturday 28th January 2023

No incidents reported

Friday 27th January 2023

No incidents reported

Thursday 26th January 2023

No incidents reported

Wednesday 25th January 2023

No incidents reported

Tuesday 24th January 2023

Database Site and API performance issues

We're investigating database issues, the site may be slow or unresponsive until this is resolved.

  • The rebuild has completed and performance has returned to normal.

  • The rebuild is going faster than expected, we're currently at 75%.

  • Performance has been stable over the last 10 hours, the RAID array is 45% rebuilt.

  • Performance seems to be stabilising, however it may be erratic over the next 96 hours or so. The initial issue was caused by some corruption on the database due to a degraded RAID array on our primary database server. While in a degraded state replication lagged and we started seeing some strange data replication. The faulty disk has been replaced and we've failed over to your secondary database server for most queries while the array rebuilds. Site and API performance may be slow until this is complete.

    Apologies for the issues.

  • The issue has been identified, we're working on a fix.

  • We've taken the API offline while we investigate.