Multiple systems offline

Incident Report for VitalSource

Postmortem

After working the issue, we identified that the root cause was a Google Cloud Networking issue which essentially took several of our Global Load Balancers offline. We escalated the issue with Google Cloud, and they acknowledge an evolving issue with their global networking and directed us to restart some of our systems to fix the issue. That closed the issue.  We are following up with Google Cloud on better notifications when these incidents evolve. 

 All our alerting and notification procedures worked as designed.

Posted Jul 22, 2021 - 10:34 EDT

Resolved

All issues are resolved
Posted Jul 22, 2021 - 03:56 EDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jul 22, 2021 - 03:09 EDT

Update

We are continuing to investigate this issue.
Posted Jul 22, 2021 - 02:47 EDT

Investigating

We are experiencing issues across the VitalSource ecosystem, we are investigating.
Posted Jul 22, 2021 - 02:45 EDT
This incident affected: V3/V4 APIs, VitalSource Launch - LTI, LMS, SAML, Office365, OAuth2 and other integrations, Bookshelf Online, Manage Platform, and Store, Sampling and Ecommerce Platform.