Shoreline closes $35M series B - Read the details

Disk Resize/Disk Clean

< Solutions Library

Highlights

Customer Experience Impact:

High – potentially hours of downtime

Frequency of Occurrence:

Weekly for fleets with hundreds of nodes

Manual repair elapsed time:

~ 1-6 hours

Shoreline repair elapsed time:

~ 1-2 minutes

Provision from Terraform RegistryFree Trial

The Problem

When hard disks fill up they can cause catastrophic app failures that can require a lot of time and effort to recover. Disk full incidents can lead to wid-spread outages and data loss that can seriously damage customer experiences and lead to lost revenue.

When managing a large fleet, it can be tricky to keep track of every node and to identify those nodes that are reaching capacity. These types of issues can come up on a weekly basis for larger fleets, especially if the fleet is supporting a wide variety of customers, each with several nodes.

Recovering from a disk full incident can also take hours. In some cases, the disk must be physically disconnected from the original server and attached to another server so that it can be repaired. In other cases, an SRE must perform a series of checks to understand why the disk is filling. Often there are multiple potential causes so the root cause analysis can take several hours.

The Solution

With Shoreline, it is easy to create an alarm that regularly checks disk capacity across every node in your fleet. Once a filling disk is detected, companies can select from two options to address the problem:

  • 1. Resize the disk by adding a preset number of gigabytes, up to a pre-set limit. If the limit is reached, then the on-call team is alerted
  • 2. Back up a set of files to a cloud storage service like S3 and then delete the files

In both of the above scenarios, the disk is automatically resized without human intervention and the entire event is tracked in Shoreline’s audit trails and dashboards. This allows customers to see how often these situations occur and when necessary, ask engineering to fix an underlying issue in the application.

Ready to get started?

Shoreline helps you eliminate repetitive tickets and increase your availability at the same time. Get started today with a free trial.