---
id: f60f5e5a-616d-11ee-8c99-0242ac120002
---

# Troubleshooting Compaction Merging SSTables in Cassandra
---

Compaction is a process in Cassandra that merges multiple SSTables (Sorted String Tables) into a single SSTable, eliminating any redundant data and improving read performance. However, sometimes compaction can fail due to various reasons such as insufficient disk space or corrupted data, resulting in degraded performance or even complete failure of the database. Troubleshooting compaction merging SSTables involves identifying and resolving the root cause of such failures to ensure the smooth functioning of the Cassandra database.

### Parameters
```shell
export DATA_DIRECTORY="PLACEHOLDER"

export CASSANDRA_LOG_FILE="PLACEHOLDER"

export THRESHOLD="PLACEHOLDER"

export NODE1_IP="PLACEHOLDER"

export NODE2_IP="PLACEHOLDER"

export NODE3_IP="PLACEHOLDER"

export NODES[@]="PLACEHOLDER"

export USERNAME="PLACEHOLDER"
```

## Debug

### Check the size of the SSTables in the data directory
```shell
du -h ${DATA_DIRECTORY}
```

### Check if disk space is running low
```shell
df -h
```

### Check the Cassandra log files for any compaction-related errors
```shell
tail -f ${CASSANDRA_LOG_FILE}
```

### Display the compaction history using nodetool.
```shell
nodetool compactionhistory
```

### Check the current compaction throughput using nodetool.
```shell
nodetool getcompactionthroughput
```

### Check the status of running compaction
```shell
nodetool compactionstats
```

### Stop any running compaction
```shell
nodetool stop COMPACTION
```

### Check for any corrupted SSTables
```shell
nodetool verify
```

### Repair any corrupted SSTables
```shell
nodetool repair
```

### Restart the compaction process
```shell
nodetool compact
```

### Check the status of the repaired SSTables
```shell
nodetool status
```

### Monitor compaction progress
```shell
nodetool compactionstats
```

## Repair

### Check for available disk space on all the nodes in the Cassandra cluster. If there is insufficient disk space available, add more storage capacity or remove unnecessary data to free up space.
```shell
bash

#!/bin/bash



# Set the threshold for minimum free disk space in GB

THRESHOLD=${THRESHOLD}



# Array of Cassandra node IP addresses

NODES=(${NODE1_IP} ${NODE2_IP} ${NODE3_IP})



for node in "${NODES[@]}"

do

  # Check the available disk space on each node

  free_space=$(ssh ${USERNAME}@$node "df -BG /dev/mapper/data | awk 'NR==2 {print \$4}' | tr -d 'G'")



  # Compare the available disk space with the threshold

  if [ $free_space -lt $THRESHOLD ]

  then

    # Alert if there is insufficient disk space

    echo "Warning: Node $node has only $free_space GB of free disk space."

  fi

done


```


Compaction is a process in Cassandra that merges multiple SSTables (Sorted String Tables) into a single SSTable, eliminating any redundant data and improving read performance. However, sometimes compaction can fail due to various reasons such as insufficient disk space or corrupted data, resulting in degraded performance or even complete failure of the database. Troubleshooting compaction merging SSTables involves identifying and resolving the root cause of such failures to ensure the smooth functioning of the Cassandra database.


This incident type refers to a situation where there is a significant delay in the execution of queries on a Cassandra cluster. This delay can cause the system to become unresponsive and result in slower performance. It may be caused by a variety of factors such as an increase in traffic, inefficient queries, or hardware issues. The issue can impact the functionality of the system and requires immediate attention to prevent further disruption.


Slow Query Performance on Cassandra Cluster.

In this incident type, there is an issue with a Cassandra cluster where one or more disks are running slow. This can cause performance issues and potentially lead to data loss or downtime. The goal is to identify and address the specific disk(s) causing the problem in order to restore normal cluster operations.


Slow Disk in Cassandra Cluster

This incident type refers to a performance degradation issue in a Cassandra database caused by the new generation garbage collection (GC) process. Garbage collection is a process by which the unused memory occupied by objects in a program is cleared. In some cases, the new generation GC process may take longer than usual, causing performance issues in the Cassandra database. This could lead to slowness in queries or even crashing of the database.


Performance Degradation Due to New Generation Garbage Collection on Cassandra

Misconfigured compaction strategy is an incident type that occurs when the way data is compacted within a database cluster is not properly configured. This can lead to excessive compaction activity that can negatively impact the cluster's performance and stability. The incident can cause slow query response times, increased disk usage, and high CPU utilization, among other issues. It is crucial to identify and fix the misconfiguration promptly to ensure the cluster's smooth operation.


Misconfigured Compaction Strategy.

This incident type refers to a situation where a delay or slowness occurs in a system that uses Cassandra database due to the shared storage. Shared storage means multiple servers are accessing the same storage unit, and this can cause latency issues. This type of incident can lead to performance degradation, and it needs to be addressed promptly to ensure optimal system performance.


Latency Caused by Shared Storage in Cassandra

```shell
export DATA_DIRECTORY="PLACEHOLDER"

export CASSANDRA_LOG_FILE="PLACEHOLDER"

export THRESHOLD="PLACEHOLDER"

export NODE1_IP="PLACEHOLDER"

export NODE2_IP="PLACEHOLDER"

export NODE3_IP="PLACEHOLDER"

export NODES[@]="PLACEHOLDER"

export USERNAME="PLACEHOLDER"
```


### Check the size of the SSTables in the data directory

```shell
du -h ${DATA_DIRECTORY}
```

### Check if disk space is running low

```shell
df -h
```

### Check the Cassandra log files for any compaction-related errors

```shell
tail -f ${CASSANDRA_LOG_FILE}
```

### Display the compaction history using nodetool.

```shell
nodetool compactionhistory
```

### Check the current compaction throughput using nodetool.

```shell
nodetool getcompactionthroughput
```

### Check the status of running compaction

```shell
nodetool compactionstats
```

### Stop any running compaction

```shell
nodetool stop COMPACTION
```

### Check for any corrupted SSTables

```shell
nodetool verify
```

### Repair any corrupted SSTables

```shell
nodetool repair
```

### Restart the compaction process

```shell
nodetool compact
```

### Check the status of the repaired SSTables

```shell
nodetool status
```

### Monitor compaction progress

```shell
nodetool compactionstats
```


### Check for available disk space on all the nodes in the Cassandra cluster. If there is insufficient disk space available, add more storage capacity or remove unnecessary data to free up space.

```shell
bash

#!/bin/bash



# Set the threshold for minimum free disk space in GB

THRESHOLD=${THRESHOLD}



# Array of Cassandra node IP addresses

NODES=(${NODE1_IP} ${NODE2_IP} ${NODE3_IP})



for node in "${NODES[@]}"

do

  # Check the available disk space on each node

  free_space=$(ssh ${USERNAME}@$node "df -BG /dev/mapper/data | awk 'NR==2 {print \$4}' | tr -d 'G'")



  # Compare the available disk space with the threshold

  if [ $free_space -lt $THRESHOLD ]

  then

    # Alert if there is insufficient disk space

    echo "Warning: Node $node has only $free_space GB of free disk space."

  fi

done


```


Troubleshooting Compaction Merging SSTables in Cassandra

Overview

Parameters

Debug

Check the size of the SSTables in the data directory

Check if disk space is running low

Display the compaction history using nodetool.

Check the current compaction throughput using nodetool.

Check the status of running compaction

Stop any running compaction

Check for any corrupted SSTables

Repair any corrupted SSTables

Restart the compaction process

Check the status of the repaired SSTables

Monitor compaction progress

Repair

Check for available disk space on all the nodes in the Cassandra cluster. If there is insufficient disk space available, add more storage capacity or remove unnecessary data to free up space.

Learn more

Related Runbooks

Slow Query Performance on Cassandra Cluster.

Slow Disk in Cassandra Cluster

Performance Degradation Due to New Generation Garbage Collection on Cassandra

Misconfigured Compaction Strategy.

Support