---
id: 5bcc7dd3-b827-416f-9290-cdc5a7d2efc4
---

# Unused Topic Partition in Kafka
---

This incident type refers to the detection of an unused topic partition in Kafka Platform, a popular data streaming platform. Unused partitions can increase broker overhead and reduce efficiency, so it is recommended to delete them. The incident is triggered by a test notification or a monitoring query that checks the value of relevant metrics. The incident is resolved once the partition is deleted or confirmed to be intentional.

### Parameters
```shell
# Environment Variables

export TOPIC_NAME="PLACEHOLDER"

export KAFKA_BOOTSTRAP_SERVER="PLACEHOLDER"

export PARTITION_NUMBER="PLACEHOLDER"

export PARTITION_DIRECTORY="PLACEHOLDER"

export CONSUMER_GROUP="PLACEHOLDER"

export NUMBER_OF_THE_PARTITION="PLACEHOLDER"

export ZOOKEEPER_CONNECTION_STRING="PLACEHOLDER"

```

## Debug

### Check if the topic exists in the Kafka cluster
```shell
kafka-topics.sh --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME}
```

### Check if the partition is marked as "unused"
```shell
kafka-log-dirs.sh --describe --topic-list ${TOPIC_NAME} --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER} | grep -A 2 "${PARTITION_NUMBER}"
```

### Check the disk usage for the partition directory
```shell
du -sh ${PARTITION_DIRECTORY}
```

### Check the broker overhead for the partition
```shell
kafka-consumer-groups.sh --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER} --describe --group ${CONSUMER_GROUP} --topic ${TOPIC_NAME}
```

### Check the status of the Kafka brokers
```shell
kafka-broker-api-versions.sh --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER}
```

### Check the status of the ZooKeeper nodes
```shell
zkCli.sh -server $ZOOKEEPER_CONNECTION_STRING ls /brokers/ids
```

## Repair

### Delete the unused topic partition if it's no longer needed. You can use the Kafka command line tool to delete the topic partition.
```shell
#!/bin/bash

# Set the variables

TOPIC=${TOPIC_NAME}

PARTITION=${NUMBER_OF_THE_PARTITION}

ZOOKEEPER=${ZOOKEEPER_CONNECTION_STRING}

# Stop the Kafka broker

/bin/kafka-server-stop.sh

# Delete the topic partition

/bin/kafka-topics.sh --delete --zookeeper $ZOOKEEPER --topic $TOPIC --partition $PARTITION

# Start the Kafka broker

/bin/kafka-server-start.sh

```

This incident type refers to the detection of an unused topic partition in Kafka Platform, a popular data streaming platform. Unused partitions can increase broker overhead and reduce efficiency, so it is recommended to delete them. The incident is triggered by a test notification or a monitoring query that checks the value of relevant metrics. The incident is resolved once the partition is deleted or confirmed to be intentional.


This incident type refers to a situation where there is an issue with the communication between Kafka brokers due to network partitioning. A network partition is a situation where a network is split into two or more parts, and communication between them is disrupted. This issue can lead to data inconsistency, loss of messages, delayed message delivery, and other related problems. As a result, it is important to quickly detect and resolve these incidents to prevent any downstream impact on the system.


Parameters

This incident type refers to an issue where the retention size for a Kafka topic is set too low. The retention size is the amount of time that Kafka retains messages in a topic before they are deleted. When the retention size is set too low, messages may be deleted before they can be consumed or processed, which can result in data loss or other issues. This can be caused by misconfiguration or changes in the data ingestion rate. It is important to monitor retention sizes and adjust them as necessary to prevent data loss and ensure smooth data processing.


Low Retention Size for Kafka Topic.

This type of incident occurs when the offset of a Kafka topic decreases unexpectedly. In Kafka, an offset is a unique identifier that represents the position of a message within a partition. When the offset decreases, it means that some messages have been lost or deleted, which can cause data inconsistencies or other issues within the system. This incident type is critical for Kafka-based systems as it affects data integrity and can lead to data loss.


Kafka topic offset decreased incident.

This type of incident occurs when a node in the Kafka cluster is not being fully utilized. Kafka is a distributed streaming platform, and each node in the cluster is responsible for processing and storing a portion of the data. If a node is underutilized, it means that it is not processing as much data as it should be, which can lead to performance issues and potential data loss. Therefore, it is important to identify and address underutilized nodes in a Kafka cluster to ensure optimal performance and data reliability.


Kafka node underutilization check incident.

```shell
# Environment Variables

export TOPIC_NAME="PLACEHOLDER"

export KAFKA_BOOTSTRAP_SERVER="PLACEHOLDER"

export PARTITION_NUMBER="PLACEHOLDER"

export PARTITION_DIRECTORY="PLACEHOLDER"

export CONSUMER_GROUP="PLACEHOLDER"

export NUMBER_OF_THE_PARTITION="PLACEHOLDER"

export ZOOKEEPER_CONNECTION_STRING="PLACEHOLDER"

```


### Check if the topic exists in the Kafka cluster

```shell
kafka-topics.sh --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME}
```

### Check if the partition is marked as "unused"

```shell
kafka-log-dirs.sh --describe --topic-list ${TOPIC_NAME} --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER} | grep -A 2 "${PARTITION_NUMBER}"
```

### Check the disk usage for the partition directory

```shell
du -sh ${PARTITION_DIRECTORY}
```

### Check the broker overhead for the partition

```shell
kafka-consumer-groups.sh --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER} --describe --group ${CONSUMER_GROUP} --topic ${TOPIC_NAME}
```

### Check the status of the Kafka brokers

```shell
kafka-broker-api-versions.sh --bootstrap-server ${KAFKA_BOOTSTRAP_SERVER}
```

### Check the status of the ZooKeeper nodes

```shell
zkCli.sh -server $ZOOKEEPER_CONNECTION_STRING ls /brokers/ids
```


### Delete the unused topic partition if it's no longer needed. You can use the Kafka command line tool to delete the topic partition.

```shell
#!/bin/bash

# Set the variables

TOPIC=${TOPIC_NAME}

PARTITION=${NUMBER_OF_THE_PARTITION}

ZOOKEEPER=${ZOOKEEPER_CONNECTION_STRING}

# Stop the Kafka broker

/bin/kafka-server-stop.sh

# Delete the topic partition

/bin/kafka-topics.sh --delete --zookeeper $ZOOKEEPER --topic $TOPIC --partition $PARTITION

# Start the Kafka broker

/bin/kafka-server-start.sh

```


Unused Topic Partition in Kafka

Overview

Parameters

Debug

Check if the topic exists in the Kafka cluster

Check if the partition is marked as "unused"

Check the disk usage for the partition directory

Check the broker overhead for the partition

Check the status of the Kafka brokers

Check the status of the ZooKeeper nodes

Repair

Delete the unused topic partition if it's no longer needed. You can use the Kafka command line tool to delete the topic partition.

Learn more

Related Runbooks

Network Partitioning Between Kafka Brokers.

Parameters

Low Retention Size for Kafka Topic.

Kafka topic offset decreased incident.

Support