---
id: 521fc8ec-bfd0-442b-b45a-f404d0cbab08
---
# Kafka Offline Partition Detected.
---

When a Kafka partition becomes offline, it means that the partition is no longer available for reading and writing data. This can occur due to various reasons such as hardware failure, network issues, or software bugs. When a Kafka offline partition is detected, it can impact the availability and reliability of the entire system. It is important to quickly troubleshoot and resolve the issue to minimize the impact on the system.

### Parameters
```shell
export BOOTSTRAP_SERVER="PLACEHOLDER"

export TOPIC_NAME="PLACEHOLDER"

export PARTITION_NUMBER="PLACEHOLDER"

export PATH_TO_KAFKA_LOGS="PLACEHOLDER"

export NUM_PARTITIONS="PLACEHOLDER"

export KAFKA_CLUSTER_URL="PLACEHOLDER"
```

## Debug

### 1. Check if Kafka service is running
```shell
systemctl status kafka
```

### 2. List all topics in the cluster
```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --list
```

### 3. Describe the topic to see the number of partitions and their status
```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME}
```

### 4. Check the status of all partitions for a topic
```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME} | grep Partition
```

### 5. Check the status of a specific partition
```shell
kafka-log-dirs.sh --describe --bootstrap-server ${BOOTSTRAP_SERVER} --topic-list ${TOPIC_NAME} --partition-list ${PARTITION_NUMBER}
```

### 6. Check if the broker hosting the partition is offline
```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME} | grep ${PARTITION_NUMBER} | awk '{print $6}' | cut -d: -f1 | xargs -I {} sh -c '${PATH_TO_KAFKA_BIN}/kafka-broker-api-versions.sh --bootstrap-server {}:9092 --version'
```

### 7. Check the logs for any error messages related to the partition
```shell
tail -n 100 ${PATH_TO_KAFKA_LOGS}/server.log | grep ${TOPIC_NAME} | grep ${PARTITION_NUMBER}
```

## Repair

### Rebalance the partitions: If the partition is offline due to an imbalance in the Kafka cluster, rebalancing the partitions can help bring the partition back online.
```shell
bash

#!/bin/bash

# Rebalance the partitions

kafka-topics --bootstrap-server ${BOOTSTRAP_SERVER}  --alter --topic ${TOPIC_NAME} --partitions ${NUM_PARTITIONS}


```


When a Kafka partition becomes offline, it means that the partition is no longer available for reading and writing data. This can occur due to various reasons such as hardware failure, network issues, or software bugs. When a Kafka offline partition is detected, it can impact the availability and reliability of the entire system. It is important to quickly troubleshoot and resolve the issue to minimize the impact on the system.


This incident type refers to the detection of an unused topic partition in Kafka Platform, a popular data streaming platform. Unused partitions can increase broker overhead and reduce efficiency, so it is recommended to delete them. The incident is triggered by a test notification or a monitoring query that checks the value of relevant metrics. The incident is resolved once the partition is deleted or confirmed to be intentional.


Unused Topic Partition in Kafka

This incident type refers to a situation where there is an issue with the communication between Kafka brokers due to network partitioning. A network partition is a situation where a network is split into two or more parts, and communication between them is disrupted. This issue can lead to data inconsistency, loss of messages, delayed message delivery, and other related problems. As a result, it is important to quickly detect and resolve these incidents to prevent any downstream impact on the system.


Parameters

This type of incident occurs when the offset of a Kafka topic decreases unexpectedly. In Kafka, an offset is a unique identifier that represents the position of a message within a partition. When the offset decreases, it means that some messages have been lost or deleted, which can cause data inconsistencies or other issues within the system. This incident type is critical for Kafka-based systems as it affects data integrity and can lead to data loss.


Kafka topic offset decreased incident.

This incident type refers to when a Kafka node runs out of storage space to store data. This can cause issues with data processing and storage, potentially leading to data loss and system downtime.


Kafka node insufficient storage check

```shell
export BOOTSTRAP_SERVER="PLACEHOLDER"

export TOPIC_NAME="PLACEHOLDER"

export PARTITION_NUMBER="PLACEHOLDER"

export PATH_TO_KAFKA_LOGS="PLACEHOLDER"

export NUM_PARTITIONS="PLACEHOLDER"

export KAFKA_CLUSTER_URL="PLACEHOLDER"
```


### 1. Check if Kafka service is running

```shell
systemctl status kafka
```

### 2. List all topics in the cluster

```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --list
```

### 3. Describe the topic to see the number of partitions and their status

```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME}
```

### 4. Check the status of all partitions for a topic

```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME} | grep Partition
```

### 5. Check the status of a specific partition

```shell
kafka-log-dirs.sh --describe --bootstrap-server ${BOOTSTRAP_SERVER} --topic-list ${TOPIC_NAME} --partition-list ${PARTITION_NUMBER}
```

### 6. Check if the broker hosting the partition is offline

```shell
kafka-topics.sh --bootstrap-server ${BOOTSTRAP_SERVER} --describe --topic ${TOPIC_NAME} | grep ${PARTITION_NUMBER} | awk '{print $6}' | cut -d: -f1 | xargs -I {} sh -c '${PATH_TO_KAFKA_BIN}/kafka-broker-api-versions.sh --bootstrap-server {}:9092 --version'
```

### 7. Check the logs for any error messages related to the partition

```shell
tail -n 100 ${PATH_TO_KAFKA_LOGS}/server.log | grep ${TOPIC_NAME} | grep ${PARTITION_NUMBER}
```


### Rebalance the partitions: If the partition is offline due to an imbalance in the Kafka cluster, rebalancing the partitions can help bring the partition back online.

```shell
bash

#!/bin/bash

# Rebalance the partitions

kafka-topics --bootstrap-server ${BOOTSTRAP_SERVER}  --alter --topic ${TOPIC_NAME} --partitions ${NUM_PARTITIONS}


```


Kafka Offline Partition Detected.

Overview

Parameters

Debug

1. Check if Kafka service is running

2. List all topics in the cluster

3. Describe the topic to see the number of partitions and their status

4. Check the status of all partitions for a topic

5. Check the status of a specific partition

6. Check if the broker hosting the partition is offline

Repair

Rebalance the partitions: If the partition is offline due to an imbalance in the Kafka cluster, rebalancing the partitions can help bring the partition back online.

Learn more

Related Runbooks

Unused Topic Partition in Kafka

Network Partitioning Between Kafka Brokers.

Parameters

Kafka topic offset decreased incident.

Support