---
id: 03b2e50a-936e-4e51-adf6-4ff53dce2f5b
---

# Kafka Version Mismatch Across Brokers.
---

This incident type refers to a situation where the different brokers in a Kafka cluster are running different versions of the software. This can lead to compatibility issues and potentially cause failures in the overall system. It is important to ensure that all brokers are running the same version of Kafka to avoid any version mismatch related incidents.

### Parameters
```shell


export BROKER_LIST="PLACEHOLDER"

export CONTROLLER_BROKER="PLACEHOLDER"

export ANY_TOPIC="PLACEHOLDER"

export TARGET_VERSION="PLACEHOLDER"

export BROKER_HOST="PLACEHOLDER"
```

## Debug

### Connect to the cluster and list all brokers
```shell
kafka-topics.sh --bootstrap-server ${BROKER_LIST} --list
```

### Check the version of Kafka running on each broker
```shell
kafka-broker-api-versions.sh --bootstrap-server ${BROKER_LIST}
```

### Compare the version of Kafka running on each broker
```shell
kafka-broker-api-versions.sh --bootstrap-server ${BROKER_LIST} | grep version
```

### Check if there are any incompatible API versions between brokers
```shell
kafka-broker-api-versions.sh --bootstrap-server ${BROKER_LIST} | grep -i incompatible
```

### Check the version of Kafka running on the controller
```shell
kafka-topics.sh --bootstrap-server ${CONTROLLER_BROKER} --describe --topic ${ANY_TOPIC}
```

### Check if the controller is up-to-date with the version of Kafka running on the brokers
```shell
kafka-broker-api-versions.sh --bootstrap-server ${CONTROLLER_BROKER} | grep -i version
```

## Repair

### Upgrade the broker(s) running the older version of Kafka to the target version.
```shell


#!/bin/bash



# Set variables

BROKER_HOST=${BROKER_HOST}

TARGET_VERSION=${TARGET_VERSION}



# Stop Kafka on the target broker

'/usr/local/kafka/bin/kafka-server-stop.sh'



# Upgrade Kafka on the target broker
wget https://archive.apache.org/dist/kafka/$TARGET_VERSION/kafka_2.11-$TARGET_VERSION.tgz

tar -xzf kafka_2.11-$TARGET_VERSION.tgz

rm kafka_2.11-$TARGET_VERSION.tgz

ln -sfn kafka_2.11-$TARGET_VERSION kafka


# Start Kafka on the target broker

/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties


```

This incident type refers to a situation where the different brokers in a Kafka cluster are running different versions of the software. This can lead to compatibility issues and potentially cause failures in the overall system. It is important to ensure that all brokers are running the same version of Kafka to avoid any version mismatch related incidents.


```shell


export BROKER_LIST="PLACEHOLDER"

export CONTROLLER_BROKER="PLACEHOLDER"

export ANY_TOPIC="PLACEHOLDER"

export TARGET_VERSION="PLACEHOLDER"

export BROKER_HOST="PLACEHOLDER"
```


### Connect to the cluster and list all brokers

```shell
kafka-topics.sh --bootstrap-server ${BROKER_LIST} --list
```

### Check the version of Kafka running on each broker

```shell
kafka-broker-api-versions.sh --bootstrap-server ${BROKER_LIST}
```

### Compare the version of Kafka running on each broker

```shell
kafka-broker-api-versions.sh --bootstrap-server ${BROKER_LIST} | grep version
```

### Check if there are any incompatible API versions between brokers

```shell
kafka-broker-api-versions.sh --bootstrap-server ${BROKER_LIST} | grep -i incompatible
```

### Check the version of Kafka running on the controller

```shell
kafka-topics.sh --bootstrap-server ${CONTROLLER_BROKER} --describe --topic ${ANY_TOPIC}
```

### Check if the controller is up-to-date with the version of Kafka running on the brokers

```shell
kafka-broker-api-versions.sh --bootstrap-server ${CONTROLLER_BROKER} | grep -i version
```


### Upgrade the broker(s) running the older version of Kafka to the target version.

```shell


#!/bin/bash



# Set variables

BROKER_HOST=${BROKER_HOST}

TARGET_VERSION=${TARGET_VERSION}



# Stop Kafka on the target broker

'/usr/local/kafka/bin/kafka-server-stop.sh'



# Upgrade Kafka on the target broker
wget https://archive.apache.org/dist/kafka/$TARGET_VERSION/kafka_2.11-$TARGET_VERSION.tgz

tar -xzf kafka_2.11-$TARGET_VERSION.tgz

rm kafka_2.11-$TARGET_VERSION.tgz

ln -sfn kafka_2.11-$TARGET_VERSION kafka


# Start Kafka on the target broker

/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties


```


Kafka Version Mismatch Across Brokers.

Overview

Parameters

Debug

Connect to the cluster and list all brokers

Check the version of Kafka running on each broker

Compare the version of Kafka running on each broker

Check if there are any incompatible API versions between brokers

Check the version of Kafka running on the controller

Check if the controller is up-to-date with the version of Kafka running on the brokers

Repair

Upgrade the broker(s) running the older version of Kafka to the target version.

Learn more

Related Runbooks

Support