---
id: 1bd49ea8-9b28-4941-861d-0b1fa94e7785
---

# Redis missing master incident
---

The Redis missing master incident occurs when the Redis cluster has no node marked as master. This can cause problems with the Redis service and may result in service disruptions or failures. It is important to address this issue promptly to ensure the smooth operation of the Redis service.

### Parameters
```shell
# Environment Variables

export REDIS_HOST="PLACEHOLDER"

export REDIS_PORT="PLACEHOLDER"

export IP_ADDRESS_OF_REDIS_MASTER_NODE="PLACEHOLDER"

export NODE_NAME="PLACEHOLDER"

export REDIS_CONFIG_FILE_PATH="PLACEHOLDER"

export MASTER_IP_ADDRESS="PLACEHOLDER"
```

## Debug

### Check Redis cluster status
```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster info
```

### Check cluster nodes
```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster nodes
```

### Check master node
```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster nodes | grep master
```

### Check Redis logs for errors
```shell
tail -n 100 /var/log/redis/redis-server.log
```

### Check Redis configuration file
```shell
cat /etc/redis/redis.conf
```

### Check Redis process status
```shell
systemctl status redis
```

### Check Redis service logs for errors
```shell
journalctl -u redis.service --no-pager | tail -n 100
```

### A node that was previously marked as the Redis master has failed or is no longer available.
```shell
bash

#!/bin/bash



# Set the IP address of the Redis cluster master node

REDIS_MASTER=${IP_ADDRESS_OF_REDIS_MASTER_NODE}



# Ping the Redis master node to check if it is reachable

ping -c 1 $REDIS_MASTER



# Check if the Redis master node is responding to Redis requests

redis-cli -h $REDIS_MASTER ping


```

## Repair

### If there are no Redis nodes marked as a master, promote a suitable node to become the master.
```shell


#!/bin/bash



# Set the Redis node that will be promoted to master

REDIS_NODE=${NODE_NAME}



# Promote the Redis node to master

redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster failover force


```

### If the master node is down, bring it back up and ensure it is correctly marked as the master.
```shell


#!/bin/bash



# Set the IP address of the Redis master node

REDIS_MASTER=${MASTER_IP_ADDRESS}



# Check if the Redis master node is responding

if ping -c 1 $REDIS_MASTER > /dev/null; then

    echo "Redis master node is up"

else

    # If the node is down, bring it back up

    echo "Redis master node is down, attempting to bring it back up..."

    sudo service redis-server start



    # Wait for Redis to start up

    sleep 10s



    # Check if Redis is running

    if systemctl is-active --quiet redis-server; then

        # If Redis is running, promote the node to become the new master

        redis-cli -h $REDIS_MASTER slaveof no one

        echo "Redis master node has been restarted and promoted"

    else

        # If Redis is not running, print an error message

        echo "Failed to start Redis master node"

    fi

fi


```


The Redis missing master incident occurs when the Redis cluster has no node marked as master. This can cause problems with the Redis service and may result in service disruptions or failures. It is important to address this issue promptly to ensure the smooth operation of the Redis service.


The Redis too many masters incident occurs when there are too many master nodes in a Redis cluster, leading to connection issues and potential data loss. This can happen due to misconfiguration, network issues, or other factors, and requires immediate attention to prevent further damage.


Redis too many masters incident

This incident type refers to an issue with Redis replication, which means that there is a problem with the synchronization of data between Redis instances. This issue could impact the availability and performance of the system and may require immediate attention to restore the replication and ensure data consistency. The incident could be caused by various factors, such as network problems, hardware failures, or configuration issues. The incident must be investigated and resolved as soon as possible to avoid any data loss or downtime.


Redis replication broken incident.

This incident type refers to a situation where Redis, a popular in-memory data structure store, has rejected connections. This could happen due to various reasons such as server overload, network issues, or misconfiguration. When this occurs, it can cause disruptions in the performance of applications that rely on Redis for data storage or caching. Thus, it requires immediate attention and resolution to minimize the impact on the applications and users.


Redis Rejected Connections Incident

The Redis missing backup incident refers to a situation where Redis, a popular in-memory data structure store, has not been backed up for a certain period of time. This could lead to data loss or corruption in the event of a system failure or other issues. The incident typically requires immediate attention to ensure that the data is properly backed up and that any potential issues are resolved quickly to avoid further complications.


Redis missing backup incident.

A Redis instance down incident refers to a situation where the Redis server, which is a popular open-source in-memory data structure store, is not functioning or accessible. This can cause interruption in the service and impact the performance of applications that rely on Redis for caching or data storage. Typically, this type of incident is considered high urgency and requires immediate attention from the responsible team to investigate, diagnose, and resolve the issue.


Redis instance down incident

```shell
# Environment Variables

export REDIS_HOST="PLACEHOLDER"

export REDIS_PORT="PLACEHOLDER"

export IP_ADDRESS_OF_REDIS_MASTER_NODE="PLACEHOLDER"

export NODE_NAME="PLACEHOLDER"

export REDIS_CONFIG_FILE_PATH="PLACEHOLDER"

export MASTER_IP_ADDRESS="PLACEHOLDER"
```


### Check Redis cluster status

```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster info
```

### Check cluster nodes

```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster nodes
```

### Check master node

```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster nodes | grep master
```

### Check Redis logs for errors

```shell
tail -n 100 /var/log/redis/redis-server.log
```

### Check Redis configuration file

```shell
cat /etc/redis/redis.conf
```

### Check Redis process status

```shell
systemctl status redis
```

### Check Redis service logs for errors

```shell
journalctl -u redis.service --no-pager | tail -n 100
```

### A node that was previously marked as the Redis master has failed or is no longer available.

```shell
bash

#!/bin/bash



# Set the IP address of the Redis cluster master node

REDIS_MASTER=${IP_ADDRESS_OF_REDIS_MASTER_NODE}



# Ping the Redis master node to check if it is reachable

ping -c 1 $REDIS_MASTER



# Check if the Redis master node is responding to Redis requests

redis-cli -h $REDIS_MASTER ping


```


### If there are no Redis nodes marked as a master, promote a suitable node to become the master.

```shell


#!/bin/bash



# Set the Redis node that will be promoted to master

REDIS_NODE=${NODE_NAME}



# Promote the Redis node to master

redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} cluster failover force


```

### If the master node is down, bring it back up and ensure it is correctly marked as the master.

```shell


#!/bin/bash



# Set the IP address of the Redis master node

REDIS_MASTER=${MASTER_IP_ADDRESS}



# Check if the Redis master node is responding

if ping -c 1 $REDIS_MASTER > /dev/null; then

    echo "Redis master node is up"

else

    # If the node is down, bring it back up

    echo "Redis master node is down, attempting to bring it back up..."

    sudo service redis-server start



    # Wait for Redis to start up

    sleep 10s



    # Check if Redis is running

    if systemctl is-active --quiet redis-server; then

        # If Redis is running, promote the node to become the new master

        redis-cli -h $REDIS_MASTER slaveof no one

        echo "Redis master node has been restarted and promoted"

    else

        # If Redis is not running, print an error message

        echo "Failed to start Redis master node"

    fi

fi


```


Redis missing master incident

Overview

Parameters

Debug

Check Redis cluster status

Check cluster nodes

Check master node

Check Redis logs for errors

Check Redis configuration file

Check Redis process status

Check Redis service logs for errors

A node that was previously marked as the Redis master has failed or is no longer available.

Repair

If there are no Redis nodes marked as a master, promote a suitable node to become the master.

If the master node is down, bring it back up and ensure it is correctly marked as the master.

Learn more

Related Runbooks

Redis too many masters incident

Redis replication broken incident.

Redis Rejected Connections Incident

Redis missing backup incident.

Support