---
id: 78075240-0eb0-4a0d-b6b6-bec5f1580d55
---

# Host CPU high iowait incident.
---

A Host CPU high iowait incident occurs when the CPU iowait is greater than 10%, indicating that the system is disk or network bound. This can cause performance issues and may require investigation to identify the root cause and resolve the issue.

### Parameters
```shell
export INTERVAL="PLACEHOLDER"

export COUNT="PLACEHOLDER"

export DURATION="PLACEHOLDER"

export PORT="PLACEHOLDER"

export INTERFACE="PLACEHOLDER"

export DISK_NAME="PLACEHOLDER"

export DIRECTORY_PATH="PLACEHOLDER"

export DAYS_MODIFIED="PLACEHOLDER"
```

## Debug

### Check CPU usage by process
```shell
top
```

### Check I/O usage by process
```shell
iotop
```

### Check disk I/O statistics
```shell
iostat -x ${INTERVAL} ${COUNT}
```

### Check network I/O statistics
```shell
ifstat ${INTERVAL} ${COUNT}
```

### Check for high disk utilization by process
```shell
sudo pidstat -d ${INTERVAL} ${COUNT}
```

### Check for high network utilization by process
```shell
sudo tcpdump -i ${INTERFACE} -nn -s0 -v port ${PORT} -w tcpdump.pcap &

sudo timeout ${DURATION} sudo tcpdump -i ${INTERFACE} -nn -s0 -v port ${PORT}

sudo killall tcpdump
```

### Insufficient disk I/O throughput causing the CPU to wait for data to be read or written to the disk.
```shell


#!/bin/bash



# Define variables

IOWAIT_THRESHOLD=10

DISK_NAME=${DISK_NAME}



# Get current iowait percentage

IOWAIT=$(iostat -c | awk '/^ /{print $4}')



# Check if iowait is above threshold

if (( $(echo "$IOWAIT > $IOWAIT_THRESHOLD" | bc -l) )); then

    # Get current disk I/O throughput

    DISK_IO=$(iostat -x 1 2 | awk '/'$DISK_NAME'/{getline; print $6}')



    # Check if disk I/O is below threshold

    if (( $(echo "$DISK_IO < 100" | bc -l) )); then

        echo "Disk I/O throughput is insufficient. Current value: $DISK_IO"

    else

        echo "Iowait is high, but disk I/O throughput is not the issue."

    fi

else

    echo "Iowait is normal."

fi


```

## Repair

### Increase the available disk space by deleting unnecessary files or adding more storage capacity.
```shell
bash

#!/bin/bash



# Set variables

TARGET_DIR=${DIRECTORY_PATH}
DAYS_MODIFIED=${DAYS_MODIFIED}


# Delete unnecessary files

sudo find $TARGET_DIR -type f -name "*.log" -ctime $DAYS_MODIFIED -delete



# Check available disk space

df -h $TARGET_DIR


```

A Host CPU high iowait incident occurs when the CPU iowait is greater than 10%, indicating that the system is disk or network bound. This can cause performance issues and may require investigation to identify the root cause and resolve the issue.


This incident type refers to an increase in the number of errors per second on a Tomcat server, which could indicate an issue with the server itself, the host, a deployed application, or an application servlet. This could include errors generated when the Tomcat server runs out of memory, can't find a requested file or servlet, or is unable to serve a JSP due to syntax errors in the servlet codebase. This incident type requires immediate attention to diagnose and address the underlying issue.


Increase of the errors/second rate for Tomcat server

This incident type indicates that there is a high latency issue in the execution of a Spark job. Spark is a distributed computing framework that is used for processing large datasets. High latency in this context means that the time taken to execute the Spark job is significantly longer than expected or normal. This can result in delays in processing data and can impact the performance of the application or system that is utilizing Spark.


High Latency Incident for Spark Job Execution.

This incident type refers to a situation where the MySQL database is experiencing a high level of threads running, which could indicate a performance issue. This may cause issues with the application that relies on the database, such as slow response times or even downtime. Resolving this incident typically involves investigating the root cause of the high thread count and optimizing the database configuration or query performance to reduce the load on the database.


MySQL high threads running incident.

This incident type involves the detection of high CPU usage on a MongoDB host, often triggered by an alert monitor. The incident may be indicated by anomalous CPU usage values that are more than two standard deviations above predicted values during a specific time period, such as the last 15 minutes. The incident can cause performance issues and may require investigation to identify the root cause and resolve the problem.


High CPU usage on MongoDB host

This incident type refers to the failure of a systemd service on a particular host instance. The incident could be triggered by various causes such as a software bug, hardware failure, or system overload. This type of incident can cause downtime or service disruption to the affected host instance, which may require immediate resolution to restore normal operations.


Host systemd service crashed (instance) incident.

```shell
export INTERVAL="PLACEHOLDER"

export COUNT="PLACEHOLDER"

export DURATION="PLACEHOLDER"

export PORT="PLACEHOLDER"

export INTERFACE="PLACEHOLDER"

export DISK_NAME="PLACEHOLDER"

export DIRECTORY_PATH="PLACEHOLDER"

export DAYS_MODIFIED="PLACEHOLDER"
```


### Check CPU usage by process

```shell
top
```

### Check I/O usage by process

```shell
iotop
```

### Check disk I/O statistics

```shell
iostat -x ${INTERVAL} ${COUNT}
```

### Check network I/O statistics

```shell
ifstat ${INTERVAL} ${COUNT}
```

### Check for high disk utilization by process

```shell
sudo pidstat -d ${INTERVAL} ${COUNT}
```

### Check for high network utilization by process

```shell
sudo tcpdump -i ${INTERFACE} -nn -s0 -v port ${PORT} -w tcpdump.pcap &

sudo timeout ${DURATION} sudo tcpdump -i ${INTERFACE} -nn -s0 -v port ${PORT}

sudo killall tcpdump
```

### Insufficient disk I/O throughput causing the CPU to wait for data to be read or written to the disk.

```shell


#!/bin/bash



# Define variables

IOWAIT_THRESHOLD=10

DISK_NAME=${DISK_NAME}



# Get current iowait percentage

IOWAIT=$(iostat -c | awk '/^ /{print $4}')



# Check if iowait is above threshold

if (( $(echo "$IOWAIT > $IOWAIT_THRESHOLD" | bc -l) )); then

    # Get current disk I/O throughput

    DISK_IO=$(iostat -x 1 2 | awk '/'$DISK_NAME'/{getline; print $6}')



    # Check if disk I/O is below threshold

    if (( $(echo "$DISK_IO < 100" | bc -l) )); then

        echo "Disk I/O throughput is insufficient. Current value: $DISK_IO"

    else

        echo "Iowait is high, but disk I/O throughput is not the issue."

    fi

else

    echo "Iowait is normal."

fi


```


### Increase the available disk space by deleting unnecessary files or adding more storage capacity.

```shell
bash

#!/bin/bash



# Set variables

TARGET_DIR=${DIRECTORY_PATH}
DAYS_MODIFIED=${DAYS_MODIFIED}


# Delete unnecessary files

sudo find $TARGET_DIR -type f -name "*.log" -ctime $DAYS_MODIFIED -delete



# Check available disk space

df -h $TARGET_DIR


```


Host CPU high iowait incident.

Overview

Parameters

Debug

Check CPU usage by process

Check I/O usage by process

Check disk I/O statistics

Check network I/O statistics

Check for high disk utilization by process

Check for high network utilization by process

Insufficient disk I/O throughput causing the CPU to wait for data to be read or written to the disk.

Repair

Increase the available disk space by deleting unnecessary files or adding more storage capacity.

Learn more

Related Runbooks

Increase of the errors/second rate for Tomcat server

High Latency Incident for Spark Job Execution.

MySQL high threads running incident.

High CPU usage on MongoDB host

Support