---
id: c8d27a4e-a9ac-4fe5-aad1-2c005397a582
---

# Tomcat Connection Pool Exhaustion Incident
---

Tomcat Connection Pool Exhaustion Incident occurs when the maximum number of connections that a Tomcat server can handle has been reached. This can happen due to an increase in traffic or a sudden surge in user activity. When the connection pool is exhausted, new requests cannot be handled, and the application may become unavailable. This type of incident requires immediate attention to ensure that the application is restored to its normal functioning state.

### Parameters
```shell
export VERSION="PLACEHOLDER"

export WEBAPP_NAME="PLACEHOLDER"

export USERNAME="PLACEHOLDER"

export PASSWORD="PLACEHOLDER"

export NEW_CONNECTION_POOL_SIZE="PLACEHOLDER"

export PATH_TO_TOMCAT="PLACEHOLDER"

export PATH_TO_TOMCAT_CONFIG_FILE="PLACEHOLDER"
```

## Debug

### Check if Tomcat server is running
```shell
systemctl status tomcat ${VERSION}
```

### Check if there are any errors in the Tomcat logs
```shell
tail -n 100 /var/log/tomcat${VERSION}/catalina.out
```

### Check the number of active database connections
```shell
sudo -u postgres psql -c "SELECT datname,numbackends FROM pg_stat_database;"
```

### Check the maximum number of connections allowed in the connection pool configuration
```shell
grep "maxActive" /opt/tomcat${VERSION}/conf/context.xml
```

### Check the current number of open connections in the connection pool
```shell
curl -u ${USERNAME}:${PASSWORD} http://localhost:8080/manager/jmxproxy/?get=Tomcat%3Atype%3DDatasource%2Ccontext%3D%2F${WEBAPP_NAME}%2Chost%3Dlocalhost&att=Active
```

### Check the available memory on the server
```shell
free -m
```

## Repair

### Increase the connection pool size to accommodate more incoming connections.
```shell


#!/bin/bash



# Set the variables

TOMCAT_HOME=${PATH_TO_TOMCAT}

CONFIG_FILE=${PATH_TO_TOMCAT_CONFIG_FILE}

POOL_SIZE=${NEW_CONNECTION_POOL_SIZE}



# Stop the Tomcat server

$TOMCAT_HOME/bin/shutdown.sh



# Update the connection pool size in the Tomcat config file

sed -i "s/maxActive=\"[0-9]*\"/maxActive=\"$POOL_SIZE\"/g" $CONFIG_FILE



# Start the Tomcat server

$TOMCAT_HOME/bin/startup.sh


```

Tomcat Connection Pool Exhaustion Incident occurs when the maximum number of connections that a Tomcat server can handle has been reached. This can happen due to an increase in traffic or a sudden surge in user activity. When the connection pool is exhausted, new requests cannot be handled, and the application may become unavailable. This type of incident requires immediate attention to ensure that the application is restored to its normal functioning state.


When too many requests are sent to a Tomcat server, the thread pools that handle these requests can run out of available threads. This means that incoming requests will not be processed, leading to service degradation or even a complete outage. This incident type is critical for web applications that rely on Tomcat as their application server.


Tomcat thread pool exhaustion.

Tomcat Server Crash incident type refers to a situation where the Tomcat Server, an open-source web server and servlet container, becomes unresponsive and stops functioning. This can result in a complete or partial system outage, leading to a disruption in the delivery of web applications and services. The root cause of the crash can be attributed to a variety of factors such as misconfiguration, network or hardware issues, software bugs, or excessive server load. It is critical to address this incident quickly to minimize the impact on users and restore normal system functionality.


Tomcat Server Crash

This incident type occurs when the PermGen or Metaspace usage of the Tomcat server exceeds 90%. PermGen and Metaspace are memory pools used for storing class definitions and metadata of the Java virtual machine (JVM). When these memory pools reach their maximum capacity, the server may experience performance issues or even crash. This incident requires immediate attention from the software engineer to investigate the root cause and take appropriate actions to resolve the issue.


Tomcat PermGen or Metaspace Usage exceeds 90%

This incident type is related to the Tomcat server, which is a popular open-source Java application server. It occurs when the available threads in the connection pool are running low, which means that there are not enough resources to serve incoming requests. This can cause delays or even failures in the application's performance, impacting the end-user experience. It is important to monitor and address this issue promptly to ensure that the application runs smoothly and efficiently.


Tomcat Low Available Threads in Connection Pool

In software engineering, a large number of suspended threads in Tomcat is an incident type that occurs when a high number of threads are created but are blocked from executing. This can happen due to a variety of reasons, such as slow database queries, network latency, or resource contention. When this occurs, the system may become unresponsive or slow, causing a degradation in performance or even a complete system failure. It is important to investigate and resolve this issue as quickly as possible to ensure the system can continue to function properly.


Tomcat Large Number of Suspended Threads Incident.

```shell
export VERSION="PLACEHOLDER"

export WEBAPP_NAME="PLACEHOLDER"

export USERNAME="PLACEHOLDER"

export PASSWORD="PLACEHOLDER"

export NEW_CONNECTION_POOL_SIZE="PLACEHOLDER"

export PATH_TO_TOMCAT="PLACEHOLDER"

export PATH_TO_TOMCAT_CONFIG_FILE="PLACEHOLDER"
```


### Check if Tomcat server is running

```shell
systemctl status tomcat ${VERSION}
```

### Check if there are any errors in the Tomcat logs

```shell
tail -n 100 /var/log/tomcat${VERSION}/catalina.out
```

### Check the number of active database connections

```shell
sudo -u postgres psql -c "SELECT datname,numbackends FROM pg_stat_database;"
```

### Check the maximum number of connections allowed in the connection pool configuration

```shell
grep "maxActive" /opt/tomcat${VERSION}/conf/context.xml
```

### Check the current number of open connections in the connection pool

```shell
curl -u ${USERNAME}:${PASSWORD} http://localhost:8080/manager/jmxproxy/?get=Tomcat%3Atype%3DDatasource%2Ccontext%3D%2F${WEBAPP_NAME}%2Chost%3Dlocalhost&att=Active
```

### Check the available memory on the server

```shell
free -m
```


### Increase the connection pool size to accommodate more incoming connections.

```shell


#!/bin/bash



# Set the variables

TOMCAT_HOME=${PATH_TO_TOMCAT}

CONFIG_FILE=${PATH_TO_TOMCAT_CONFIG_FILE}

POOL_SIZE=${NEW_CONNECTION_POOL_SIZE}



# Stop the Tomcat server

$TOMCAT_HOME/bin/shutdown.sh



# Update the connection pool size in the Tomcat config file

sed -i "s/maxActive=\"[0-9]*\"/maxActive=\"$POOL_SIZE\"/g" $CONFIG_FILE



# Start the Tomcat server

$TOMCAT_HOME/bin/startup.sh


```


Tomcat Connection Pool Exhaustion Incident

Overview

Parameters

Debug

Check if Tomcat server is running

Check if there are any errors in the Tomcat logs

Check the number of active database connections

Check the maximum number of connections allowed in the connection pool configuration

Check the current number of open connections in the connection pool

Check the available memory on the server

Repair

Increase the connection pool size to accommodate more incoming connections.

Learn more

Related Runbooks

Tomcat thread pool exhaustion.

Tomcat Server Crash

Tomcat PermGen or Metaspace Usage exceeds 90%

Tomcat Low Available Threads in Connection Pool

Support