Runbook

Increase of the errors/second rate for Tomcat server

Back to Runbooks

Overview

This incident type refers to an increase in the number of errors per second on a Tomcat server, which could indicate an issue with the server itself, the host, a deployed application, or an application servlet. This could include errors generated when the Tomcat server runs out of memory, can't find a requested file or servlet, or is unable to serve a JSP due to syntax errors in the servlet codebase. This incident type requires immediate attention to diagnose and address the underlying issue.

Parameters

Debug

Check the status of the Tomcat pod

Check the logs of the Tomcat container for any errors

Check the CPU and memory usage of the Tomcat container

Check the network traffic of the Tomcat container

Check the health of the Kubernetes cluster nodes where the Tomcat pod is running

Check the status of the Kubernetes services and endpoints

Check the status of the Kubernetes ingress controller

The host machine running the Tomcat server is experiencing high CPU or memory usage, causing the server to become unresponsive and generate errors.

A software update or configuration change was made to the Tomcat server that caused it to malfunction and produce errors.

Diagnostics

  • A software update or configuration change was made to the Tomcat server that caused it to malfunction and produce errors.
  • The host machine running the Tomcat server is experiencing high CPU or memory usage, causing the server to become unresponsive and generate errors.
  • A deployed application or application servlet on the Tomcat server is producing errors due to bugs or conflicts with other software components.
  • The Tomcat server is under heavy load or is being attacked by malicious traffic, causing it to produce errors and become unresponsive.
  • The Tomcat server is not properly configured or optimized for the workload it is handling, leading to performance issues and errors.

Repair

Check the deployed application to ensure that it is functioning correctly and that there are no coding errors or configuration issues.

Remediations

  • Check the server logs to determine the cause of the errors and identify any patterns or trends that may be contributing to the increase in errors.
  • Verify that there is no issue with the host server, such as insufficient resources or network connectivity problems.
  • Check the deployed application to ensure that it is functioning correctly and that there are no coding errors or configuration issues.
  • If necessary, restart the Tomcat server to clear any cached data or memory leaks that may be causing the errors.
  • Ensure that the latest version of Tomcat and any deployed applications are being used, as older versions may have known bugs or security vulnerabilities that could be contributing to the errors.
  • Monitor the server and deployed applications going forward to detect any recurring issues and to prevent future incidents.

Learn more

Related Runbooks

Check out these related runbooks to help you debug and resolve similar issues.