Important but Ignored performance tips for Tomcat.

Below are the important tip but yet mostly ignored by developers.

• Logging
• Connectors
• JVM
• Troubleshooting

Logging:
Some default configurations are generic in tomcat and can prove not ideal for production

1. Duplication of logs
2. Synchronized Logging

Duplication of Logs

For example
catch-all logger log to file and stdout

Solution
Remove duplicate logging in tomcat by directing all log into one location

Change this
.handlers = 1catalina.org.apache.juli.FileHandler,java.util.logging.ConsoleHandler

To

.handler = 1catalina.org.apache.juli.FileHandler

Synchronized Logging
Synchronous logging can prove to be a major bottleneck for some applications and Disk I/O factor can prove to be a limiting factor

Solution

Asynchronous Logging
Use Log queue with limited queue size to avoid out of memory errors
Fall back to synchronized logging instead of breaking down the application

Connectors
Developer should understand the application design and its connection usage patterns for example TCP Connections, HTTP transactions, HTTP Keep-Alive, SSl etc

Special consideration should be given to connection pooling

Which Connector you should use ?

I think we should put more stress in java NIO Connectors since it is pure java and also easy to switch between NIO and BIO with SSL

After choosing the respective connector special consideration must be put on number of threads the respective connector will use

Typical values is around 200 - 800 but in case of high CPU usage developer can decrease the number of threads and vice versa

For HTTP Connections typical value for maxKeepAliveRequest is around 1000 i.e maximum number of HTTP requests per TCP connection
Set to 1 disable keep alive

Default value to connection timeout is 20000 which is considered too high for most of the production environment as a result can be set around 3000

JVM

Xms/-Xmx
The above mentioned options are used to define size of java heap try to set as low as possible because setting too high wastes memory and can cause long garbage collection pauses

-XX:MYSize -XX:MyRatio

Set to 25% to 33% of total java heap because setting too or too low can lead to inefficient garbage collection

Garbage Collection pauses application no matter what GC algorithm is and the respective pause can impact the response time of application. By setting the GC pause time goals more frequent will will result in to shorter pauses.

Troubleshooting

Collecting and analyzing log data
Common problems
* Broken pipe - For HTTP Connector indicates that the remote client aborted the request. For web server JK Connector indicates that the web server process or thread was terminated. These are normal and rarely due to a problem with Tomcat. However, if you have long request, the connectionTimeout may close the connection before you send your response back.

• Tomcat freezes or pauses with no request being processed - usually due to a long pause of JVM GC. A long pause can cause a cascading effect and high load once Tomcat starts handling requests again. Don’t set the “acceptCount” too high and use java -verbose:gc startup argument to collect GC data.
• Out of Memory Exception - look into application code to fix the leak (profile tool can help). Increase available memory on the system via -Xmx. Restart tomcat!

• Database connection failure - connection used up when traffic is spike.
• Random connection close exception - when you close your connection twice. first close(), the connection returns to the pool. It may be picked up by another thread. Now, second close() may close a connection that is being used by other thread. Don’t close connection twice, use JDBC Template from Spring to avoid this problem.

Ref: JavaWorld GC Article