Tomcat in Production stops being about new features the moment real users hit it. It becomes about threads, queues, and the connector you pick. Right now everyone is juggling Apache httpd in front, mod jk or mod proxy, a mix of static files, REST calls, and session heavy apps. Oracle now owns Sun, so we are all watching where Java heads, but our servers still need to carry traffic today. This is a practitioner’s look at Tomcat connectors and threads that holds up under live load, not just on a laptop.
Definitions
Tomcat speaks to the network through connectors. Each connector has its own threading model and tuning knobs.
- HTTP BIO: The classic blocking connector. One request ties up one thread until it finishes. Simple and predictable. Good default for many apps.
- HTTP NIO: Non blocking I O introduced in the Tomcat 6 line. Fewer selector threads can watch many sockets, while worker threads handle actual request work. Helpful when you have many keep alive sockets or some slow clients.
- APR Native: Uses the Apache Portable Runtime with the Tomcat Native library. You get OpenSSL, sendfile, and a different socket path that can drop CPU use on busy hosts. Needs the native lib installed on the box.
- AJP: The binary protocol that pairs Tomcat with Apache httpd through mod jk or mod proxy ajp. Small headers, sticky sessions by habit, and a long history in front of Apache setups.
Key threading and queue knobs:
- maxThreads: Upper bound of concurrent request processing for a connector or for an executor. If all threads are busy, new requests wait in the accept queue.
- minSpareThreads: Threads kept ready to avoid ramp up latency during bursts.
- acceptCount: Backlog of sockets waiting for a worker thread. Past this, the connector will refuse the connection. This is your back pressure valve.
- connectionTimeout: Milliseconds to wait for the next request on a keep alive socket or for the request body. Too high wastes threads. Too low hurts slow clients.
- maxKeepAliveRequests: Caps how many requests a client can send on one connection. Can prevent one client from camping forever.
- executor: A shared thread pool that several connectors can use. Keeps thread counts under control if you run both HTTP and AJP.
Examples
Static heavy site behind Apache httpd: Apache serves images and css. Tomcat handles JSP and controllers. AJP with mod jk is a solid pick. Set maxThreads near the expected peak of concurrent requests to the app, for example 200 on a mid range box, acceptCount at 100 to add a little cushion, and connectionTimeout around 10 seconds. Keep enableLookups off to avoid reverse DNS. You get clear separation and easier log parsing.
API style service with many short calls: Go with HTTP NIO. Many idle keep alive sockets do not cost you one thread each, and selector threads handle that traffic well. Set maxThreads to a number the CPU can handle, often two to four times the core count when work is I O bound. Keep acceptCount small to surface pressure early. Tighten connectionTimeout to avoid stranded sockets.
Busy SSL on a single host: If you can install the native library, APR with OpenSSL can drop CPU use and improve handshake speed. Match maxThreads to the cores and memory headroom you have. If native is not available on your distro, stay with HTTP NIO and be careful with thread counts.
Small EC2 node with bursty load: Keep it simple with HTTP BIO or NIO, fronted by ELB or by Apache only if you must. Start with maxThreads 150, minSpareThreads 25, acceptCount 150, then measure. Save memory for the heap. Thread counts are memory too.
Counterexamples
Turning maxThreads to 1000 and calling it a day: Threads are not free. Each has a stack and lives inside your process. You will spend more time on context switches and less on real work. GC pauses grow with more live objects across all those requests. Start lower, load test, then increase only if you see queueing and spare CPU.
Disabling keep alive to save threads: That forces clients to reconnect on every request. TLS handshakes and three way handshakes multiply, and your latency goes up. Keep alive is your friend. If threads are the concern, move to NIO or use an executor so several connectors share one pool.
Assuming AJP is always faster: AJP shines in some stacks, but modern mod proxy http is fine too. What matters is thread and queue sizing, keep alive between proxy and Tomcat, and not copying bytes more than needed. Measure both on your traffic mix.
Decision rubric
- Do you have Apache httpd in front? If yes, pick AJP with mod jk or HTTP with mod proxy http. If not, pick HTTP NIO unless the simplest path is BIO for your team.
- How chatty are clients? Many short calls and long lived keep alive favors NIO. Few longer requests can be fine on BIO.
- Need OpenSSL or sendfile? If you can install native libs, consider APR. If not, stay with NIO.
- Pick a starting thread count: If work is I O bound, set maxThreads to two to four times core count. If CPU bound, start near core count and grow slowly. Keep minSpareThreads around ten percent of max.
- Size the queue: Set acceptCount roughly equal to maxThreads. If you see drops and spare CPU, raise it. If latency spikes before the queue fills, you are past the safe region.
- Be strict on timeouts: Keep connectionTimeout near 5 to 15 seconds for most web apps. For uploads, raise the read timeout only on the connector that handles uploads.
- Measure and repeat: Use access logs and JMX. Watch busy threads, request backlog, and response times under load. Change one knob at a time.
Lesson learned
The best production story with Tomcat comes from clear choices. Pick the connector that matches your traffic, set maxThreads to what your CPU and memory can hold, give yourself a sane acceptCount for back pressure, and keep timeouts honest. A small, well understood setup will beat a clever one that nobody can explain at three in the morning. Load test with real patterns, not hello world, and let the numbers decide.