Hacking Tomcat

Secrets Revealed

Talk Sponsored By

Actual Sponsor

Who am I bla bla

fhanik@apache.org Tomcat Committer / ASF member Co-designed the Comet implementation Implemented NIO connector in 6 Responsible for session replication and clustering Been involved with ASF since 2001

Who are We
Top Level Project tomcat.apache.org 24 committers on file Active/core group is much smaller Find us on dev@tomcat.apache.org Work on what interests you and work without guidance Our people skills are improving ;)

Welcome to Tomcat

What we will cover

History of Tomcat The Basics
Configuration and Container Architecture

The Advanced
Swappable Components Tomcat Connectors Servlet Container JSP Compiler Developing/Debugging Tomcat

What We Will Not Cover

Too Basic stuff This is a technical presentation Configuration details How the actual JSP .jsp to .java compiler works Forked Tomcat code bases, how they differ and why they happened Older versions of Tomcat, we will work with Tomcat 6, no looking back

History of Tomcat
Started out as a reference implemenation by Sun Microsystem Donated to ASF Tomcat 3 (roughly 1999) Tomcat 4 New specs & First rewrite Codename Catalina Tomcat 5.0 New specs Tomcat 5.5 2nd Rewrite Performance Tomcat 6.0 New specs, New Cool Features

Basics
server.xml
Main configuration file Builds server on the fly Parsed using commons-digester Tomcat has hard coded rule sets for the parsing Every element/component is swappable

<ElementName className=the implementation attribute=call setAttribute/>

Basics

Example: <Server className=o.a.c.core.StandardServer port="8005 shutdown="SHUTDOWN">

Basics
Entire server.xml parsed based on rules Look for these rules:
Catalina.java org/apache/catalina/startup/

Even web.xml is parsed using the digester

Basics Catalina.java-createStartDigester
Digester digester = new Digester(); digester.setValidating(false); digester.setClassLoader( StandardServer.class.getClassLoader());

digester.addObjectCreate("Server", "org.apache.catalina.core.StandardServer, "className");

digester.addSetProperties("Server");

Basics
The exception <Connector> <Connector className=ignored protocol=nested object
ConnectorCreateRule.java - begin digester.push( new Connector( attributes.getValue("protocol"))); protocol -> nested className

Tomcat The Server

Services

Basics
Engine (Catalina)
Hosts
Realm

AJP Connector 8009 SSL Connector 8443 HTTP Connector 8080

Valves

Context
Valves

JSPs

Servlets

Valves

Basics
Service/Engine/Host/Context
All are Containers All implement LifecycleListeners

LifecycleEvents
How objects get initialized, started and stopped

Object relationships are established during creation(digester parsing)

Basics
Last Basics I promise conf/web.xml
Default web.xml Merged with webapps WEB-INF/web.xml DefaultServlet static content JSP Servlet JSP files

conf/context.xml
Merged with apps <Context> definition

Advanced
Connectors the entry point Servlet Engine and Container Design Jasper The JSP engine Valves interceptor pattern Developing and Debugging How to join if you are interested

Performance Tip
Tomcat produces very little GC
Most objects are pooled Even though makers of VM say, never pool objects Prevents CPU/GC jigsaw pattern Resetting fields is faster than GC old object, create new object and initialize No GC lock up surprises

Connectors
HTTP Connector protocol=
o.a.coyote.http11.Http11Protocol o.a.coyote.http11.Http11AprProtocol o.a.coyote.http11.Http11NioProtocol HTTP/1.1 aliased to Http11 and Http11Apr

AJP Connector
org.apache.jk.server.JkCoyoteHandler org.apache.coyote.ajp.AjpAprProtocol AJP/1.3 aliased to the two above

Connectors
Connector Protocol
<Handler>

Processor Adapter Engine

EndPoint

There are some pretty ugly interdependencies here. While re-factoring would resolve that, time has been spent improving performance.

Connectors
Request Process
Protocol Connector

Processor Adapter

EndPoint Engine

Once the request Processor is1. parsed New sets HTTP up input/output Request buffers The CoyoteAdapter HTTP Parsing (bridge between logic is logic in connector here and engine) All java.io/java.nio/apr socket is in the EndPoint Passes the request Parses to request, the servlet if request engineis available

Performance Tip
MessageBytes.java All HTTP Parsing doesnt deal with strings Every chunk of data gets parsed into a MessageBytes object This object represents a string in the HTTP header Avoid string comparison routines Doesnt contain copy of byte[], but a pointer to the original data with an offset and length

Performance Tip
Use Http11Protocol
Keep Alive is turned off Kernel accept filter is in place

Use Http11AprProtocol
Take advantage of SEND_FILE Native SSL handling Comet Support

Use Http11NioProtocol
Take advantage of SEND_FILE Large number of sockets using Keep Alive APR is not available or JNI is not preferred Comet Support

Advanced
CoyoteAdapter.java
Creates Request/Response objects Maps Request/Response to
A Host object (StandardHost) A Context object (StandardContext) A Servlet (StandardWrapper)

Parses Session Cookie

URL Cookie

Grabs Engines valve and passes the request into the servlet engine

Performance Tip
DefaultServlet.java
Handles all static content Gets deployed into every webapp through conf/web.xml If SEND_FILE support is enabled it will use it Its a REGULAR SERVLET!! Files and their attributes are cached You can replace with your own implementation

Advanced
Servlet Invokation Chain
o.a.c.servlets.DefaultServlet.doGet javax.servlet.http.HttpServlet.service o.a.c.core.ApplicationFilterChain.doFilter o.a.c.core.StandardWrapperValve.invoke o.a.c.core.StandardContextValve.invoke o.a.c.core.StandardHostValve.invoke o.a.c.valves.ErrorReportValve.invoke o.a.c.core.StandardEngineValve.invoke o.a.c.connector.CoyoteAdapter.service o.a.coyote.http11.Http11NioProcessor.process o.a.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process o.a.tomcat.util.net.NioEndpoint$SocketProcessor.run java.util.concurrent.ThreadPoolExecutor$Worker.runTask java.util.concurrent.ThreadPoolExecutor$Worker.run 8. StandardHostValve 11. 3. 6. 7. 9. SocketProcessor StandardEngineValve ErrorReportValve StandardContextValve ApplicationFilterChain the thread 1. 10. 12. 4. 2.Everything 5. HttpProcessor NIO CoyoteAdapter The StandardWrapperValve Connector Servlet starts defaults with java.lang.Thread.run

Sets context class loader Represents Simple First Catches Invokes valve Runnable (spec) Throwable in Servlet theservlet engine to (spec) invoke Execution Invokes parses to ThreadPoolExecutor Creates (spec) HTTP of Request the request FilterChain, Handler.process container Reports ServletRequestListeners 400+ invokes errorsservlet FilterChain Response pair

Reading Data From the InputStream

Advanced

o.a.c.http11.InternalNioInputBuffer$SocketInputBuffer.doRead o.a.c.http11.filters.ChunkedInputFilter.readBytes o.a.c.http11.http11.filters.ChunkedInputFilter.parseChunkHeader o.a.c.http11.http11.filters.ChunkedInputFilter.doRead o.a.c.http11.http11.InternalNioInputBuffer.doRead o.a.c.http11.Request.doRead o.a.catalina.connector.InputBuffer.realReadBytes o.a.t.u.buf.ByteChunk.substract o.a.catalina.connector.InputBuffer.read o.a.catalina.connector.CoyoteInputStream.read comet.CometEchoServlet.echo

Performance Tip
JspServlet.java
Handles all JSP files Gets deployed into every webapp through conf/web.xml Mapping done through URL patterns (per spec) Its a REGULAR SERVLET!! Connects into the Jasper engine

Advanced
How are JSP files handled
Through the JspServlet (same invocation path) JspServletWrapper created
Contains JspCompilationContext Holds a JSP class loader Invokes compile on the compilation context Processes annotations Loads .class file through class loader Finally invokes service(req,resp) on the generated servlet

o.a.jasper.compiler.Compiler
Generates .java file from .jsp

o.a.jasper.compiler.JDTCompiler
Generates .class file from .java

Advanced
Deployment of Applications <Context> - StandardContext <Listener> - LifecycleListener <Loader> - WebappLoader <Manager> - StandardManager <Realm> - No default <Resources> - FileDirContext <Valve> - No default </Context> ContextRuleSet.java parses contexts in server.xml

Advanced
Deployment of applications
The deployer is HostConfig.java Each StandardHost.java object holds a reference to a HostConfig Deploy order
context.xml files WAR files Directories

/ROOT is hardcoded for path= Runtime deployment triggered by LifecycleEvent

Advanced
Developing/Debugging
SVN Repo for TC 6 is simplified trunk/java all you need svn co/ant download/ant builds the system Run inside a debugger by emulating the catalina.sh/catalina.bat if you wish Everything is java, breakpoints anywhere

Performance Tip
Does it scale
Yes, its been tested with 16k concurrent and active connections on a Xmx512m system Performance increases well as new CPUs are added RAM is your max # concurrent connection limitation Simple tests run at http://blog.covalent.net/roller/covalent/entry/20070308

Performance Tip
Tuning
Mostly in the application itself, Tomcat default is pretty good When it comes down to nuts and bolts, the tuning is in the connectors NIO connector, by far the most tuning options (see docs) Socket and App buffers usually the most important aspect for write-speed APR connector, speedy little devil, not as many options, but relies on APR below being well tuned.

Performance Tip
Tuning the servlet engine
Sure, it can be done, but not through configuration Most common bottlenecks turn out to be synchronized statements or locks Compared to the webapp or the connector, spending time tuning the engine is not worth the time

Conclusion
Not so difficult on first impression Slightly confusing on second impression Once you get a hang of it, go crazy Modular design, and nasty deps Not the typical text book java design Find something interesting? want to contribute? take initiative, dont be shy

Want to join?
Ideas for needed projects
Better deployer Documentation Administration tool Better JMX support Remote and Cluster deployments Live status dash board SIP support The list goes on, take your pick!

Q&A
Lots and Lots covered
Only a drop in the sea, but enough to get you started not enough time

fhanik@apache.org anytime dev@tomcat.apache.org be brave http://people.apache.org/~fhanik for the presentation