March 11th, 2010

I’ve had a bit of bother trying to configure our Tomcat server to use the APR libraries today.  I’ve managed to get it working eventually but thought it worth posting something about it as the documentation is a little scarce.  First off I’ll document what I should have done from the start and then I’ll list the errors I encountered figuring all this out.

How do I know if Tomcat is using APR or not?

If Tomcat is not using APR then you’ll see the following log line (or something similar) when you start it.

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/tomcat6live/java.home/jre/lib/i386/server:/home/tomcat6live/java.home/jre/lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib

If it is using APR then you’ll see something like

INFO: Loaded APR based Apache Tomcat Native library 1.1.19.

If you don’t see either or these lines then either you’ve configured logging above INFO on org.apache.catalina.core.AprLifecycleListener or you haven’t included the APR listener in your server.xml (see below).

Prerequisites

You need the APR and openSSL libraries installed on your server. We’d already installed them on our server so I’m not going to blog about how to install them here.

Install the Tomcat APR native libraries

As root:

cd /usr/local/apache-tomcat-6.0.24/bin
tar xfz tomcat-native.tar.gz
cd tomcat-native-1.1.19-src/jni/native
./configure --with-apr=/usr/local/apr --with-ssl=/usr/local --prefix=/usr/local/apache-tomcat-6.0.24  --with-java-home=/usr/java/jdk1.6.0_17
make
make install

Where you should adjust the ./configure parameters so that they match you APR, SSL, Tomcat and Java installations on your server.

Configure Tomcat to use the APR native libraries

The LD_LIBRARY path variable needs to include the Tomcat lib directory.  I have a setenv file that is referenced by our init.d start-up script so I simply added the following lines to that.

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CATALINA_HOME/lib
export LD_LIBRARY_PATH

Your start-up script needs to do something similar.

Now all you need to do is ensure the APR listener is configured in server.xml.  We have the following (which you’ll need to adjust if you’re using SSL).

<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="off" />

All done!

Problems getting Tomcat to use the APR native libraries

Like I mentioned earlier I encounted a number of issues along the way.

Error initializing endpoint / Invalid Server SSL Protocol

Tomcat started but with lots of errors. This being the first:

SEVERE: Error initializing endpoint
java.lang.Exception: Invalid Server SSL Protocol
        at org.apache.tomcat.jni.SSLContext.make(Native Method)
        at org.apache.tomcat.util.net.AprEndpoint.init(AprEndpoint.java:716)
        at org.apache.coyote.http11.Http11AprProtocol.init(Http11AprProtocol.java:107)
        at org.apache.catalina.connector.Connector.initialize(Connector.java:1007)
        at org.apache.catalina.core.StandardService.initialize(StandardService.java:677)
        at org.apache.catalina.core.StandardServer.initialize(StandardServer.java:795)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:540)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:261)
        at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:276)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)

This was because I had SSLEngine="off" in my APR listener configuration

<Listener className="org.apache.catalina.core.AprLifecycleListener"  SSLEngine="off" />

But had a http connector configured to use SSL

<Connector protocol="HTTP/1.1" ... SSLEnabled="true" ... />

Fix seemed simple – switch the ssl engine on!

<Listener className="org.apache.catalina.core.AprLifecycleListener"   SSLEngine="offon" />

symbol lookup error / undefined symbol: SSL_CTX_set_info_callback

Now Tomcat wouldn’t start at all! This was the error:

jsvc.exec: symbol lookup error: /usr/local/apache-tomcat-6.0.24/lib/libtcnative-1.so.0.1.19: undefined symbol: SSL_CTX_set_info_callback

This was due to the fact I hadn’t used the correct value for the --with-ssl option when I ran ./configure (see above).  I’d used /usr/share/ssl when I should have used /usr/local.  To fix I reran ./configure with the correct values.  You can check the output to make sure it finds a compatible version of ssl.  You then need to remake the libraries.

make clean
make
make install

Error initializing endpoint / No Certificate file specified or invalid file format

Now Tomcat starts but with a number of SEVERE errors.  In particular:

SEVERE: Error initializing endpoint
java.lang.Exception: No Certificate file specified or invalid file format
        at org.apache.tomcat.jni.SSLContext.setCertificate(Native Method)

This was caused by a misconfiguration of our http connector that was using SSL.  It turns out that: The APR connector uses different attributes for SSL keys and certificates. How annoying!  We only need SSL on our local development environments as Apache looks after the SSL handshake on our production servers so I simply switched off the SSL Engine on our APR listener and disabled SSL on the http connector in server.xml. If you need SSL with APR then you can find instructions in the Tomcat 6.0 documentation page on SSL.

<Listener className="org.apache.catalina.core.AprLifecycleListener"   SSLEngine="onoff" />
…
<Connector protocol="HTTP/1.1" ... SSLEnabled="truefalse" ... />

At last everything works!