This howto describes the tomcat+apache setup using mod_jk. The setup described here is where an instance of Apache load balances multiple tomcat instances. The web applications served by both the Tomcat instances must be identical for this to work. This setup can also be modified easily if each tomcat instance is supposed to serve a different web application.
Assumptions
- Apache webserver installed and configured in the default setup.
- Tomcat downloaded and installed. Since this setup is to load balance tomcat instances through Apache, two identical instances of tomcat need to be installed like – /opt/apache-tomcat1 and /opt/apache-tomcat2. Do not symlink as we will modify the server.xml file later, differently for each tomcat instance. Its not mandatory to install tomcat in /opt, for example you can choose to install tomcat as provided by your unix/linux distribution, but we need two instances, so in most cases you would be better off installing it directly from apache’s website. For this document, I have assumed that it is installed in /opt.
- mod_jk downloaded and installed for your distribution. Again it will be better if you download it from apache’s website.
- Java SDK/Runtime. I chose to install jdk1.5.0_09 and jre1.5.0_09. JDK is not required unless you plan to serve servlets through your instances. I normally create symlinks to original JDK and JRE directories, so that other things are not impacted when I upgrade them.
I have defined the following environment variables in /etc/profile:
JRE_HOME=/opt/jre JAVA_HOME=/opt/jdk JDK_HOME=/opt/jdk PATH=$PATH:$JRE_HOME/bin:$JDK_HOME/bin export JAVA_HOME JDK_HOME JRE_HOME PATH
This is how my /opt looks like:
[root@webhst01 ~]# ls -l /opt total 48 lrwxrwxrwx 1 root root 34 Dec 28 12:45 apache-tomcat-1 -> /opt/apache-tomcat-5.5.17_tomcat1/ lrwxrwxrwx 1 root root 34 Dec 28 12:45 apache-tomcat-2 -> /opt/apache-tomcat-5.5.17_tomcat2/ drwxr-xr-x 11 tomcat tomcat 4096 Dec 28 12:42 apache-tomcat-5.5.17_tomcat1 drwxr-xr-x 11 tomcat tomcat 4096 Dec 28 12:42 apache-tomcat-5.5.17_tomcat2 lrwxrwxrwx 1 root root 16 Dec 7 05:49 jdk -> /opt/jdk1.5.0_09 drwxr-xr-x 9 root root 4096 Sep 7 22:14 jdk1.5.0_09 lrwxrwxrwx 1 root root 16 Dec 7 05:49 jre -> /opt/jre1.5.0_09 drwxr-xr-x 5 root root 4096 Sep 7 22:13 jre1.5.0_09
Setting up Tomcat
Tomcat by default runs as root which is not secure. I personally prefer to run it as an ordinary user and start using a custom init script rather than the startup.sh and shutdown.sh scripts.
Creating Tomcat User
[root@webhst01 ~]# /usr/sbin/groupadd tomcat [root@webhst01 ~]# /usr/sbin/useradd -m -g tomcat -c "Tomcat Application User" -d /home/tomcat tomcat [root@webhst01 ~]# /bin/chown -R tomcat:tomcat /opt/apache-tomcat-1/;/bin/chown -R tomcat:tomcat /opt/apache-tomcat-2/ [root@webhst01 ~]# mkdir /var/run/tomcat;/bin/chown tomcat:tomcat /var/run/tomcat
Creating Tomcat Init Script
#!/bin/sh
#
# Tomcat Startup script for the Apache Tomcat Server instance 1
#
# chkconfig: - 85 15
# description: Tomcat Java JSP/Servelet Server instance 1
#
# Local variables
TOMCAT_INSTANCE_NAME="tomcat-1" # Change as required
TOMCAT_INSTANCE_HOME="/opt/apache-$TOMCAT_INSTANCE_NAME"
# Create the Profile Environment... you will need to modify these settings.
export CATALINA_HOME=$TOMCAT_INSTANCE_HOME
export CATALINA_BASE=$CATALINA_HOME
export BASEDIR=$CATALINA_HOME
export CATALINA_PID='/var/run/tomcat/$TOMCAT_INSTANCE_NAME.pid'
export JAVA_HOME='/opt/jdk'
export JDK_HOME='/opt/jdk'
export JRE_HOME='/opt/jre'
if [ "$JAVA_HOME" = "" ]; then
echo
echo "The JAVA_HOME variable is not set. Please set JAVA_HOME within this script."
echo "JAVA_HOME should point to the apropriate base dir of your Java installation, such"
echo "as '/usr/java/jre1.5.0_08' for example."
echo
echo "This script will not start Tomcat without JAVA_HOME set"
echo
exit 1
fi
user=tomcat
catalina=/opt/apache-$TOMCAT_INSTANCE_NAME/bin/catalina.sh
pidfile=/var/run/tomcat/$TOMCAT_INSTANCE_NAME.pid
if [ -e $pidfile ]; then
pid=`cat $pidfile`
fi
start() {
if [ ! -e $pidfile ]; then
echo "Starting Tomcat . . ."
sudo -u $user $catalina start
else
echo "Tomcat appears to be running. Process ID is $pid"
fi
}
stop() {
if [ -e $pidfile ]; then
echo "Shutting down Tomcat. . ."
$catalina stop
if [ "$pidfile" != "/" ]; then
rm -rf $pidfile
fi
else
echo "Tomcat does not appear to be running. No process id found."
exit 1
fi
}
restart() {
stop
start
}
status() {
if [ -e $pidfile ]; then
echo "Tomcat is running with process ID $pid"
else
echo "Tomcat does not appear to be running. No process id found."
fi
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart)
restart
;;
*)
echo $"Usage: $0 {start|stop|status|restart}"
exit 1
esac
exit $?
Setting up server.xml file
server.xml file allows to configure tomcat using a simple XML descriptor. This file resides in conf directory within the tomcat installation. In our case the server.xml file will be almost similar except the port numbers.
Following are the two server.xml files:
tomcat-1 server.xml file
[root@webhst01 ~]# mv /opt/apache-tomcat-1/conf/server.xml /opt/apache-tomcat-1/conf/server.xml.original [root@webhst01 ~]# vi /opt/apache-tomcat-1/conf/server.xml <Server port="8005" shutdown="144b6c105668ca5b10a63545abea46d5"> <!-- Define the Tomcat Stand-Alone Service --> <Service name="Catalina"> <!-- Define an AJP 1.3 Connector on port 8009 --> <Connector port="8009" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" /> <!-- Define the top level container in our container hierarchy --> <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1"> <Host name="localhost" appBase="/var/www/tomcat-1" unpackWARs="true" autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false"> </Host> </Engine> </Service> </Server>
tomcat-2 server.xml file
root@webhst01 ~]# mv /opt/apache-tomcat-2/conf/server.xml /opt/apache-tomcat-2/conf/server.xml.original [root@webhst01 ~]# vi /opt/apache-tomcat-2/conf/server.xml <Server port="8105" shutdown="144b6c105668ca5b10a63545abea46d5"> <!-- Define the Tomcat Stand-Alone Service --> <Service name="Catalina"> <!-- Define an AJP 1.3 Connector on port 8109 --> <Connector port="8109" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" /> <!-- Define the top level container in our container hierarchy --> <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2"> <Host name="localhost" appBase="/var/www/tomcat-2" unpackWARs="true" autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false"> </Host> </Engine> </Service> </Server>
As you can see the port numbers of both the servers are different, one listens on 8005 and the other on 8105. The Server shutdown property is the text string that is sent over a socket connection to stop Tomcat. The default value if “SHUTDOWN”. The shutdown port is always on a loopback interface, which provides host level protection. However, still for better security, replace the default text with some difficult to guess random strings. A random string can be generated as follows:
head -1024c /dev/random | md5sum chmod 600 /opt/apache-tomcat-1/conf/server.xml chmod 600 /opt/apache-tomcat-2/conf/server.xml
The AJP instances are listening on ports 8009 and 8109 respectively.
The mod_jk Connector
mod_jk connector is the communication link between Apache and Tomcat. It listens on a defined port for requests from Apache and forwards those requests to Tomcat. Although tomcat can be run in standalone mode as an Apache alternative and sometimes its reportedly faster as well, but still there are following benefits of using Apache as an frontend to Tomcat:
- Apache can be used to buffer slow connections. Tomcat uses java.io, which uses a thread for each request, so Tomcat can run out of connections as the n umber of slow requests grows. This is a typical problem with application being used by large number of dialup users.
- mod_jk can be used to load balance amongst several Tomcat instances.
- Apache features can be used like cgi, php, modules etc
- Virtual hosts can be isolated in their own Tomcat instances.
Install the mod_jk connector for your distribution, most of the time you need to build it yourself from source. I was lucky enough to find binaries for RHEL4_x86-64 and Debian. I am saying lucky enough because when writing this, I went to tomcat-connectors download site, I could not find the Red Hat connector, which I downloaded earlier, Debian has it in its repositories. Perhaps sometime later I will put in instructions here on how did I compiled the mod_jk connector.
The mod_jk connector file needs to be put in the apache modules directory, which is /etc/httpd/modules (Red Hat and derivatives) or /etc/apache2/modules (Debian and derivatives).
[root@webhst01 ~]# cp mod_jk.so /etc/httpd/modules
I load the mod_jk as follows by putting these lines in the apache configuration files:
<IfModule !mod_jk.c> LoadModule jk_module "/etc/httpd/modules/mod_jk.so" </IfModule> JkWorkersFile "/etc/httpd/conf/workers.properties" JkLogFile "/var/log/httpd/mod_jk.log" JkLogLevel debug JkLogStampFormat "[%a %b %d %H:%M:%S %Y] " JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories JkRequestLogFormat "%w %V %T"
Please note that if you are configuring virtual servers in Apache (as I did) then these lines needs to be put outside the virtual server area. If you put these lines in virtual server then it will never work, I don’t know why, but if you know do tell me.
The workers.properties file
The workers.properties file contains information so mod_jk can connect to Tomcat worker processes. I have set up Apache and Tomcat on the same box, but they can be on different boxes as well. You just need to change localhost in the workers.properties file to appropriate host name or IP address. Ofcourse our Tomcat has been restricted to listen only on localhost, which ofcourse needs to be changed if we want it to accept mod_jk requests from a different host. Perhaps I will update this document someday with that as well.
# File - workers.properties # Purpose - Configures load balanced tomcat worker threads. # Author - Ajitabh Pandey <ajitabhpandey@ajitabhpandey.info> # History - # ver 0.1 - Created on 19th Oct 2006 - AP # ps=/ # # List of all workers by name # worker.list=loadbalancer # # First tomcat instance # worker.tomcat1.port=8009 worker.tomcat1.host=localhost worker.tomcat1.type=ajp13 # lbfactor must be >0. A low lbfactor value for a worker means that worker # performs less work worker.tomcat1.lbfactor=100 # # Second tomcat instance # worker.tomcat2.port=8109 worker.tomcat2.host=localhost worker.tomcat2.type=ajp13 # lbfactor must be >0. A low lbfactor value for a worker means that worker # performs less work worker.tomcat2.lbfactor=100 # # Load Balancer worker instance # The loadbalancer (type lb) worker performs weighted round-robin # load balancing with sticky sessions. # If a worker dies, the load balancer will check its state once in # a while. Until then all work is redirected to peer worker. # worker.loadbalancer.type=lb worker.loadbalancer.balance_workers=tomcat1,tomcat2
Apache virtual host configuration
I prefer to use virtual hosts in Apache even if there is only one host. I have created the following virtual host with the mod_jk mount points defined. So any requests for these mount points will be passed to a Tomcat instance by mod_jk connector as per load balancing algorith defined in workers.properties file.
[root@webhst01 ~]# vi /etc/httpd/conf.d/00_webhst01.unixclinic.net.conf # File - 00_webhst01.unixclinic.net # Purpose - Virtual Server configuration file for webhst01.unixclinic.net # Author - Ajitabh Pandey # History - # ver 0.1 - Created the file on 20th Sept 2006 - AP # <VirtualHost *:80> ServerName webhst01.unixclinic.net ServerAlias webhst01 ServerAdmin webmaster@unixclinic.net DocumentRoot /var/www/webhst01.unixclinic.net # Globally deny access to the WEB-INF directory <LocationMatch '.*WEB-INF.*'> AllowOverride None deny from all </LocationMatch> # Requests for these URIs go to loadbalancer worker thread JkMount /servlets-examples/* loadbalancer </VirtualHost>