Apache 2 or Nginx as a highly secure (PFS) SSL encrypting reverse proxy for eXo Platform 4.0 or any other web application

Eric April 08, 2014 nginx | apache | exo platform | social | exo | security | ssl | debian | proxy | ubuntu



The goal of this tutorial is to explain, including all the subtleties, how to run eXo Platform 4 behind a reverse Proxy using Apache 2 or Nginx on GNU/Linux (Debian).



The goals of the Reverse Proxy are:

  • Securing the eXo platform by hiding it behind the proxy
  • Offloading SSL encryption to the proxy and supporting Perfect Forward Secrecy
  • Using advanced modules such as:
    • caching modules
    • security modules:
      • mod_security on apache or naxsi web application firewall on nginx
    • Google's mod pagespeed, etc.
    • Google SPDY 3.1 (only stable on Nginx at the moment)



Using advanced modules is beyond the scope of this topic and might be included in a future post.


All the instructions below are valid for any Web server. For instance it is valid for Hudson/Jenkins install, the Atlassian webapps such as Jira, Fisheye, Bamboo etc. It should also work with your own webapp running on any platform. It is pure http proxying.

For the sake of simplicity we'll use the eXo platform installed in directory EXO_PLATFORM running on default port 8080.

More information on this great enterprise social portal platform here:

http://www.exoplatform.com

At the time of this writing the latest community release is 4.0.5 available in 12 languages.

Let's move on to the main event of the evening!

Getting Started

Requirements:

  • Basic knowledge of GNU/Linux (Debian specially as instructions will be given for Debian based distributions but may be easily applicable for RedHat/CentOS using the yum package manager)
  • Being familiar with Apache or Nginx configuration directives
  • eXo Platform installed and running on Tomcat (using public platform distribution) listening on port 8080
  • Debian server(s) installed and running.

Preliminary steps

Installing required components/servers

The reverse proxy (Nginx or Apache) can be installed on the same server as eXo or on a separate server.
Nevertheless, it is highly recommended to install the proxy on a dedicated server as it will be more secure. In fact, running the proxy on another server will allow to only expose the proxy ports (80 & 443) to the "outside" world and keep eXo only listening to the internal infrastructure requests.

  • Enough with the words. Let's see what it looks like:



Installing the Reverse Proxy

First you have to choose between Nginx or Apache 2. I personnally favor Nginx as it has a lot better support for cuttting edge web technologies such as websockets, streaming, SPDY 3 etc. Moreover, it seems it scales very well while keeping a smaller memory footprint.

  • On Debian/Ubuntu install the proxy:

For Nginx:

For Debian/Ubuntu, in order to authenticate the nginx repository signature and to eliminate warnings about missing PGP key during installation of the nginx package, it is necessary to add the key used to sign the nginx packages and repository to the apt program keyring. Please download this key from our web site, and add it to the apt program keyring with the following command:

sudo apt-key add nginx_signing.key

For Debian replace codename with Debian distribution codename (i.e.: wheezy), and append the following to the end of the /etc/apt/sources.list file:

deb http://nginx.org/packages/debian/ codename nginx
deb-src http://nginx.org/packages/debian/ codename nginx

For Ubuntu replace codename with Ubuntu distribution codename, and append the following to the end of the /etc/apt/sources.list file:

deb http://nginx.org/packages/ubuntu/ codename nginx
deb-src http://nginx.org/packages/ubuntu/ codename nginx

For Debian/Ubuntu run the following commands:

apt-get update
apt-get install nginx

NOTE: These instructions come straight from Nginx official documentation: http://nginx.org/en/linux_packages.html#mainline

For Apache 2:

sudo apt-get install apache2
sudo apt-get install libapache2-mod-proxy-html

After successful install you need to activate modssl and modproxy_http and it's dependencies. This is easily done on Debian:

sudo a2enmod proxy_http
sudo a2enmod ssl


Configuring the Virtual Host for eXo's frontend

Let's assume your eXo platform installation is reachable at the following address:

https://exo.mydomain.com

We'll also assume you have a Certificate and Private key for ssl encryption located at /etc/ssl/certs/exo.mydomain.com.crt and /etc/ssl/private/exo.mydomain.com.key

Last but not least, as shown on the diagram above we'll assume the proxy server's IP address is 192.168.1.1 and the eXo Platform server's IP address is 192.168.1.2.

Please adapt to your infrastructure's hostname (exo.mydomain.com) and IP addresses if need be.

On Nginx:

Create a file called exo.mydomain.com.conf in:

   /etc/nginx/conf.d

with the following contents:

server {
    listen 443 default ssl;
    keepalive_timeout 70;
    server_name exo.mydomain.com;

# beginning of ssl config

    ssl_certificate  /etc/ssl/certs/exo.mydomain.com.crt;
    ssl_certificate_key  /etc/ssl/private/exo.mydomain.com.key;


# enable session resumption to improve https performance
# http://vincent.bernat.im/en/blog/2011-ssl-session-reuse-rfc5077.html
ssl_session_cache shared:SSL:50m;
ssl_session_timeout 5m;

# Diffie-Hellman parameter for DHE ciphersuites,      recommended 2048 bits
ssl_dhparam /etc/nginx/ssl/dhparam.pem;

# enables server-side protection from BEAST attacks
# http://blog.ivanristic.com/2013/09/is-beast-still-a-threat.html
ssl_prefer_server_ciphers on;
# disable SSLv3(enabled by default since nginx 0.8.19)     since it's less secure then TLS   http://en.wikipedia.org/wiki/Secure_Sockets_Layer#SSL_3.0
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
# ciphers chosen for forward secrecy and compatibility
# http://blog.ivanristic.com/2013/08/configuring-apache-nginx-and-openssl-for-forward-secrecy.html
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK';

# enable ocsp stapling (mechanism by which a site can convey certificate revocation information to visitors in a privacy-preserving, scalable manner)
# http://blog.mozilla.org/security/2013/07/29/ocsp-   stapling-in-firefox/
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.4.4 8.8.8.8 valid=300s;
resolver_timeout 10s;



# config to enable HSTS(HTTP Strict Transport Security) https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security
# to avoid ssl stripping https://en.wikipedia.org/wiki/SSL_stripping#SSL_stripping
add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";   


 #end of ssl config


    root /var/www;

    access_log /var/log/nginx/exo.mydomain.com-access_log;
    error_log  /var/log/nginx/exo.mydomain.com-error_log;


    location / {
        try_files $uri $uri @exo;
    }


    location @exo {
        proxy_pass  http://192.168.1.2:8080;
        proxy_set_header  X-Real-IP $remote_addr;
        proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header  Host $http_host;
        proxy_set_header  X-Forwarded-Proto https;

         # By default we dont want redirect it
        proxy_redirect off;

        # Cache
        proxy_buffering off;
        proxy_cache off;

        client_max_body_size 10m;
    }
}

The directive try_files specifies that Nginx will try to locate the requested resource in its document root (/var/www/ in this example), if it fails to find it it will proxy the request to the @exo backend. This is quite useful as it allows you to place static files in /var/www and they will be served.

The directive client_max_body_size 10m; sets the maximum HTTP post size (i.e. max file size upload amongst others)

You can also play with caching using the following directive (Refer to NGinx documentation here http://wiki.nginx.org/ReverseProxyCachingExample):

proxy_cache

This is setup is the most secure ssl setups configuration (see Forward Secrecy below) as it avoids the weak ciphers and prefers the best ones (it might be too secure for old browsers surch as Internet Explorer 6, you should not be using Internet Explorer 6 anyways).

CREDITS: https://gist.github.com/plentz/6737338

More on Perfect Forward secrecy here: https://en.wikipedia.org/wiki/Forward_secrecy

You need to generate a Diffie-Hellman cert with this command (this will take some time depending on the performance of your server, adjust paths if needed):

cd /etc/nginx/ssl
sudo openssl dhparam -out dhparam.pem 2048

Note: You can also enable Google's SPDY protocol if your Nginx build supports it (the packages installed from sources above have SPDY 3.1, 1.5.12 works perfectly).
You can check by running nginx -V to see the compile time options. If you see spdy, then your nginx build is SPDY enabled.

To enable it in your eXo's virtual host , just add spdy after the ssl directive. The configuration line (2nd line) above should look like this:

listen 443 default ssl spdy;

Google's SPDY 3.1 protocol speeds considerably pages loading time. More infos here:
https://code.google.com/p/mod-spdy/

This module is bundled in Nginx's latest mainline release (1.5.12 on April 2014).
On Ubuntu you can add the following official Nginx Mainline PPA:
https://launchpad.net/~nginx/+archive/development

On Apache:

Create a file called exo.mydomain.com (append .conf to the filename if running Apache version >=2.4) in /etc/apache/sites-available with the following contents

/etc/apache/sites-available/exo.mydomain.com(.conf) :

<VirtualHost *:443>
    SSLEngine on
    SSLCertificateFile /etc/ssl/certs/exo.mydomain.com.crt
    SSLCertificateKeyFile /etc/ssl/private/exo.mydomain.com.key

    SSLProtocol all -SSLv2 -SSLv3
    SSLHonorCipherOrder on
    SSLCipherSuite  "EECDH+ECDSA+AESGCM   EECDH+aRSA+AESGCM 
    EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EEDH+aRSA+SHA384
    EECDH+aRSA+SHA256 EECDH+aRSA+RC4 
    EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS"

    ServerAdmin admin@mydomain.com
    ServerName exo.mydomain.com
    ProxyPassReverse /  http://192.168.1.2:8080/
    ProxyPass / http://192.168.1.2:8080/
    ProxyPreserveHost on
    ProxyRequests Off
    Header always append Access-Control-Allow-Origin: "mydomain.com"
</VirtualHost>


Credits: https://community.qualys.com/blogs/securitylabs/2013/08/05/configuring-apache-nginx-and-openssl-for-forward-secrecy



IMPORTANT NOTE: if you host other NAMED BASED virtual hosts (multiple vhosts on one single IP address) it is advised to set eXo's Virtual Host as the default one as non TLS/SNI web clients will only see the default virtual host


On Nginx this is supplied by the 'default' keyword in the following line:

    listen 443 default ssl;

or if you enabled SPDY:

    listen 443 default ssl spdy;

On Apache 2 you just need to write exo's virtual host as the first one.

Activate the Virtual Host

For Nginx:

If you created the config file as described above in /etc/nginx/conf.d/exo.mydomain.com.conf you don't need to do anything else.

For Apache:

 sudo a2ensite exo.mydomain.com


Configuring eXo's Tomcat connector for Reverse Proxy operations

Open EXO_PLATFORM/conf/server.xml and modify the port 8080 connector as follows:

<Connector address="0.0.0.0" port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
           enableLookups="false" 
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
maxHttpHeaderSize="8192"
acceptCount="100" 
           connectionTimeout="20000" disableUploadTimeout="true"
           URIEncoding="UTF-8"
 proxyName="exo.mydomain.com" proxyPort="443"  scheme="https" />

The important line is:

 proxyName="exo.mydomain.com" proxyPort="443" scheme="https"

This tells Tomcat it's running behind an SSL enabled reverse proxy for which the DNS name is exo.mydomain.com and connection scheme is https.

Increasing Max Open Files

As this configuration is optimized for speed, the SPDY feature can lead to a vast amount of open files on Tomcat 7.

Therefore, depending on your GNU/Linux setup you might need to increase max open files.

On Debian this is done in two steps:

System wide:

Edit /etc/sysctl.conf and add/modify this entry (you may increase/decrease the values):

fs.file-max = 70000

Then for the user running eXo platform:

Edit /etc/security/limits.conf and add (in the example below user exo is running Tomcat):

exo       soft    nofile  50000
exo       hard    nofile  70000

Then check /etc/pam.d/su file and make the following line is not commented.:

session    required   pam_limits.so


Automatic redirection of plain http to https

Below you will find typical Apache & Nginx Rewrite rules to redirect http to https.

Warning: it breaks mobile app on android (haven't tested on iOS). Developer is aware and answered on Google playstore comment.

Add this on top of your virtual host configuration file:

For Nginx:

server {
    listen 80 default;
    server_name exo.mydomain.com;
      ## redirect http to https ##
      rewrite        ^ https://exo.mydomain.com$request_uri? permanent;
}

For Apache:

<VirtualHost *:80>
  ServerAdmin admin@mydomain.com
  ServerName exo.mydomain.com
  Redirect permanent / https://exo.mydomain.com/
</VirtualHost>


Cherry on the cake: Custom error pages to handle eXo maintenance

Both Nginx and Apache can handle custom pages for http errors. Specially a nice placeholder page for error 503 for under maintenance (when the eXo Tomcat server is down or restarting).

This proves particularly useful to handle the scheduled downtimes occurring during offline backups where the eXo server might need to be stopped.

You can also, using the same method, create a custom error page for 404 not found.

First you need to create an html page (50x.html in the example below) and store it (together with its assets) under the document root (defined in your virtual host configuration file, /var/www in this tutorial).

Than add these directives in the virtual host configuration file.

For Nginx:

    error_page 502 @maintenance;
    error_page 503 @maintenance;
    error_page 504 @maintenance;

    location @maintenance {
          rewrite ^(.*)$ /50x.html break;
    }

For Apache:

    ErrorDocument 502 /50x.html
    ErrorDocument 503 /50x.html
    ErrorDocument 504 /50x.html


Result: Qualys SSL Audit

For the Nginx configuration above:

Full Audit in PDF format available here

You can test your installation at: https://www.ssllabs.com/ssltest/analyze.html


That is all for the now.

Stay tuned for additions to this tutorial and other posts related to JavaEE, Systems, Networks and Software architecture.

Thanks for reading, sharing, +1 ing etc...