UPDATED 06/09/2015: Added support for MYSQL_ environment variables in exoplf image allowing for a fully custom exo+mysql setup (i.e.: change mysql password, database name etc.)

1. Introduction

This post's goal is to explain how to build a dockerized eXo Platform following Docker's core principle of "one task, one container".

In this example eXo platform will be using MySql as the database making it more production ready.

Also eXo's JCR data as well as MySql data will be stored in separate and dedicated Docker Data containers.
To link all this containers together we will be using the newly released docker-compose tool.

The final docker composition will look like this:

2. Containers Descriptions

We will need 4 containers:

• mysql
  • using the official mysql/mysql-server:latest image
• mysqldata
  • will expose volume /var/lib/mysql
• exoplf
  • using a tweaked exo platform image, supporting server.xml customization and custom jcr data volume
• exodata
  • will expose volume /srv/exo where eXo's JCR data will be stored

3. Containers Docker Files

Lets describe these containers.

Mysql

We will pull the official mysql/mysql-server:latest available on DockerHub Registry here:

https://registry.hub.docker.com/_/mysql/

It supports the initial creation of the database and user through Environment variables.

 MYSQL_DATABASE: exo
 MYSQL_USER: exo
 MYSQL_PASSWORD: changeme
 MYSQL_ROOT_PASSWORD: changeme

You can change any of these but note you will need to adjust the server.xml file in the exo-plf container (described later on)

This container will mount /var/lib/mysql from the mysql-data container

Mysql Data

This is the container that will store the Mysql data directory.
It is very simple.
It's Dockerfile looks like this:

FROM busybox:latest  
MAINTAINER Eric Taïeb Walch <teknologist@gmail.com>  
RUN mkdir -p /var/lib/mysql && chmod -R 777 /var/lib/mysql  
VOLUME /var/lib/mysql  

The important line here is:

VOLUME /var/lib/mysql  

It makes the /var/lib/mysql directory mountable by other containers.

DockerHub image is teknologist/mysql-docker-data available at the DockerHub Registry here:

https://registry.hub.docker.com/u/teknologist/mysql-docker-data/

GitHub repo: https://github.com/teknologist/mysql-docker-data

eXo Data

Same as above, this image will be used to spawn a container that holds /srv/exo directory.
This directory contains all the persistent data for eXo's JCR files.

Again it is very simple.
It's Dockerfile looks like this:

FROM busybox:latest  
MAINTAINER Eric Taïeb Walch <teknologist@gmail.com>  
RUN mkdir -p /srv/exo  
RUN chmod 777 -R /srv/exo  
VOLUME /srv/exo  

DockerHub image is teknologist/exo-data

Available at the DockerHub Registry:

https://registry.hub.docker.com/u/teknologist/exo-data/

GitHub repo: https://github.com/teknologist/docker-data

eXo Platform

This image is a fork of eXo's original image.

Changes are:

• Upgraded to PLF 4.2.0-RC1
• Added MySQL JDBC Connector to lib/ directory
• Added server.xml to switch default HSQL database to MySQL
• Added environment variables to setup mysql connection properties

The github repo for this Docker image is available here:
https://github.com/teknologist/ubuntu-jdk7-exo

As you can see the server.xml refers to a mysql database host named mysql.
This is important as this is how containers are linked using docker-compose.

Short story:
If you name a container mysql and link it in the docker-compose.yml file, then docker-compose will set an entry in the hosts file named mysql pointing at the IP of the container.
This makes linking container trivial.

The Docker image is teknologist/ubuntu-jdk7-exo-mysql available at the DockerHub Registry here:

https://registry.hub.docker.com/u/teknologist/ubuntu-jdk7-exo-mysql/

4. Docker composing all these containers together

This is where the magic happens. By writing a simple 25 lines docker-compose.ym we will spawn an entire eXo Platform infrastructure in minutes.

First of all you need to have a recent docker version installed (1.3 or higher) for docker-compose to work.
As of this writing I am using Docker version 1.6.2, build 7c8fca2.
You can find how to install docker here: https://docs.docker.com/installation/

Next step is installing docker-compose.
On supported platforms (GNU/Linux, OSX, etc.) this is as simple as entering the following commands:

curl -L https://github.com/docker/compose/releases/download/1.2.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose  
chmod +x /usr/local/bin/docker-compose  

See https://docs.docker.com/compose/install/ for more information.

Once your docker + docker-compose environment is ready, create a directory, exo-compose for example. In that directory create a file called docker-compose.yml with the following content:

exodata:  
  restart: on-failure:5
  image: teknologist/exo-data
mysqldata:  
  restart: on-failure:5
  image: teknologist/mysql-docker-data
mysql:  
  restart: on-failure:5
  image: mysql/mysql-server:latest
  environment:
    MYSQL_DATABASE: exo
    MYSQL_USER: exo
    MYSQL_PASSWORD: changeme
    MYSQL_ROOT_PASSWORD: changeme
  volumes_from:
  - mysqldata
exoplf:  
  restart: on-failure:5
  ports:
  - 8080:8080/tcp
  image: teknologist/ubuntu-jdk7-exo-mysql
  environment:
    MYSQL_DATABASE: exo
    MYSQL_USER: exo
    MYSQL_PASSWORD: changeme
  links:
  - mysql
  volumes_from:
  - exodata

UPDATED 09/16/2015 to include MYSQL_ environment variables to the exoplf service.

Beware this a yaml file so indentation is of the utmost importance. Gist available here.

As you can see 3 things are important here and let you link everything together:

volumes_from  

This directive instructs each container to mount the volumes from the respective data container.

links:  
 - mysql

This is the main compose directive that links exo platform to mysql. With this directive, docker-compose:

• creates a network connection between the exo and mysql containers
• creates an entry named mysql in eXo container's /etc/hosts file pointing to the mysql container IP

If you look back at the server.xml file, this is how the mysql host in the jdbc connection string gets resolved.

You can find docker-compose full documentation here:
https://docs.docker.com/compose/

Last but not least, both mysql and exoplf services' MYSQL_ environment variables:

    MYSQL_DATABASE: exo
    MYSQL_USER: exo
    MYSQL_PASSWORD: changeme
    MYSQL_ROOT_PASSWORD: changeme

NOTE: MYSQL_ROOT_PASSWORD is only needed in mysql service

NOTE: You may want (highly recommended) change mysql exo user password. In order to do that you need to pull my ubuntu-jdk7-exo-mysql image and modify the server.xml. I recommend setting it up as an environment variable in the docker-compose.yml exoplf section, then changing the server.xml to use that environment variable. I may update the images later to reflect this. UPDATE: I have updated the exoplf image to support MYSQL_ environment variables. It is know trivial to setup custom MySQL connection details. Just set the MYSQL_ environment variables correctly on both exoplf and mysql services in docker-compose.yml - Content has been updated in this post to reflect this.

Now you are ready to spawn the infrastructure. In the directory where you created the docker-compose.yml file, just launch: docker-compose up

eric@morpheus:~/exo-compose$ docker-compose up  
Creating exocompose_mysqldata_1...  
Creating exocompose_exodata_1...  
Creating exocompose_mysql_1...  
Creating exocompose_exoplf_1...  
Attaching to exocompose_mysql_1, exocompose_exoplf_1  
mysql_1  | Running mysql_install_db  
mysql_1  | 2015-06-08 12:07:00 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).  
mysql_1  | 2015-06-08 12:07:00 0 [Note] /usr/sbin/mysqld (mysqld 5.6.25) starting as process 22 ...  
exoplf_1 | 2015-06-08 12:07:01,891 | INFO  | The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib [o.a.catalina.core.AprLifecycleListener<main>]  
exoplf_1 | 2015-06-08 12:07:02,334 | INFO  | Initializing ProtocolHandler ["http-nio-0.0.0.0-8080"] [o.apache.coyote.http11.Http11NioProtocol<main>]  
exoplf_1 | 2015-06-08 12:07:02,384 | INFO  | Using a shared selector for servlet write/read [o.apache.tomcat.util.net.NioSelectorPool<main>]  
exoplf_1 | 2015-06-08 12:07:02,386 | INFO  | Initializing ProtocolHandler ["ajp-bio-0.0.0.0-8009"] [org.apache.coyote.ajp.AjpProtocol<main>]  
exoplf_1 | 2015-06-08 12:07:02,405 | INFO  | Initialization processed in 1216 ms [org.apache.catalina.startup.Catalina<main>]  

NOTE: If you want to run your services in the background, you can pass the -d flag (for daemon mode) to docker-compose up and then use docker-compose ps to see what is currently running. Of course, each docker container supports classic docker commands, so you could check exoplf's logs via a docker logs commands.

I like to start the first time without -d so I can see what is going on in the initial containers setup.

NOTE: Now the thing is that docker-compose starts containers fast and the initial startup of mysql creates the database and user so at immediate startup exo's container will show an error connecting to the database. You can ignore that because while exo continues its startup process the database will become available. This only happens the first time as the databases need to be created.

Depending on your docker's host performance after some time you'll see:

exoplf_1 | 2015-06-08 12:09:11,830 | INFO  | Starting ProtocolHandler ["http-nio-0.0.0.0-8080"] [o.apache.coyote.http11.Http11NioProtocol<main>]  
exoplf_1 | 2015-06-08 12:09:11,835 | INFO  | Starting ProtocolHandler ["ajp-bio-0.0.0.0-8009"] [org.apache.coyote.ajp.AjpProtocol<main>]  
exoplf_1 | 2015-06-08 12:09:11,838 | INFO  | Server startup in 129433 ms [org.apache.catalina.startup.Catalina<main>]  

And you can connect to your docker's host port 8080 (provided you didn't change it in the docker-compose ports directive) and you'll be greeted with the initial eXo setup page:

Complete the rest of the setup wizard and you are set.

5. Operating your eXo/Mysql Docker composition

You can stop and start your infrastructure as you wish. Persistence data is kept in the data containers so no loss of data/configuration.

Stopping your existing infrastructure

If you started in foreground , just hit [CTRL]-C.
If you started in background, just enter: docker-compose stop

eric@morpheus:~/exo-compose$ docker-compose stop  
Stopping exocompose_exoplf_1...  
Stopping exocompose_mysql_1...  

Note you don't see anything related to the data containers as they don't really run, they just provide volumes.

Starting your existing infrastructure

Again, just enter: docker-compose start

eric@morpheus:~/exo-compose$ docker-compose start  
Starting exocompose_mysqldata_1...  
Starting exocompose_exodata_1...  
Starting exocompose_mysql_1...  
Starting exocompose_exoplf_1...  

To see what is currently running in your compose infrastructure

Just enter: docker-compose ps

eric@morpheus:~/exo-compose$ docker-compose ps  
         Name                       Command               State            Ports
-----------------------------------------------------------------------------------------
exocompose_exodata_1     /bin/sh                          Exit 0  
exocompose_exoplf_1      /bin/sh -c ${EXO_APP_DIR}/ ...   Up       0.0.0.0:8080->8080/tcp  
exocompose_mysql_1       /entrypoint.sh mysqld            Up       3306/tcp  
exocompose_mysqldata_1   /bin/sh                          Exit 0  

Again, you see the data containers are not in running state and that is normal.
Also note mysql port 3306 is not exposed outside the container. docker-compose does all the tunnelling for you.

To see your container's logs

If you want to tail exoplf's log just type: docker logs exocompose_exoplf_1

eric@morpheus:~/exo-compose$ docker logs exocompose_exoplf_1  
2015-06-08 12:22:12,758 | INFO  | The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib [o.a.catalina.core.AprLifecycleListener<main>]  
2015-06-08 12:22:13,129 | INFO  | Initializing ProtocolHandler ["http-nio-0.0.0.0-8080"] [o.apache.coyote.http11.Http11NioProtocol<main>]  
2015-06-08 12:22:13,177 | INFO  | Using a shared selector for servlet write/read [o.apache.tomcat.util.net.NioSelectorPool<main>]  
2015-06-08 12:22:13,179 | INFO  | Initializing ProtocolHandler ["ajp-bio-0.0.0.0-8009"] [org.apache.coyote.ajp.AjpProtocol<main>]  
2015-06-08 12:22:13,198 | INFO  | Initialization processed in 1274 ms [org.apache.catalina.startup.Catalina<main>]  
2015-06-08 12:22:13,517 | INFO  | Starting service Catalina [org.apache.catalina.core.StandardService<main>]  
2015-06-08 12:22:13,517 | INFO  | Starting Servlet Engine: Apache Tomcat/7.0.55 [org.apache.catalina.core.StandardEngine<main>]  
2015-06-08 12:22:13,543 | INFO  | Deploying web application archive /opt/exo/platform-community-4.2.0-RC1/webapps/forum-gadgets.war [org.apache.catalina.startup.HostConfig<localhost-startStop-5>]  
2015-06-08 12:22:13,543 | INFO  | Deploying web application archive /opt/exo/platform-community-4.2.0-RC1/webapps/wiki.war [org.apache.catalina.startup.HostConfig<localhost-startStop-9>]  
2015-06-08 12:22:13,543 | INFO  | Deploying web application archive /opt/exo/platform-community-4.2.0-RC1/webapps/eXoPlatformResources.war [org.apache.catalina.startup.HostConfig<localhost-startStop-10>]  

Again, each of this containers supports all docker commands.

6. Final thoughts

As you can see docker-compose makes building docker based infrastructures very easy.

Other tools such as Google's Kubernetes are very promising too. They not only let you compose but also scale (through minions, yes I didn't invent that).

Another very promising tool I have used to explore docker composition on CoreOS is Rancher

It looks like we are moving more and more from VM based cloud infrastructure to highly dense container based ones. Think lower overhead (if not zero), think also green!

Thanks for reading, sharing, commenting, +1 ing etc...

This tutorial aims to explain how to create a Codenvy Factory to provide a cloud based development and testing environment for developing an eXo extension based on Docker containers.

A Codenvy Factory can be used for:

• Creating a pre configured environment for a platform add ons (extension, skins etc.) development
• Providing a development environment for modifying an existing software and testing it out of the box
• Many more... Use your imagination
  
  
  

Contents

• Introduction
• Getting Started: Requirements
• Preliminary steps: Installing Codenvy CLI
• Anatomy of a Codenvy Factory: Brief overview of the configuration file
• Preparing the Codenvy Custom Runner Dockerfile: How the Codenvy custom runners and the Docker images work
• Publishing the Codenvy custom runner Dockerfile
• Assembling the Factory file: factory.json
• Create the Factory using the factory.json file
• Use or share your Factory

Introduction

About a month ago I had the chance to be invited to play with Codenvy's new platform and more specifically Factories.

I was asked to try to build factories to help eXo developers speed up/automate the setup of a development environment ready to code, test and deploy their extensions in an eXo platform automatically provisioned runtime, all running inside codenvy platform in the cloud.

I was amazed by Codenvy's overall value proposition. The platform is very impresive and I said twitted early on, in my opinion this signals clearly the dawn of a new generation of IDEs.

As a result of this work, I produced 2 factories for eXo. Both are available on eXo platform's add on catalog. One which allows to extend and code on the chat application extension and server. The other being an empty extension skeleton project. This tutorial is based on the latter.

In Codenvy, there are three types of Codenvy factories:

• Hack Factories
• Tracked Factories (require a special account)
• Branded Factories (require a special account)

More on Codenvy Factory types here: Codenvy Factories For Sales & Support

Basically, Tracked and Branded Factories allow more customization (ie: custom Welcome screens, etc.) and provide powerful analytics features.

NOTE: Creating a Tracked Factory requires a special account while Hack Factories are free and available to everyone. If you wish to use Tracked Factories, contact Codenvy using the link above for more information.

In this example we will address Hack Factories (we'll call them normal factories) and Tracked Factories. We will use the eXo platform 4.1RC1 and an empty eXo add-on maven project skeleton to create a factory which allows jumpstarting the development of an extension for the eXo Platform.

More information on this great enterprise social portal platform here:

eXo Platform website

We will show you how to create a custom runner that spawns a Docker container with eXo Platform 4.1RC1, builds your maven based project and deploys it to the eXo Platform 4.1RC1 runtime using eXo's add-on manager, then starts the eXo runtime in order to see the results.

NOTE: In a future codenvy release you might be able to build all kinds of maven projects in the IDE. At the time of this writing it is difficult to succesfully retrieve multiple build artifacts for multi modules maven projects successfully.

Therefore, in order to be as flexible and as universally useable as possible, we do the build inside the custom runner using maven.

For example, eXo add-ons are maven multi modules projects which produce variuous artifacts, one of them being a zip file that we will provide to eXo's add-on manager in order to have it installed. This is the case for eXo empty add-on project we use in this tutorial.

Getting Started

Requirements:

• Basic knowledge of GNU/Linux
• A Codenvy account
• Codenvy CLI installed on your local machine (see Codenvy CLI Docs)
• Basic Docker knowledge (particularly: Docker Best Practices)
• Space to server html files and Dockerfile recipes (ie: webserver etc.)
• Github account
• DockerHub account linked to your GitHub account. See: Docker Hub and Docker Hub Documentation

For the sake of simplicity, in this tutorial, we will use a Dropbox account to host the html files (Only for tracked factories).

Preliminary steps

Installing required components: Codenvy CLI

• If you don't have a codenvy account, create one by navigating to Create a Codenvy account in your favorite modern browser.
• Install the Codenvy CLI on your local machine following these steps: Codenvy CLI Docs
• Verify you can login by opening a terminal and trying:

    # codenvy login
    Username: yourusername
    Password for yourusername:**********

If everything is OK you should see this message:

      Login success on default remote 'default' [https://codenvy.com]

NOTE: Make sure you are using bash as your shell. I have found issues using other shells such as fish. Also make sure you have correctly setup your PATH so codenvy binary is in your PATH.

Anatomy of a Codenvy Factory

Codenvy official documentation for custom runners is available here: Codenvy Custom Runners Documentation

Long story short, we will be writing a JSON file (we will name it exo-extension-factory.json) describing the factory. Then we will create the factory using the codenvy cli and passing that json file.

That JSON file will among other have:

• a reference to a custom runner Dockerfile recipe
• a reference to an html resource to be used as Welcome screen (Tracked Factories only)

To build the Custom runner Docker image recipe we will use intermediate images. All Docker images will be built and pushed to the DockerHub public repository, except for the last one which will always be built by Codenvy at runtime in order to deploy the newly compiled add-on (see Docker Inheritance diagrams in the next section for more information).

In this next section I will show you how to create such a Dockerfile.

Preparing the Codenvy Custom Runner Dockerfile

In codenvy a runner is the term to describe an execution runtime. In order to deploy custom runners, Codenvy uses Docker recipes. In this example our custom runner will be an eXo Platform distribution Docker image.

For the sake of simplicity we will write a Dockerfile recipe based on an already prepared image which has:

• Ubuntu minimal
• Java JDK 7
• Apache Maven including all eXo maven dependencies pre installed in .m2/repository (speeds up build time)
• eXo Platform 4.1RC1 with first time wizard completed (this speeds up dramatically the runner startup time)

This image is hosted on the DockerHub at this address: https://registry.hub.docker.com/u/teknologist/exo-maven-setup-done/

You can play with it your docker installation by pulling the image:

docker pull teknologist/exo-maven-setup-done

The base image for this pre-setup image is available here https://registry.hub.docker.com/u/teknologist/exo-maven/:
It is strictly the same except for the fact that the eXo platform is already setup through the first time wizard.
The source for this docker image is available here: https://github.com/teknologist/exo-maven-docker

In order to install an eXo add-on using the bundled add-on manager script we need to provide a local.json file in EXO PLATFORM directory. This file describes the add-on, its availability, compatibility, license etc.

A sample file local.json file:

[
    {
        id: "exo-custom-addon",
        version: "1.0.0",
        unstable: true,
        name: "Custom Add-on",
        description: "eXo custom extension",
        releaseDate: "2014-11-12T22:00:00.000Z",
        sourceUrl: "https://github.com/teknologist/exo-empty-extension",
        downloadUrl: "file:///home/exo/.m2/repository/org/exoplatform/archetype/empty-extension-packaging-addon/1.0.0-SNAPSHOT/empty-extension-packaging-addon-1.0.0-SNAPSHOT.zip",
        vendor: "eXo",
        license: "LGPLv3",
        licenseUrl: "https://www.gnu.org/licenses/lgpl-3.0.txt",
        mustAcceptLicense: false,
        supportedDistributions: "community,enterprise",
        supportedApplicationServers: "tomcat",
        compatibility: "[4.1.0-RC1,)"
    }
]

As you see we include the full path to the built maven artifact:

/home/exo/.m2/repository/org/exoplatform/archetype/empty-extension-packaging-addon/1.0.0-SNAPSHOT/empty-extension-packaging-addon-1.0.0-SNAPSHOT.zip

as 'maven install' command will compile, assemble and install the artifact to exo's user maven repository in the codenvy custom runner Docker conrtainer.

You have 2 choices now:

• include the local.json file in your source code repository
• include it in your docker recipe repository

Basically, choose the first option if you have control over the source code repository (ie: you own the repository, or you are ok with forking it), choose the second option if you don't have control over the source code repository and you don't want to fork it.

Docker Images' names are in Bold, except for the ephemeral image built by Codenvy at runtime which is unpublished and therefore doesn't have a name.

One of the main reasons we choose to put most of the content on pre built Docker images (i.e.: images we then pull from the DockerHub repository) as opposed to delegating it to the Codenvy custom runner DockerFile, is to speed up startup times for the custom runner.

As the Codenvy custom runner Docker image is built every time you start the runner, we save a substantial amout of startup time by having most of its contents pre built at the Dockerhub repository. This is especially true for a large platform such as eXo or platforms which require lots of additional components downloaded and installed (i.e.: MongoDb, Databases etc.).

Find below diagram for option 1:

The second option is a bit more difficult as you will have to build an intermediate Dockerfile repository to include the local.json file.

Find below diagram shown earlier, modified for option 2:

NOTE: I highly recommend using the first option. Only use the second option as a last resort, as it adds another docker intermediate image.

First option: include the local.json file in your source code repository

We will assume the local.json file is at the root of your source code repository.

In this example, for an empty eXo extension the Dockerfile will have this content:

FROM teknologist/exo-maven-setup-done

ENV EXOADDON_SRC_DIR /home/${EXO_USER}/src 

USER exo
RUN mkdir -p ${EXOADDON_SRC_DIR}

#Get app src from Codenvy Project
ADD $app_src$ ${EXOADDON_SRC_DIR}/app_src.zip

#ADDON: Unpack, build and install with addon manager
RUN cd ${EXOADDON_SRC_DIR} && unzip -q app_src.zip && \
cd ${EXOADDON_SRC_DIR} && ${M2_HOME}/bin/mvn clean install -q -Dmaven.test.skip --batch-mode  && \
cp ${EXOADDON_SRC_DIR}/local.json ${EXO_APP_DIR}/current/addons/
cd ${EXO_APP_DIR}/current/ && \
 ./addon install exo-custom-addon:1.0.0 --offline --snapshots --unstable

There are 3 important instructions here:

This is a codenvy specific which copies your source code tree to your docker based custom runner:

    ADD $app_src$ ${EXOADDON_SRC_DIR}/app_src.zip

This one adds the local.json (eXo add-on desciption) to your eXo installation:

    cp ${EXOADDON_SRC_DIR}/local.json ${EXO_APP_DIR}/current/addons/

The last one installs your referenced add-on using eXo bundled addon manager:

 ./addon install exo-custom-addon:1.0.0 --offline --snapshots --unstable

Second option: include local.json in your docker recipe repository

In this case you are going to create an intermediate docker image.

First create a repository on github to hold your Docker recipe files.
We will assume the local.json file is at the root of your source code repository.

You will have 3 files:

• Dockerfile
• local.json
• README.MD (optional)

The Dockerfile is super simple:

FROM teknologist/exo-maven-setup-done
ENV EXOADDON_SRC_DIR /home/${EXO_USER}/src

USER exo  
RUN mkdir -p ${EXOADDON_SRC_DIR}
ADD local.json ${EXO_APP_DIR}/current/addons/

This Docker recipe simply adds the local.json to the eXo installation from the base exo-maven-setup-done Docker image.

Now head up to your DockerHub account and create an automated build based on the Github repository you just created.

You should now have a new docker image automatically built and available.

You can find a fully working githuib docker recipe repository example here: https://github.com/teknologist/exo-empty-addon-docker

The corresponding DockerHub automatically built image is available here: https://registry.hub.docker.com/u/teknologist/exo-empty-addon-docker/

It's named teknologist/exo-empty-addon-docker.

Therefore the codenvy custom runner's Dockerfile, which will inherit teknologist/exo-empty-addon-docker image, will have this content:

FROM teknologist/exo-empty-addon-docker

EXPOSE 8080
ENV CODENVY_APP_PORT_8080_HTTP 8080
ENV EXOADDON_SRC_DIR /home/${EXO_USER}/src 

USER exo
RUN mkdir -p ${EXOADDON_SRC_DIR}

#Get app src from Codenvy Project
ADD $app_src$ ${EXOADDON_SRC_DIR}/app_src.zip

#ADDON: Unpack, build and install with addon manager
RUN cd ${EXOADDON_SRC_DIR} && unzip -q app_src.zip && \
cd ${EXOADDON_SRC_DIR} && ${M2_HOME}/bin/mvn clean install -q -Dmaven.test.skip --batch-mode  && \
cd ${EXO_APP_DIR}/current/ && \
 ./addon install exo-custom-addon:1.0.0 --offline --snapshots --unstable

There are 2 important instructions here:

This is a codenvy specific which copies your source code tree to your docker based custom runner:

    ADD $app_src$ ${EXOADDON_SRC_DIR}/app_src.zip

The last one installs your referenced add-on using eXo bundled addon manager:

 ./addon install exo-custom-addon:1.0.0 --offline --snapshots --unstable

NOTE: We don't have to deal with the local.json file anymore as it is already included in the intermediate image.

Publishing the Codenvy custom runner Dockerfile

For the rest of the tutorial we will use Option 1.

Therefore our custom runner Dockerfile looks like this:

FROM teknologist/exo-empty-addon-docker

EXPOSE 8080
ENV CODENVY_APP_PORT_8080_HTTP 8080
ENV EXOADDON_SRC_DIR /home/${EXO_USER}/src 

USER exo
RUN mkdir -p ${EXOADDON_SRC_DIR}

#Get app src from Codenvy Project
ADD $app_src$ ${EXOADDON_SRC_DIR}/app_src.zip

#ADDON: Unpack, build and install with addon manager
RUN cd ${EXOADDON_SRC_DIR} && unzip -q app_src.zip && \
cd ${EXOADDON_SRC_DIR} && ${M2_HOME}/bin/mvn clean install -q -Dmaven.test.skip --batch-mode  && \
cp ${EXOADDON_SRC_DIR}/local.json ${EXO_APP_DIR}/current/addons/ && \
cd ${EXO_APP_DIR}/current/ && \
 ./addon install exo-custom-addon:1.0.0 --offline --snapshots --unstable

We need to publish this as an http available resource. We will choose a Github Gist as it is flexible, lets you maintain revisions and doesn't need any infrastructure management on your side.

I have published this Dockerfile as Gist on my account. It is available here: Dockerfile Gist

And the raw Gist URL needed by the factory.json file is: Dockerfile Gist for factory.json

Assembling the Factory file: factory.json

The factory.json configuration file looks like this:

{
    "project": {
        "name": "eXoEmptyExtensionAddOn",
        "visibility": "public",
        "type": "maven",
         "builders":{
         "default":"maven"
      },
        "runners":{
            "configs":{
                "project:/docker/exo-empty-extension":{
                    "ram":4096,
                    "variables":{

                    },
                    "options":{

                    }
                }
            },
            "default":"project:/docker/exo-empty-extension"
        },
        "description":"eXo Platform 4.1 with an empty addon skeleton.",
        "attributes": {}
    },
    "source": {
        "project": {
            "location": "https://github.com/teknologist/exo-empty-extension.git",
            "type": "git",
            "parameters": {
                "commitId": "1c8d67612df4c2d388f82c18d3f22a0e5d96d5ca"
            }
        },
        "runners": {
            "/docker/exo-empty-extension" : {
                "location" : "https://gist.githubusercontent.com/teknologist/ff5ae3149acbe0dab1b7/raw/2958a761771a4dee5625d07812fa297dd4743514/exo-empty-extension-docker-runner"
            }
        }
    },
    "actions": {

            "findReplace" : [
            {
                "files" : [
                "pom.xml"
                ],
                "entries" : [
                {
                    "find":"<modelVersion>4.0.0<\/modelVersion>",
                    "replace" : "<modelVersion>4.0.0</modelVersion> <repositories><repository><id>exo-public-repository-group</id><name>eXo Public Maven Repository Group</name><url>http://repository.exoplatform.org/public</url><layout>default</layout><releases><enabled>true</enabled><updatePolicy>never</updatePolicy></releases><snapshots><enabled>true</enabled><updatePolicy>never</updatePolicy></snapshots></repository></repositories>",
                    "replacemode" : "text_multipass"
                }
                ]

            }

            ]
  },
    "variables": [],
    "v": "2.0"
}

The important lines here are:

"visibility": "public"

Obviously makes your Factory publicly visible/shareable.

"type": "maven"

This example project is of type maven

        "builders":{
            "default":"maven"
        },

Builder defaults to maven. This is for building the addon in Codenvy's Editor (distinct from the build we execute later on in the custom runner)

Then we define the project source code repository and custom runner we have created earlier in the tutorial.

This is done here:

"source": {
    "project": {
        "location": "https://github.com/teknologist/exo-empty-extension.git",
        "type": "git",
        "parameters": {
            "commitId": "1c8d67612df4c2d388f82c18d3f22a0e5d96d5ca"
        }
    },
    "runners": {
        "/docker/exo-empty-extension" : {
            "location" : "https://gist.githubusercontent.com/teknologist/ff5ae3149acbe0dab1b7/raw/2958a761771a4dee5625d07812fa297dd4743514/exo-empty-extension-docker-runner"
        }
    }
}

And then set up the default runner here together with its memory resources (other options/variables may be added. See Codenvy Documentation):

"runners":{
            "configs":{
                "project:/docker/exo-empty-extension":{
                    "ram":4096,
                    "variables":{
                    },
                    "options":{
                    }
                }
            },
            "default":"project:/docker/exo-empty-extension"
}

NOTE: We added a find/replace section to add the eXo maven repository to the pom.xml files as eXo dependencies are not in the public maven repository. That is not an issue when the built is executed in the runner as we have included the repository in the maven settings.xml.
Nevertheless, it is currently impossible to customize maven in Codenvy's IDE builder. Therefore we use this trick to be able to build using Codenvy's maven based builtin builder.

The final result for a Tracked Factory looks like this:

{
        "project": {
            "name": "eXoEmptyExtensionAddOn",
            "visibility": "public",
            "type": "maven",
             "builders":{
             "default":"maven"
          },
            "runners":{
                "configs":{
                    "project:/docker/exo-empty-extension":{
                        "ram":4096,
                        "variables":{

                        },
                        "options":{

                        }
                    }
                },
                "default":"project:/docker/exo-empty-extension"
            },
            "description":"eXo Platform 4.1 with an empty addon skeleton.",
            "attributes": {}
        },
        "source": {
            "project": {
                "location": "https://github.com/teknologist/exo-empty-extension.git",
                "type": "git",
                "parameters": {
                    "commitId": "1c8d67612df4c2d388f82c18d3f22a0e5d96d5ca"
                }
            },
            "runners": {
                "/docker/exo-empty-extension" : {
                    "location" : "https://gist.githubusercontent.com/teknologist/ff5ae3149acbe0dab1b7/raw/2958a761771a4dee5625d07812fa297dd4743514/exo-empty-extension-docker-runner"
                }
            }
        },
        "creator":{
            "name":"Your name",
            "email":"youremail@yourdomain.com",
            "accountId":"accountxxxxxxxxxxxx"
        },
        "actions": {
              "welcome": {
                   "authenticated": {
                   "title": "Welcome",
                   "contenturl": "https://dl.dropboxusercontent.com/u/663951/codenvy-exo/exo-empty-addon.html"
              },
             "nonauthenticated": {
                "title": "Welcome",
                "contenturl": "https://dl.dropboxusercontent.com/u/663951/codenvy-exo/exo-empty-addon.html"
             }
          },
            "findReplace" : [
                {
                    "files" : [
                    "pom.xml"
                    ],
                    "entries" : [
                    {
                        "find":"<modelVersion>4.0.0<\/modelVersion>",
                        "replace" : "<modelVersion>4.0.0</modelVersion> <repositories><repository><id>exo-public-repository-group</id><name>eXo Public Maven Repository Group</name><url>http://repository.exoplatform.org/public</url><layout>default</layout><releases><enabled>true</enabled><updatePolicy>never</updatePolicy></releases><snapshots><enabled>true</enabled><updatePolicy>never</updatePolicy></snapshots></repository></repositories>",
                        "replacemode" : "text_multipass"
                    }
                    ]

                }

                ]
      },
        "variables": [],
        "v": "2.0"
    }

Complete file available as Github Gist here: https://gist.github.com/teknologist/5e5bec3bb2924013ac5f

OPTIONAL: Make this factory a Tracked Factory

A Tracked Factory is a simple Factory where we add a special accountID that allows advanced customization options and enables analytics.

To make this factory a Tracked Factory we add an accountId to the creator snippet

"creator":{
    "name":"Your name",
    "email":"youremail@yourdomain.com",
    "accountId":"accountxxxxxxxxxxxx"
}

In order to retrieve you accountId:

At this moment, you need to use our rest APIs to get your account ID.

The documentation on Codenvy rest APIs can be found under the API Docs: Codenvy API Docs
You can easily make a call to the API and get your information.

Using your browser, Login under the server.

Once you see the dashboard, call the /api/account REST endpoint by entering the following URL: https://codenvy.com/api/account

In the JSON provided in response, you have the following information:

[
  {
    "userId" : "userxxxxxxxxxxxxxxxx",
    "roles" : [
      "account/owner"
    ],
    "accountReference" : {
      "id" : "accountxxxxxxxxxxxx",
      "links" : [
        {
          "method" : "GET",
          "rel" : "get by id",
          "href" : "http://nightly.codenvy-stg.com/api/account/accountxxxxxxxxxxxx",
          "produces" : "application/json",
          "parameters" : []
        }
      ],
      "name" : "yourusername"
    }
    ...
  }
]

Your Tracked Factory ID, is the value behind “id” key under “accountReference”. (Be careful, this one is different from your userID)

accountxxxxxxxxxxxx

All the REST Api documentation is available here:Codenvy API Docs - Swagger style

We can also add a Welcome screen when using a Tracked Factory:

You basically create whatever html resource you want and reference it in the factory.json file.

It will then be displayed in the right side pane as shown below:

For this example, I created the html resource on my DropBox Public folder account. It is available here:

Sample welcome file served from Dropbox

Add the welcome snippet below to the actions section :

    "actions": {
    ...

      "welcome": {
         "authenticated": {
            "title": "Welcome",
            "contenturl": "https://dl.dropboxusercontent.com/u/663951/codenvy-exo/exo-empty-addon.html"
         },
         "nonauthenticated": {
            "title": "Welcome",
            "contenturl": "https://dl.dropboxusercontent.com/u/663951/codenvy-exo/exo-empty-addon.html"
         }
      },

      ...

Complete file available as Github Gist here: Gist for Tracked Factory Dockerfile

Create the Factory using the factory.json file

We now have all resources available via http (Dockefile, html welcome screens, add-on source code git repository).

We also created a factory.json file which describes our factory and references these resources.

Creating the factory is as simple as login in codenvy with the cli and issuing a codenvy create-factory command.

This translates to:

# codenvy login
Username: yourusername
Password for yourusername:*************

If everything is OK you should see this message:

  Login success on default remote 'default' [https://codenvy.com]

Then issue (suppose you saved factory.json file as /home/user/factory.json):

   codenvy create-factory /home/user/factory.json

If the command is successful it should output a Factory URL:

   Factory URL: https://codenvy.com/f?id=6r7mhjhjhsd444nlv

NOTE: The Factory URL above is obviously invalid. Launching it would yield to invalid factory URL message

Use or share your Factory

Open the produced URL in a modern browser and you'll have a temporary workspace with your project sources and runner configured.

In order to build the add'on, hit the [BUILD] button:

In order to deploy and run, hit the [RUN] button:

You are done.
You may now share that URL with anyone you want.
It will open in a temporary workspace and the user will have the choice to persist it.

If you BUILD the project:

Codenvy will use its builtin maven builder to compile and package the project using its pom.xml files. With this eXo sample, this is made possible because we added via the find/replace factory feature the eXo maven repositories.

Your should see a "BUILD SUCCESS" message.

If you RUN the project:

Codenvy will first build your Docker image based Custom runner (This might take some time depending on the docker images cache status and size/complexity of the image):

It will then deploy your eXo add-on using eXo's bundled add-on manager:

Then it will start eXo Platform:

Wait a bit for eXo to finish starting up (around 60s currently on Codenvy) and Click on the URL displayed in blue by Codenvy for the Application :

You should be greated with the eXo Login Screen:

As stated in the Docker Image README file for the base image, default credentials are:

Default usernames and password:

Normal user :

• username: exo
• password: password

Administrative user:

• username: root
• password: password

That is all for the now.

More on Codenvy: Codenvy Homepage

I would like to thank Stevan Le Meur, Tyler Jewell, Benjamin Mestrallet, Andrey Parfonov and Eugene Ivantsov from Codenvy for their precious help throughout the project.

Thanks for reading, sharing, +1 ing etc...