Pivotal Cloud Foundry and Docker - Part 2

This is the second part of a multi-part post for Pivotal Cloud Foundry (PCF) and Docker integration.  In Part 1 of the post, I went over the initial steps to setup PCF for allowing Docker images and how to push apps from a Docker registry (public and private).  This part will go over PCF Docker caveats and the needed security/connectivity between PCF and your private Docker registry.  In this example, I will still be using the JFrog Artifactory tile supported by PCF that was referenced in the last post.

PCF Docker Caveats

There are a few things to be aware of when using Docker images on PCF.

• PCF does not use a Docker client to deploy the images:  From the last post, I mentioned how the Docker images are deployed within the Garden-Linux containers like any other application.  According to PCF/Docker documentation, the Cloud Controller always runs Docker containers on Diego with user namespaces enabled.  This security restriction disables some features from working inside of the Docker container.  
• Private Docker registry using an internal certificate authority (CA):  If you are within an organization, you probably have your own certificate authority that issues certs to your servers.  Since the CA is not well known (Digicert, etc), you will have to either add your CA chain certificates or whitelist your registry (Insecure Registry flag).  These options are available for PCF version 1.7.  On how to implement either of these options, here is a link to the PCF Docker Trusted Registry article.
• Diskspace on Diego Cells:  You will need to keep an eye on your diskspace on your Diego Cell components inside of PCF.  Multiple deployments (cf push) of Docker images leaves left over files on the Diego Cells which will fill up the disk if not maintained.  If the disks fills up on all of your Diego Cells, users will not be able to push application to your environment.  There is a clean up option called "Docker Images Disk-Cleanup Scheduling on Cell VMs" that can be setup inside of the Pivotal Elastic Runtime Tile.  It is under the Application Containers tab.  Again, this feature is on PCF version 1.7.
• PCF requires read access to the source registry for deployment:  There is no spot during a cf push command to put in credentials to authenticate against a source registry.  The registry will need to be set to anonymous read access.  You can still set permissions around uploading Docker images to the source registry.
• Diego requires that the source registry for a Docker image be available when creating new application instances:  If the registry is not available, you will not be able to start or restart the application.  

With that last point, I will now discuss what access needs to be opened between the Artifactory Tile VMs and the PCF environment.

PCF Docker and Artifactory Tile Integration

Here is the workflow from the last post that we will be working with again. The parts of this diagram we will be focusing on are the steps 5 to 10.  The summary of these steps is the request for the Docker image from the private registry to starting the container inside of the PCF Diego Cell.

The Artifactory PCF tile will come preconfigured with two routes setup for accessing the private Docker registry.  These routes are the following:

• https://artifactory-docker-dev.system.domain.com
• https://artifactory-docker-prod.system.domain.com

These routes are configured on the NGINX server (reverse proxy) that comes with the Artifactory PCF tile.  The ports that are used for each of these URLs on the NGINX configuration to route to the HA server pair are the following:

• Port 5001 = artifactory-docker-dev.system.domain.com
• Port 5002 = artifactory-docker-prod.system.domain.com

These ports are important because you need to allow the Diego components access to these ports as well as port 443 to be able to start and restart your Docker applications.  If you do not allow these ports, your cf push command will get stuck on trying to start your container.  So make sure you have the ports allowed in the firewall/load balancer as well as any security groups you might have enabled on your PCF Org/Space.  

These URLs are also mapped to virtual repositories inside of the Artifactory configuration.  The mapping for these items is the following:

• docker-dev = artifactory-docker-dev.system.domain.com
• docker-prod = artifactory-docker-prod.system.domain.com

Now that you know the specific repositories the URLs map to, you can set it so the anonymous user can view the contents of these virtual repositories and the local/remote repos they reference.  Again, this access is needed for successful cf push deployments.  For more information on the Docker registry setup inside of the Artifactory tile, please review their doc site located here.

Summary
The take-away from this post is that there is some configuration behind the scenes to make a cf push to a private Docker registry work.  Not all of it is spelled out inside of one document, but multiple sources.  Hopefully, you find this a good source for a starting point for your PCF Docker setup.  The next post on this topic will go through using the Cloudbees Jenkins PCF tile to publish a Docker build to the Artifactory PCF tile.

~RRRII