Cloud Foundry: Life Is Too Short

Pivotal’s public cloud version of Cloud Foundry really struggles with the loose integration of third-party services. To appeal to ISVs and others with real-world complexity in their applications, Pivotal needs to identify a coherent product and concentrate on delivering something that works. I tried assiduously to use it and ultimately failed. In case you think I’m being a bit harsh on Pivotal, this system has been in beta for more than two years. By now, it should work.

I’ve been promising for some time to write a post on my experiences with Cloud Foundry, and recent announcements spurred me on to have a look. We are starting to see a preference for cloud deployments being expressed by enterprises procuring independent software vendor (ISV) applications, and the ISVs need to respond to this preference. The benefit to the enterprise is that a cloud deployment takes much of the complexity out of setting up an ISV application. There is no requirement to source a hardware platform and some rack space in a data center, and the sizing problem is pushed straight into the lap of the ISV.

In responding to this growing enterprise preference, mid-scale ISVs have been finding PaaS quite appealing. PaaS removes many of the additional enterprise requirements—auto scaling, resilience, backup, security, etc.—from the ISV’s responsibility and transfers them to the remit of the cloud provider. This allows the ISV to concentrate on its core competencies while still providing a self-contained service offering to the customer. In addition, enterprises may (perhaps foolishly) consider a large PaaS cloud provider like Pivotal or Red Hat to be more credible in the delivery of such underlying services than the ISV itself.

On this basis, I set about trying to understand how to deploy an ISV application to Cloud Foundry. The application in question is a Grails/JavaScript single-page web application. This should be simple, right? After all, Grails comes from Pivotal. However, it isn’t necessarily all that easy: the level of simplicity depends on how the deployment to Cloud Foundry is integrated with any existing lifecycle processes the ISV may have and may continue to require. Pivotal talks about its “Golden On-Ramp,” its effort to help developers learn Cloud Foundry and quickly get applications deployed. This obviously has to be a goal.

The first of Pivotal’s problems is the multiplicity of different application lifecycle management processes that may already be in place. Mike Kavis has been posting here about agile, and here we are, talking about integrating PaaS into a more-or-less agile development process, one its the key factors being the requirement to integrate, test, build, and deploy as continuously as possible. With ISV applications, this process isn’t normally extended all the way to customer deployment. Customers are supported against defined builds, for which there is typically additional testing and traceability. However, there should at least be a nightly build and unit test followed by a deployment.

Build is always trickier than one would like it to be. In the case of this particular ISV application, the build server used is Jenkins, which stitches together the JavaScript, Groovy/Java, and a few other things into a WAR file for deployment, deploys it, and runs tests. Ultimately, the process always involves turning some source code into an executable and running that executable. There are two basic  ways to go: (1) embed the source code repository in the PaaS (this is the dominant philosophy of Red Hat OpenShift, which I will look at in another post), or (2) assume the build is outside the platform, and provide support for deployment of built artifacts (e.g., WAR files). This is the way Pivotal has gone. Pivotal also provides support for third-party build tools. One of the benefits of Pivotal’s approach to ISVs is that it allows them to maintain a single set of traceable build artifacts for both cloud-hosted and on-premises deployment.

I tried Cloud Foundry once before with this application, and failed. One of my abiding memories of the failed attempt was of the command line interface (CLI), required to configure and deploy the application. And…the CLI is still there. To be fair, there appear to be two alternatives, both of which I explored first:

1) A new CLI has been written and is currently in beta. The build server for this ISV application runs on a 32-bit Linux VM; the build process isn’t onerous, and there has never been a reason to upgrade that machine. The new CLI doesn’t have a 32-bit Linux download. I tried to build it from source, but there is a rat’s nest of dependencies, so I soon gave that up.

2) Third party tools: In particular, there  is a documented process for running Jenkins as a build server, from… wait for it… a CloudBees account. In other words, you push your codebase to a public GIT repository, link your CloudBees account with your Pivotal account through OAuth, and then run a Jenkins job on CloudBees, which uploads to Cloud Foundry. Really, Pivotal. Life is too short. While doing all this, you doubtless have had plenty of time to ponder why you don’t just run your application on CloudBees and forget about Cloud Foundry. In any case, this also seems to involve putting your source code into a public GIT repository, which doesn’t sound terribly appealing.

Having settled on using the same antediluvian CLI as I did the last time, I didn’t hold out much hope for success, and at this point things only got worse. The CLI has a dependency on Ruby, but not just any old Ruby: a newer version of Ruby than was supported by the O/S on the build server. So I had to find a way to install Ruby without using package management. Google came back with the answer: Download the sources and build them. I found that yes, this does work, and it gave me an excellent opportunity to wander off and have a cup of tea, after which I managed successfully to install the CLI.

At  this point, you need to set about uploading the WAR file. There is some documentation, and it’s fairly intuitive. You have to point at the pivotal servers, log in, and create the application by running “cf push” on the WAR file (or by getting Jenkins to do this for you). You can also configure your services. The uploading machine was fairly well-connected to the Internet; it wasn’t a desktop. In my previous attempt to use Cloud Foundry on this application, the upload failed. Today I waited quite a long time for the upload, and it did work, although the application didn’t start.

To cut a long story short, I needed to:

1. Give the application more memory
2. Use the Grails Cloud Foundry plugin to set up database access
3. Configure a MySQL data source—why couldn’t Cloud Foundry just work this out?

While I was fixing this, the edit, commit, build, upload cycle was quite long (about half an hour). The Web GUI to Cloud Foundry gives you the ability to start, stop, and configure the application, but it discloses no useful diagnostics. For log files, you need to use the CLI. The logging was of fairly limited use, and at the end of it I ran into the Tomcat 6+: Infamous “SEVERE: Error listenerStart” message halfway through the application bootstrap. The support forums eventually led me to the way to retrieve Tomcat’s additional logs, which can help diagnose these errors. However, when I followed the instructions, I was met with a “Getting file contents…FAILED CFoundry::BadResponse: 500: Internal server error.” At that point, I went home.

By the way, the MySQL data source is provided by a third party, ClearDB, and is hosted somewhere on Amazon. Unless you re-code your application with SSL support, it is accessed insecurely. Over the weekend, I received an email from ClearDB, relayed by Pivotal, saying that I had exceeded my 5MB quota. Given the price of hard drives these days, 5MB does seem a bit miserly, but I whipped out my credit card and tried to buy some more. Turns out you can’t, or at least you have to go to ClearDB separately from Pivotal. So much for seamless integration. At this point I had had enough. Life really is too short.