I’ve continued work on my Vaadin example project and am trying to make it reusable for new projects. There’s already some code for the Vaadin on Google AppEngine project on GitHub, but for now it’s just a “Hello World” type application. Before I continued, I wanted to see how performance is affected by the different features I’m using in this project. Turns out there is a fairly major penalty to be paid on the loading request, but once loaded, applications tend to not be impacted that much.
Before we get started, a small disclaimer: I wouldn’t want to publish this experiment in a peer-reviewed magazine, but I did my best to create fairly reproducible experiments. I conducted all experiments several times to avoid one-off peaks in the results. At the very least, they should give a good general hint on what might be slowing down an application on Google AppEngine.
For this test, I created the most basic Vaadin application that simply shows a “Hello World” label. These are the variants I tested:
- A Vaadin only version, no Spring at all.
- A Vaadin only version in which I extracted the Vaadin resources (in the VAADIN folder). This is a suggested practice as otherwise all requests for theme resources have to go through the Vaadin servlet.
- An application that gets its message from a simple Spring bean. Everything is configured via a Spring xml config file.
- A similar application, but now the bean is injected via annotations and there is no direct access to the Spring application context in the Vaadin application. I explained this technique for obtaining Spring beans in a non-Spring-managed object earlier.
- Exactly the same application as above, only now I wrapped a Spring Security filter around the entire app. There was no actually user involved as I only used anonymous access. My Vaadin Spring Security setup is explained in this post.
- Last but not least, I also tried the very first application with all jar libraries needed by the last application. About halfway through I got the impression that performance was directly proportional to the amount of jar files. So I had to try this out.
As far as response times when the application is fully loaded and active, I noticed virtually no difference between any of the scenarios. The very first version seemed a little slower, but given the number of times I ran the experiments I’m not going to draw any conclusions. The example only uses one stylesheet and the loading indicator JPG. So I didn’t expect to see a huge difference anyway.
I was happy with this result. For “normal” use of the application, you can use all those fancy Spring features. It won’t perform worse than on any other application server.
However, when I measured the CPU load for loading requests, the story was entirely different. I measured the following times (average of only 3 runs, those take a long while to do):
I think that chart speaks for itself. Clearly, the more stuff I enabled, the longer the loading request became. The final one takes over 7 seconds, which is worrisome. For a popular application this won’t matter, but a lot of users might run into it when the application is becoming popular.
Also important: it seems it doesn’t matter whether superfluous JAR files are present in the lib directory or not. If they are not used, they are not loaded and do not impact the loading request time. So my initial plan to weed through all the dependencies Maven pulled in and remove the ones that weren’t needed, is useless.
I tried looking for other solutions to speed up the loading request, but it seems there are none except removing libraries (and ease of development). With the recent announcement of App Engine integration in Spring, I expected to find more information on this, but it seems there isn’t much, apart from a few Roo tutorials. Maybe later?