XPDays Kiev: Continuous Delivery Pipeline With Jenkins, Artifactory And LiveRebel

XPDays in Kiev was a success! Great crowd, interesting talks and fruitful discussions, socializing included.

Embedded are the slides from by import continuous.delivery.* talk which was surprisingly well received and it seems a lot of people are looking for ways to release and deploy continuously, even in the large organizations.

There are numerous different tools that could be used to organize the continuous delivery pipeline. The pipeline that I've demonstrated was built with Jenkins, Artifactory (Pro) and LiveRebel.

I'm not super happy with the build pipeline plugin in Jenkins. In the talk I had to spend time explaining why the flow is organized the way it is and why the build pipeline plugin behaves the way it behaves. It is just buggy... I should find time to fix it.

Arifactory's Pro version was just primarily because of the fact that artifact promotion feature is a Pro version and some of the REST APIs that I wanted to use were under Pro license. There are alternatives, but with Artifactory it was just much easier to show the relevant stuff.

LiveRebel - well, there's no real alternative to it. The attendees were really impressed how easy it is to deploy via LiveRebel and oversee the target environment.

The cool part is that all the tools integrate well via the REST API (or CLI) and you can build a very flexible pipeline if this kind of integration is available. What I also realized is that although everything can be automated in the pipeline, it shouldn't get on your way still - you should be able to go into any step and re-execute if needed.

Another critical feature is the rollback functionality. It is not just that you select the previous version of the app if something goes wrong. There's more things you have to keep in mind when organizing the pipeline (I'm not revealing it now :P).

Enjoy the slides! :)

Do you really get your IDE?

This is bit like a philosophical post. Just some thoughts regarding our perception of developer tooling. 

First - a question. Which IDE do you use? Eclipse? NetBeans? IntelliJ IDEA? Visual Studio? Vim? Emacs? Yeah, really - Vim counts as an IDE as long as you can configure it to behave like one. Or maybe Sublime Text, a really awesome text editor?

Second question. Do you leverage the full power of your favourite IDE/editor?

Mostly, people will say that the just use the IDE and I haven't really seen many people leveraging the power of refactorings, shortcuts, other awesome IDE features that are out there. Why? Lazy? Careless? Neglectful?

I was reading a nice theory article The IDE Divide, which is already 8 years old. The point of the article is that it points out two extremes among the developers: language mavens and tooling mavens. Language mavens are those who case about the deepest nuances in the programming languages and don't really want to rely on tools (or just don't have time to explore the features of the tools). Tooling mavens are the ones who are obsessed with learning (and creating?) the tools and not spending as much time discovering the mysteries of language features. The article also mentioned that it is enormously hard to be both, the language maven and the tools maven at the same time, since the time for learning all this stuff is limited.

But generally, I think, knowing the properties language and runtime is more important as it is the produced code that will eventually run on the system. The tools are just used to create the programs. However, I rather consider myself a "tooling maven" type of developer. Not that I don't care about the languages, no. It is just the interest shift towards tools for me.

Despite the above, I noticed something (or it is just my perception). When a the new-born programmer starts, you will first try to reach the comfortable level of using the language. Once you're successful, there comes time for you to write the programs more effectively - faster, using shortcuts… click, click, click. This is where the tooling kicks in. Eventually, you start appreciate those nice features of your IDE that help you to write the code more effectively. The next stage is when you realize that actually you do not write code as much as you read it, and then you will start to appreciate the features that help you to navigate the code, analyze it, maybe refactor it. Language becomes a bit unimportant.

I was chatting with Jacek Laskowski one day and he asked an interesting question: "If you're given awesome tools/components/frameworks to work with, would you really care about which programming language to use?". Really good question. I wouldn't care, I guess. You will learn the language anyway. Or you will learn the tooling anyway once you're comfortable with the language of your choice, because normally you would like to be more effective (this is my perception of curious programmers, I hope you are a curious programmer).

What do you feel when a colleague next to you just moves around the project like a pro and finds everything he needs just in fractions of a second, and types with shortcuts creating new statements with just a few strokes? And then you try to type: 'p' 'u' 'b' 'l' 'i' 'c' '_' 's' 't' 'a' 't' 'i' 'c' '_' 'v' 'o' 'i' 'd' '_' 'm' 'a' 'i' 'l' [oooops! a typo!]. Frustrating…

It is every so often I was keeping myself back from screaming at my colleague "just Ctrl+Shift+E !!!!" while the team mate was looking for the class in the project tree the name of which he did not remember.

 Modern IDEs have revolutionized the way in which we are able to work with the code. Sadly, most programmers are held back by some mysterious myth that if you learn the tools too much you're doomed as a programmer as you start depending on those tools. Don't be held back by such fears! Go learn some tooling instead - it will save you some time later!

Do you really get classloaders?

JRebel team has produced a nice-looking PDF about Java classloaders. This should be a very nice read for everyone who wants to know more about Java platform fundamentals.

Maven support in IntelliJ IDEA (screencast)

The screencast demonstrates Maven integration features available in IntelliJ IDEA. Apache Camel maven archetype is used to generate the application. The main features are:

• Project setup automation and automatic dependency resolution
• Editor assistance and autocompletion
• Refactorings
• Dependency graph
• Lifecycle management using the tooling window

A great read on Scala

A great article has been published at ZeroTurnaround's blog: Scala 2013: A Pragmatic Guide to Scala Adoption in Your Java Organization. You can download it as a PDF (which looks really cool!). The nice part is that it includes an interview with Martin Odersky and quotes by Josh Suereth.

JavaZone 2012: Taming Java Agents

At JavaZone I've presented a talk called Taming Java Agents. It is not about the distributed computing or messaging as the title might imply. It is about the tools that exercise -javaagent JVM argument to hook into class loading process in order to perform some necessary evil to the bytecode. The tools can do amazing things to the application by rewriting the bytecode - performance & memory monitoring, class reloading, tracing, etc. I think this is an interesting topic in general - the tools do not exist for no reason.

Here are the slides and the video from the talk:

IDE Project Files In Version Control - Yes or No? Of Course, Not!

NB! This topic is mostly related to Java projects. The situation might be a little different in case of other technologies and IDEs.

Just recently I had some discussions with the clients who were claiming that they keep IDE project files in version control system hence they avoid any changes to those files. For reference, those are Eclipse generated .project and .classpath. From my point of view it is a bad practice by all means, however I usually prefer to collect some information on the topic before I say it loudly. So I asked from my G+ and Twitter followers, IDE project files in version control - Yes/No? Surprisingly, I've got quite a number of responses, so I decided to summarize the information as a blog post.

Basically, the answer to this question isn't binary, yes or no, but it also could be "yes, but... " or "no, if...". So there might be some argument why someone prefers one way or another.

The vast majority of the answers is "No, no, noooooooo!! Never!". And just a couple of answers were "yes" with some weak arguments of why someone should keep those files in version control.

Here are some yes answers first.

Bad stuff. Seems that the team isn't really competent with the tools. The project should be easy to setup and why on earth I should press 100500 buttons to setup a project. If this is the case, the first thing that team lead should do, is to simplify the project structure, setup and build process. No excuses there.

Yeah, yeah, I can already hear some arguments like "but we have a complex project" or "this is the way our setup is done". Bullsh*t!

OK, there is some optimization for the setup, IDE specific though. If all the team members use the same setup, it might even make sense for those who want to keep hands off the console.

Here's another one:

"Specific Eclipse plugins" - yeah! that's is actually an argument for not keeping the files in version control! Eclipse plugins usually modify .project files as they add a "nature" or any other project specific settings to the configuration. And actually, IntelliJ does the same, but (let me bash to tools a bit) IntelliJ can suggest the settings as you open a project from scratch. And with Eclipse, you have to do that manually. What if your colleague uses some awesome eclipse plugin that makes modifications to .project file, and you hate that plugin and do not want to install it, and the .project file is in version control? Here's just a simple example that you could see while exporting a project with existing project files:

This is so annoying to resolve this kind of problems. And all you want to do is just to open the project and proceed with your normal work.

Here's more:

Currently, I use IntelliJ and the rest of the team uses Eclipse. And I absolutely do not care if they put the project files into version control, because I will import it into my IDE in two clicks anyway. So the assumption that "absolutely yes, given that the team is working with the same IDEs" is wrong, and no viable argument here as well. 

However, here we have some much more interesting ideas:

Wow, this is really interesting one - "those that define formatting or source code". Indeed! In cases when you work on varions projects for different clients, the requirements and code styles might be quite different and it makes sense even to keep this kind of files in version control, so you can share it with the team and restore the code style settings if you lost them. Good point here!

However, you can probably see the vector of my post now.

So the point is, IDEs support Maven quite well, so why on earth I would need to keep the IDE settings in version control if we already have what we need: pom.xml. IntelliJ and NetBeans cope with that quite well, and Eclipse also, if you use JBoss Tools.

But what if I'm a Maven hater? (I really am). Here's an interesting conversation:

Oh sure, Gradle that is! Well, the IDE support isn't there yet. Luckily, STS provides a nice Gradle plugin for Eclipse, but the support for IntelliJ and NetBeans isn't quite there yet.

However, my claim is that Maven or Gradle isn't the prerequisite to avoid the project files in version control. The real prerequisite is the simple setup of you project and a clean structure, so that importing the project is just a couple of clicks. Then you can cope with any kind of project, even if it doesn't contain pom.xml.

And here's what most of them say about the topic:

Myself, I have tried both ways - keeping the project files in version control and not checking them in, and my take is that under no circumstances it is a good idea to check the project files in. Again, a much better solution is to keep your project structure clean and simple, which might be harder than to check the files into version control, but much more beneficial on the long run.

Thanks everyone for the input! Take care...

I'm speaking at JavaZone again this year. Hope to give an overview of javaagent and Instrumentation API with some referrals to the interesting use cases.

Con-FESS 2012, Day 2

I had great fun speaking at Con-FESS today. Here are my slides for the Why Doesn't Java Have Instant Turnaround talk. The internet connection was great so I even managed to make a JRebel Remoting demo with Jelastic PaaS.

After that, we had a great time investigating Griffon framework internals with Andres Almiray. Although it is quite tricky to update desktop applications on the fly, it seems it is still possible and Griffon will be able to leverage JRebel facilities.

Peter Niederwieser talked about Gradle in the enterprise. Gradle is just about to go 1.0 which is great. The tooling is getting better and there are very nice features in Gradle that simplify build procedure and build script implementation - true incremental builds, autowiring, dsl extensions, etc. What is really missing at the moment is the stable way to migrate Maven projects to Gradle (there are some scripts, but these aren't quite stable) and the native release plugin. It is quite interesting that Gradle has implemented its own dependency management solution and isn't based on Ivy any more.

Stefan Armbruster talked about using Grails with Neo4j graph database. It seems that it doesn't integrate that smoothly yet - some minor glitches still exist, but nevertheless it was used for a production application.

Why Doesn't Java Has Instant Turnaround - Con-FESS 2012

View more PowerPoint from Anton Arhipov

The video recording of the talk is available at Con-FESS website: http://2012.con-fess.com/sessions/-/details/139/Why-doesnt-Java-have-instant-turnaround

Con-FESS 2012, Day 1

Although the Con-FESS conference attendance doesn't seem to be very high, around 200 people, it still was a good idea to come here. Look at this:

Nice view, eh? :)

First, I've attended to keynote talk by Jürgen Höller, Enterprise Java in 2012 and Beyond - From Java EE 6 to Cloud Computing. The talk was about the current trends towards cloud platforms. Cloud is a hot topic for Java EE 7 these days. The term "cloud" spawns across quite different technologies, not just Java. For instance, alternative datastores are quite popular among quite providers as these allow for better scalability. As the implementations are so different it is quite hard to imagine any reasonable standardization in this area. And even for Java, the vendors agree just on some bits of the standard, like Servlets, for instance. To my mind, the diversity will only grow in the cloud era.

Also, it will be interesting to see how to tooling will evolve in some time. I'm quite sure that JBoss Tools and STS, as well as Google Eclipse plugin are aiming for this use case. With JRebel, it is now also possible to update application code in the cloud. I'm quite sure, in near future we will see some new tooling specially designed for the cloud apps but as the diversity is quite high, these tools will most likely be PaaS-specific.

Next, I've attended the talk about Groovy ecosystem by Andres Almiray. It was an overview of the frameworks and libraries build with Groovy and for Groovy: Grails, Griffon, Gaelyk, Caelyf, Gradle, Gant, Codenark, GMetrics, Easyb. Not that much into the details, but a good overview.

I learned a lot from Mark Struberg's talk about CDI and it really terrifies me how complex it might get with all the annotation-based programming model that CDI offers. Especially with the CDI extensions. Especially if people will start use this extensively. It is fine if people could stick with the libraries like CODI, Seam3, or Apache DeltaSpike, but the perspective that the annotations are scattered across the source code and the developer can't see the full view at once really frightens me.

Mark covered the standardization pitfalls as well as the gotchas when writing a CDI-based apps - very informative! He warned us from using @ApplicationScope for the beans as it isn't quite clear how this should work and the application containers can interpret this a bit differently, either for a whole EAR deployment or just for WAR deployment boundaries. It seems he had a lot more information to tell and he didn't fit everything in this talk, but we also had an interesting discussion afterwards. Here's a rage comic I came up with afterwards :)

Last, I attended the talk about systems integration of could-based services by Kai Wähner. The presentation covered the introduction to Apache Camel and how it was used to integrate with Amazon Web Services and Google App Engine.

The conference is hosted in Leogang, a small village not far from Salzburg, in Hotel Krallerhof. Surrounded with mountains, this place is amazing!

Slides: Mastering Java Bytecode

Mastering Java Bytecode - JAX.de 2012

IntelliJIDEA Tip: Configuration Profiles

How many plugins do you have installed in your IntelliJIDEA instance? How many of the plugins are actually active? Do you need all the plugins to be active at once? Thins might be a quite realistic scenario that you have a fairly large number of plugins activated but you need only a some of them to be active while working on some particular project. Unfortunately, there's no such thing in IntelliJ as "configuration profiles" out of the box, so that for each "profile" you could enable only specific features and thus drop some weight of your IDE while working on a specific task. However, the "profiles" can easily be emulated. In IntelliJ the settings are stored under USER_HOME/.IntelliJIDEAxx folder. There you will find all the settings, plugins, and other stuff related to your current IntelliJ installation. This directory is actually the current "profile" of your IntelliJ installation. In IDEA_HOME/bin/idea.properties file there is a number of properties that refer to the settings directory:

 
 idea.config.path=${user.home}/.IntelliJIdea/config
 idea.system.path=${user.home}/.IntelliJIdea/system
 idea.plugins.path=${user.home}/.IntelliJIdea/config/plugins
 idea.log.path=${user.home}/.IntelliJIdea/system/log

So in order to create a new profile you need to copy the settings directory from USER_HOME/.IntelliJIdea to USER_HOME/.MyIntelliJ and in the configuration file that points to the settings directory you need to change to pointers accordingly to:

 
 idea.config.path=${user.home}/.MyIntelliJ/config
 idea.system.path=${user.home}/.MyIntelliJ/system
 idea.plugins.path=${user.home}/.MyIntelliJ/config/plugins
 idea.log.path=${user.home}/.MyIntelliJ/system/log

Assume, I will keep the configuration in MyIntelliJ.properties file. Now, on the startup, IntelliJ just needs to know, which properties file to use. As the documentation says, we can either set up IDEA_PROPERTIES environment variable to specify custom location of this properties file, like

set IDEA_PROPERTIES=c:\config\MyIntelliJ.properties

or in unix:

export IDEA_PROPERTIES=/home/anton/MyIntelliJ.properties

In the environment variable isn't specified, the configuration file is searched according following sequence (first successful is used):
1. USER_HOME/
2. IDEA_HOME/bin

This way, it is pretty much possible to organize the profiles for IntelliJ, however, it would be cool to have such possibility out of the box via UI configuration.

33rd Degree

I'll be speaking at 33rd Degree conference in Krakow, Poland. Looking at the schedule there's plenty of interesting talks I'd like to attend myself - hopefully I will have some time for that.