The essential tools for software developer

As a software developer I have used a ton of different tools for different purposes. However, it got me thinking, if I really need a lot of tools and what are the essential tools for software developer and what tools are rather “nice to haves”.

Editor

First, we better be able to write code. Hence, the editor is absolutely required. For Java, I’ve used different IDEs, but mostly IntelliJ IDEA. NetBeans IDE is nice. Eclipse-based IDEs shine in one way or another. I usually prefer to install JBoss Developer Studio as it provides a solid support for Java EE projects.

For one-off editing, I still use Vim and can’t get rid of it. I’m not even using the full power of the editor. I’ve tried other awesome editors at different times: Notepad++, Crimson, Atom, Sublime Text, VS Code (which I do have installed on my machine). Still, those editors haven’t got much use by me for some unknown reason.

The cool thing about the editors is that there’s so much choice!

Version Control System

In 2004 I started at one financial company as a Java developer. The first task I was assigned to was something likes this: “We have an application running in production and it has a few bugs. We need to fix those, but we don’t have the source code. Please do something?”. A disaster. They didn’t use source control properly and managed to lose it.

Today, the scenario above is very unlikely. However, I still hear stories how people store the source code in folders with suffixes _1, _2, _3 on the network drive. Argh!

Which version control system would you pick? IMO, today Git is the dominant one. However, the way people often use Git (or any other distributed VCS) by avoiding branching and playing with feature flags, they probably would be just fine by using Subversion instead :)

Issue Tracker

An issue tracker is absolutely needed for the team to plan their work, tasks, issues, etc. Now the question is in the implementation. I’ve seen tasks being tracked in MS Excel, text files, sticky notes, and other modern tools like Jira, Mingle, PivotalTracker, YouTrack, ForBugz, Bugzilla, Mantis, Trello… the list is infinite!

The thing with issue trackers is that they should support the approach you take for running the project. For a product company there are quite a few reasons to track issues. First, obviously, you need to track your work, i.e. tasks. From the project management perspective, it’s nice to have the ability to track the tasks, i.e. have the project management facilities in the task tracker. Lots of teams are happy to have it all in one so it’s easier to get an overview of the progress, especially if the tool provides Kanban style boards.

Second, good products usually have users who are eager to submit questions or bug reports. For that, most of the teams are using some kind of helpdesk software. HelpScout was the one I have used at ZeroTurnaround and can recommend. Plus, some teams make use of forum software that also serves as a channel for communication. Also, some teams expose a public issue tracker so that the users would directly submit issues there.

The above means that the issue tracker you choose is better have the required integrations available if that’s a requirement.

Continuous Integration

You write code in the editor, you test the code on your machine, you push it to Git. Someone pulls the code down, tries to run it, and it doesn’t work. Sounds familiar? Yeah, reminds me the famous “Works on my machine” excuse. Automation is absolutely required for a healthy software project. Automated builds, automated tests, automated code style checks, etc.

Previously at ZeroTurnaround, I’ve seen that the CI server was a very critical piece of project infrastructure. If a source control system want offline, it wasn’t an end of the world to them. But if the build server was down, for many developers the work pretty much stalled. Now at JetBrains, I also see that their own product, TeamCity, is a very important service internally, with many teams relying on it. Dogfooding at its best!

Continuous integration was brought to the masses by eXtrame Programming practices. Today, CI is absolutely essential to any healthy software project. Years ago it all started with automating the test execution, building the artifacts, and providing the feedback quickly. Remember CruiseControl? Or Hudson, when it appeared? CI servers have been evolving the in the past years as the notion of Continuous Delivery appeared. Better visualisation of the process is required to cope with growing complexity. Also scaling the CI server has become an important aspect of the process.

With the latest trends in CI, the build servers are eagerly implementing the notion of build pipelines (build chains in TeamCity) which provide a good overview of the process. And also the trend of 2018, I think, is running the CI tasks in a Kubernetes cluster.

Artifact repository

The build server produces the artifacts. Plenty of them. Either the artifact is a final software package, or a reusable component, or a test execution report, etc. Where would you store those artifacts with all the metadata associated with it? Today, there isn’t much choice, actually.

JFrog’s Artifactory is the dominant solution for storing binaries and managing them. Plus, the final artifacts could be promoted directly to Bintray for distribution. Sonatype’s Nexus was originally the go to solution for Java/Maven projects and added support for some other technologies as well in the recent years. Apache Archiva and ProGet are the other options but that’s pretty much it.

The security aspect of software development becoming more and more critical. I think, automated security checks will become an absolute requirement for any software as well. The trend that has been ongoing for years now and binary repositories, such as Artifactory and Nexus, are actually integrating with the services that provide such vulnerability checks. So, don’t be that guy, use the binary repository!

Summary

I have listed 4 categories of tools that I think are essential to any software project:

• an editor,
• a version control system,
• CI server,
• and an artifact repository

I didn’t mention build tools, compilers, static code verifiers, etc, but those are the small tools that will be used by developers anyway. And if not, some of the tools I’ve mentioned can replace or provide such functionality as a bundled feature.

I will be speaking at JavaOne 2015

I'll be speaking at JavaOne this year again! This time I have 2 talks accepted:

CON3597 - Having Fun with Javassist. This is merely a live coding session where I demonstrate various uses of the Javassist library for Java bytecode manipulation. I've delivered this talk multiple times and every time it is different as it turns out quite interactive and attendees usually ask questions right in the middle of the talk so I have to adjust the content as I go. Usually it's quite fun, so I enjoy presenting this talk.

CON6699 - What's the Best IDE for Java EE? I'm not sure how this one turns out - it's so much to talk about and so little time. I'll be presenting this talk along with Max Rydahl Andersen and Adam Bien. This time we're focusing solely on Java EE. Basically - it's and overview of what's available for Java EE users in Eclipse, NetBeans IDE, and IntelliJ IDEA.

Both the talks can be found in the content catalogue for JavaOne.

IDE Fanboys

I just leave it here...

JavaOne 2013 - Apples and Oranges: The Highlights of Eclipse, IntelliJ IDEA, and NetBeans

IDE Project Files In Version Control - Yes or No? Of Course, Not!

NB! This topic is mostly related to Java projects. The situation might be a little different in case of other technologies and IDEs.

Just recently I had some discussions with the clients who were claiming that they keep IDE project files in version control system hence they avoid any changes to those files. For reference, those are Eclipse generated .project and .classpath. From my point of view it is a bad practice by all means, however I usually prefer to collect some information on the topic before I say it loudly. So I asked from my G+ and Twitter followers, IDE project files in version control - Yes/No? Surprisingly, I've got quite a number of responses, so I decided to summarize the information as a blog post.

Basically, the answer to this question isn't binary, yes or no, but it also could be "yes, but... " or "no, if...". So there might be some argument why someone prefers one way or another.

The vast majority of the answers is "No, no, noooooooo!! Never!". And just a couple of answers were "yes" with some weak arguments of why someone should keep those files in version control.

Here are some yes answers first.

Bad stuff. Seems that the team isn't really competent with the tools. The project should be easy to setup and why on earth I should press 100500 buttons to setup a project. If this is the case, the first thing that team lead should do, is to simplify the project structure, setup and build process. No excuses there.

Yeah, yeah, I can already hear some arguments like "but we have a complex project" or "this is the way our setup is done". Bullsh*t!

OK, there is some optimization for the setup, IDE specific though. If all the team members use the same setup, it might even make sense for those who want to keep hands off the console.

Here's another one:

"Specific Eclipse plugins" - yeah! that's is actually an argument for not keeping the files in version control! Eclipse plugins usually modify .project files as they add a "nature" or any other project specific settings to the configuration. And actually, IntelliJ does the same, but (let me bash to tools a bit) IntelliJ can suggest the settings as you open a project from scratch. And with Eclipse, you have to do that manually. What if your colleague uses some awesome eclipse plugin that makes modifications to .project file, and you hate that plugin and do not want to install it, and the .project file is in version control? Here's just a simple example that you could see while exporting a project with existing project files:

This is so annoying to resolve this kind of problems. And all you want to do is just to open the project and proceed with your normal work.

Here's more:

Currently, I use IntelliJ and the rest of the team uses Eclipse. And I absolutely do not care if they put the project files into version control, because I will import it into my IDE in two clicks anyway. So the assumption that "absolutely yes, given that the team is working with the same IDEs" is wrong, and no viable argument here as well. 

However, here we have some much more interesting ideas:

Wow, this is really interesting one - "those that define formatting or source code". Indeed! In cases when you work on varions projects for different clients, the requirements and code styles might be quite different and it makes sense even to keep this kind of files in version control, so you can share it with the team and restore the code style settings if you lost them. Good point here!

However, you can probably see the vector of my post now.

So the point is, IDEs support Maven quite well, so why on earth I would need to keep the IDE settings in version control if we already have what we need: pom.xml. IntelliJ and NetBeans cope with that quite well, and Eclipse also, if you use JBoss Tools.

But what if I'm a Maven hater? (I really am). Here's an interesting conversation:

Oh sure, Gradle that is! Well, the IDE support isn't there yet. Luckily, STS provides a nice Gradle plugin for Eclipse, but the support for IntelliJ and NetBeans isn't quite there yet.

However, my claim is that Maven or Gradle isn't the prerequisite to avoid the project files in version control. The real prerequisite is the simple setup of you project and a clean structure, so that importing the project is just a couple of clicks. Then you can cope with any kind of project, even if it doesn't contain pom.xml.

And here's what most of them say about the topic:

Myself, I have tried both ways - keeping the project files in version control and not checking them in, and my take is that under no circumstances it is a good idea to check the project files in. Again, a much better solution is to keep your project structure clean and simple, which might be harder than to check the files into version control, but much more beneficial on the long run.

Thanks everyone for the input! Take care...

Using JRebel with MyEclipse Blue and WebSphere

Just recently ZeroTurnaround and Genuitec announce JRebel for MyEclipse. Here's what it looks like in combination with WebSphere application server. It is quite easy to install and configure.

Eclipse, JBoss, EAR, JRebel

Erlybird: the Erlang plug-in for Netbeans

Erlybird looks very fine while working in Netbeans. Unfortunately Erlide (the Erlang plug-in for eclipse) wasn't that smooth.



The strange thing about the plug-in settings is that it shows itself like if it would be a JRuby plug-in:



What could be done better, I think, is when I run a script, instead of the erl interpreter I could choose a function to be used to start the script. So it could look like any other program I'm running in IDE.

The ViewPart Should Be Equal

I was preparing an example of a groovy view extension for my GSoC project tonight. First, I created a groovy class that extends org.eclipse.ui.part.ViewPart and added the required methods to that class:

class MyView extends ViewPart {
  public void createPartControl(Composite composite) {
  }
  public void setFocus(){
  }
}

def view = new MyView();
view

The first disappointment was that I couldn't write the method declarations in a groovy way, using def, because it was yelling that the abstract methods should be implemented. Thus I conclude that I cannot implement the abstract methods using def.

So far, so good... OK, when I executed the above script I got a nice stack trace, saying that "A part must be equal to itself", and/or "A part must have a meaningful equals method". This was a message from org.eclipse.ui.internal.PartTester#testWorkbenchPart(), that checks the brand new part for some strange things, like part.equals(new Integer(32)). Why 32, can anyone tell!?

So I implemented a goofy equals() for my script object, just to make it work. Here's the example code: view.groovy.

GSoC 2007: Equator Reached!

The midterm evaluation is complete and just over a month is left to do the project, and the deadline is approaching fast.

The last post about the facade implementation for each extension was just a concept showing what could be done for mapping the extension method calls to the script. But implementing a dedicated facade for each extension is not feasible just because there is an unlimited number of the extension types available.

As the previous post said, a proxy class solution could solve the problem so that I shouldn't write a dedicated facade for each extension. That said, we have a brand new solution in place: the ScriptExtensionProxy class, which implements the following interfaces:

1. IExecutableExtension required for extension initial setup in ConfigurationElement#createExecutableExtension() calling IExecutableExtension#setInitializationData()


2. IExecutableExtensionFactory required for proxy class construction using IExecutableExtensionFactory#create() call in ConfigurationElement#createExecutableExtension()


3. InvocationHandler required to delegate the extension method calls to the script in InvocationHandler#invoke() method.

The example script still works the same way as previously, but no facade class required any more.

The declaration of an extension is changed a little bit, as I have to define a proxy class, an extension interface class and a script file in the class attribute in plugin.xml:

<action class="org.eclipse.soc.yummy.core.ScriptExtensionProxy: \
   org.eclipse.ui.IWorkbenchWindowActionDelegate/action.groovy"
               icon="icons/sample.gif"
               id="ScriptAction"
               label="&Sample Action"
               menubarPath="sampleMenu/sampleGroup"
               toolbarPath="sampleGroup"
               tooltip="Hello, Eclipse world">
         </action>

It looks a little messy though...

Overall, providing the scripting support for developing plug-ins in Groovy requires only three classes at the moment: a proxy, a script execution engine, and a helper class for initialization code that is actually used in the proxy class only. LOC = 355.


Next, I will have to make many examples to see what is missing in ScriptExecutor and to find out which workbench objects should I expose to the script. I still need to refer to Dash for this purpose.

Eclipse plug-ins and JSR-223

Recently I was trying to bind the eclipse plug-in world with JSR-223 and Groovy via javax.script API for my GSoC project. Here's how it turned out.

First I need to have a class that deals with all that javax.script stuff: let there be ScriptingExecutor. Its functionality isn't very advanced - it just delegates the calls to the scripting API and maintains several fields for holding the state of the script. STOP! Why would I need to keep track of the state at all? The problem is that not all the scripting languages with the same semantics/features.

Let's see the example: an action facade and a script that implements this action. I'm using the Expando object to create a class with the same methods as for the action: run(IAction action), selectionChanged(IAction action, ISelection selection), dispose(). The reason why i would like to pass the new object back to the plug-in is that when a selection is changes I would like to save the reference for further processing in the run(IAction) method. The script below has a visibility problem and cannot use def selection to save the state.

 def selection

 def run(action){
    println "selection = ${selection}" // this won't work!!!
 }

 def selection(action, selection){
   this.selection = selection
 }

The Groovy User Manual says:

Executing this code you get an exception talking about a missing property or field.
The only things the method has access to are:
* the binding,
* attributes defined by the base class, and
* the dynamic properties defined by the MetaClass (explanations for these will follow).

So I decided to use Expando as a base class for saving the state. Simple.

Next, there are 2 options of delegating the plug-in extension to the script object:

1. Using the invokeMethod/invokeFunction methods of the javax.script.Invocable interfaces. These can be used just for simply calling the functions from the evaluated script. or...


2. Using getInterface(Class clazz) to create an object that would implement the required interface, of instance IActionDelegate.

I'm not sure which option is really better so I'm using the first one for now.

Next, I should try to implement several facades and scripts for the extensions, so that I could see the common parts and unify them in one. Implementing a facade for every single extension does not seem to be very relevant, perhaps the getInterface(Class clazz) method will help to skip that... Or I will need some sort of Proxy solution to skip all the facades.

Also I'm referencing Eclipse Monkey project as an inspiration. The reason is that for smoothing the script development a set of predefined variables could be very handy, so the user could use them out of the box. And that is what Eclipse Monkey does: it exposes some internal object into the script context so that a script contributor could use them.

The Summer of Code goes on

I feel the guilt because of not posing about the GSoC event so far. Perhaps should start blogging more about what I am doing at all.

For the project I have 3 goals now, sorted by priority:

1. Provide the ability to write the plug-ins in a scripting language.
Basically it means that a new plug-in could be implemented in any scripting language, supported by JSR-223. My mentor, Wayne, purposed a tricky solution to bind a script to a popup action using a syntax feature for the action definition in plugin.xml:

<action
  class="scriptexecutor.actions.ExecuteScriptAction:my.script"
  icon="icons/sample.gif"
  id="scriptexecutor.actions.SampleAction"
  label="&Sample Action"
  menubarPath="sampleMenu/sampleGroup"
  toolbarPath="sampleGroup"
  tooltip="Hello, Eclipse world"/>   

An action class ExecuteScriptAction will be provided by the supporting plug-in, and my.script is something that is implemented by the new plug-in developer.
ExecuteScriptAction will act as a facade to the scripting world, setting the required variables to the script context and running the script itself, using javax.scripting API.

public class ExecuteScriptAction 
  implements IWorkbenchWindowActionDelegate, 
  IExecutableExtension {

  private URL script;

  ...

  @Override
  public void setInitializationData(IConfigurationElement config, 
    String propertyName, Object data) 
      throws CoreException {
    Bundle bundle=Platform.getBundle(config.
           getContributor().getName());
    Enumeration entries = bundle.findEntries("/", 
           data.toString(), true);
    if (entries == null) return; // The script wasn't found.
    URL entry = null;
    if (entries.hasMoreElements()) entry = 
          (URL)entries.nextElement();
    this.script = entry;
  }

  ...
}

So basically, setInitializationData() sets the parameter to the action class and after that we can do whatever we would like to with the script file when IActionDelegate#run() is called for this class. Most probably I will have to pass a pointer to the current workbench to the script context, like Eclipse Monkey does.

2. Enable custom processing for plugin.xml, to be able test the new plug-ins without starting the 2nd eclipse instance.
This was actually the initial idea, but it seems that enabling scripting for the bundled plug-ins is more valuable. Probably it is better to bundle this functionality as a separate plug-in even. Processing plugin.xml will involve parsing the configuration and registering the extensions to the same workbench where the brand new plug-in is being developed.

IExtensionRegistry registry = Platform.getExtensionRegistry();
IExtensionPoint point = registry.getExtensionPoint
               (EXTENSION_POINT_ID);
IExtension[] extensions = point.getExtensions();

for (IExtension element : extensions) {
   // register the extension
}

3. The console, for providing the interactive (re)loading of the plug-in. I'm not very sure yet, will it be a Apache Derby ij style console, or will it be a set of controls, but definitely it is a good feature to implement to smooth plug-in prototyping.

Now, after a good vocation in Croatia, will have to speed up things a bit to be in time with the implementation.

Eclipse Europa is coming!

An annual release of Eclipse is coming soon! The release includes 21 projects.Eclipse is the leading platform for tools I think, but sometimes it seems too heavyweight to me. Lately I tried to download all the Eclipse projects and while running the Eclipse instance I was getting OutOfMemoryError all the time. It seems that for every programming purpose I should have a separate Eclipse instance to avoid the memory problems: one instance for JEE development, one - for BIRT reporting, one - with CDT installed. In other words keep the configuration of current instance minimal.
Here's another interesting blog entry about Eclipse platform at Application Platform Strategies Blog

JavaOne - Day 1

General Session. We've been presented a new hype technology from Sun - JavaFX script, which should bring the UIs to the next level. Also some frameworks were presented that should make the world yet better - JMaki & Phobos. Also Netbeans 6 was shown with the support of JRuby language editing/debugging.

I attended a session called Cool Things You Can Do with the Groovy Dynamic Language where Guillaume Alléon and Dierk König showd us the power of dynamic object manipulations with groovy language. That was really fun and a pleasure to listen and see. I'd suggested this session to anyone.

At Sun SPOT in Action: 3D, Virtual Reality, and Gaming I met the Sun SPOT device the 1st time live :) The talk was about the virtual reality gaming in general and what is required to archive the responsiveness of the game controls.

Easy Deployment Is Finally Here was a bit disappointing, as I expected a little more than just getjava.exe description.

Finally, I met my GSoC mentor, Wayne. We had a fruitful conversation on the GSoC project, so now I think I know where to start! :)

JSR 223: Scripting for Java

As Charles Nutter suggested, I decided to try javax.scripting API. The Java 6 Revealed book has a nice chapter on JSR 223 and also an example of Pnuts integration with JVM.
I have downloaded the other engines from java.net. It seems that for my SOC project it is reasonable to use Java 6. Virtually it will be possible to write plug-ins in any scripting language for which the engine is available. We could even use combination of many languages, just selecting the relevant engine by name or file extension:

    ScriptEngineManager m = new ScriptEngineManager();
    ScriptEngine groovyEngine = m.getEngineByName("groovy");
    ScriptEngine jrubyEngine = m.getEngineByName("jruby");

The full list of the scripting engines is available at java.net

Apache Derby with Eclipse

I sticked with Apache Derby for simple desktop application development. Derby (or JavaDB) is becoming part of the JDK, so it might be a good long term bet. The Eclipse plug-in appeared quite useful in this case.

The functionality of the plug-in makes the core jar files available in the project. One could run the ij queries just inside Eclipse console and execute custom scripts using a context menu item available via riht-button-click on the target file.