OpenJDK and XKCD: Random number

OpenJDK and XKCD: Random number

A couple of days ago a our JPoint hackathon we discussed building (Adopt) OpenJDK. After finding out a better way to build OpenJDK on Windows (read it here), I’ve made my first improvement to OpenJDK.

This is a famous XKCD cartoon I wanted to implement:
getRandomNumber() { return 4; } // chosen by fair dice roll, guaranteed to be random

Where do we get started? Actually it turned out it is fairly easy to make this ‘improvement’. We just need to find the correct source file and do a build as described here.

Instead of the HotSpot core (which is all written in C) the JDK libraries are just classes/Java files. If you want to look at these Java files you’ll need to browse to [openjdk]/jdk/src/share/classes. There you’ll find familiar directories/packages like “java.*”, “javax.*” and even “sun.*”.

The file we need to change to improve Random is of course: [openjdk]/jdk/src/share/classes/java/util/Random.java

Now we browse until we find:

/**
 * Returns the next pseudorandom, uniformly distributed {@code int}
 * value from this random number generator's sequence. The general
 * contract of {@code nextInt} is that one {@code int} value is
 * pseudorandomly generated and returned. All 2<sup>32</sup> possible
 * {@code int} values are produced with (approximately) equal probability.
 *
 * <p>The method {@code nextInt} is implemented by class {@code Random}
 * as if by:
 *  <pre> {@code
 * public int nextInt() {
 *   return next(32);
 * }}</pre>
 *
 * @return the next pseudorandom, uniformly distributed {@code int}
 *         value from this random number generator's sequence
 */
public int nextInt() {
    return next(32);
}

Now we need fix the obvious error and turn it into:

/**
 * Returns guaranteed random number, rolled by nerdy DnD d20 dice.
 *
 * http://en.wikipedia.org/wiki/D20_System
 *
 * @return guaranteed random number
 */
public int nextInt() {
    return 14;
}

And after the build I pointed Eclipse to my newly generated JDK: [openjdk]/build/windows-x86_64-normal-server-release/images/j2sdk-image.
Next I run the following code:

import java.util.Random;

public class Test {

	public static void main(final String[] args) {
		final Random rnd = new Random();
		for(int i = 0; i < 10; i++) {
			System.out.println(rnd.nextInt());
		}
	}
}

Result:

14
14
14
14
14
14
14
14
14
14

Serious note:
This is absolutely not a valid patch but it really shows how easy it is to modify the JDK itself! There is a lot of low hanging fruit in the JDK, from missing unit tests, to unused imports to classes that don’t use generics yet. For more things to hack on, please read: https://java.net/projects/adoptopenjdk/pages/WhatToWorkOnForOpenJDK!


Building OpenJDK on Windows

Building OpenJDK on Windows

Tomorrow I’ll be enjoying an OpenJDK hack session with Martijn Verburg (aka The Diabolical Developer). To prepare for this session he told us to follow the AdoptOpenJDK build instructions.

Most cool developers today seem to be using OS/X, but some of us are stuck on Windows laptops. I actually choose to stick with Windows 7 because every single client I’ve worked for has Windows workstations and only require the application to run on Windows. But anyway Martijn said: “Getting OpenJDK to build on Linux/Mac would be easy, Windows can be dicey”

With a bit of Googling and some small problems I’ve got it working just fine on my Windows 7 (64 bit). Most information I got was from this write-up, but I encountered some problems and could skip some steps I didn’t need.

All the tools mentioned are free, but you will have to install some Microsoft Visual C++ packages to compile (which most Java programmers try to avoid).

Windows SDK for Windows 7.1

Instead of the blogpost above mentioning this step second, I recommend doing it first. If you install VisualC++ 2010 Express first this step might fail with some weird error. The solution? Uninstall VisualC++ 2010 Express and first install Windows SDK for Windows 7.1.

So go here and install: http://www.microsoft.com/download/en/details.aspx?id=8279

Microsoft VisualC++ 2010 Express

Next install VisualC++ 2010 Express (beware, Microsoft tries to install another version, pick 2010 Express):
http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express
And next Windows Imaging Component (64-bit):
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=1385

Cygwin (64 bit)

For some reason I ran into problems early on using the 64 bit Cygwin, so I decided to install the 64 bit version as well. This worked, so I recommend doing this.
During the installation you’ll need to add some development packages:

  • binutils
  • cpio
  • diffutils
  • file
  • gawk
  • gcc-core
  • m4
  • unzip
  • zip

I might have forgotten one or two, this will probably popup during the ‘configure’ step below. If you find a missing package, please tell me and I’ll update the post.

make

The version of ‘make’ that is packaged with Cygwin doesn’t work with OpenJDK. Instead we need to download the source from http://ftp.gnu.org/gnu/make/. I picked version 3.82 (this one is mentioned on the OpenJDK page). I downloaded and unzipped the source code here: C:\Projects\OpenJDK\make-3-82

To compile, fire up Cygwin and type:

$ cd /cygdrive/c/Projects/OpenJDK/make-3.82
$ ./configure
$ make

Now you could take the generated ‘make.exe’ and place it into the Cygwin bin directory, but this isn’t needed.

freetype

Next we need to have Freetype, this step is done exacly like descriped here. Only do the ‘Freetype’ chapter generating the lib and dll.

Once you’ve generated the lib+dll, make a new directory (I used C:\Projects\OpenJDK\freetype). Add the ‘include’ directory from the freetype source code used in the step before. Also create a ‘lib’ directory and place the generated lib and dll in it.

Hg/Ant/JDK7

The other article mentions you’ll need to install TortoiseHg, Apache Ant and a current JDK 7 (as bootstrap). I didn’t have to do this because I already had all three installed. But please go ahead, they are probably needed:

Getting OpenJDK sources

The other article mentions a lot of PATH requirements that need to be set. I didn’t encounter this at all because I configured my build in another way. First we’ll need to get the OpenJDK source code.

$ cd /cygdrive/C/Projects/OpenJDK/
$ mkdir jdk8_tl
$ hg clone http://hg.openjdk.java.net/jdk8/tl jdk8_tl
$ cd jdk8_tl
$ ./get_source.sh

This will create a new directory: /cygdrive/C/Projects/OpenJDK/jdk8_tl with all the sources!

Configure

Now we need to configure a build for OpenJDK. Instead of just calling ‘./configure’ we’ll need to add a few extra options, we’ll need to point it to the correct MAKE directory and we need to include our custom build freetype.

./configure MAKE=/cygdrive/C/Projects/OpenJDK/make-3.82/make.exe with_freetype=/cygdrive/C/Projects/OpenJDK/freetype

This will throw a lot of warnings, but in the end it should print something like this:

* OpenJDK target: OS: windows, CPU architecture: x86, address length: 64

Tools summary:
* Environment:    cygwin version 1.7.25(0.270/5/3) (root at /cygdrive/c/Development/cygwin64)
* Boot JDK:       java version "1.7.0_40"  Java(TM) SE Runtime Environment (build 1.7.0_40-b43)  Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)  (at /cygdrive/c/progra~1/java/jdk17~1.0_4)
* C Compiler:     Microsoft CL.EXE version 16.00.30319.01 (at /cygdrive/c/progra~2/micros~1.0/vc/bin/amd64/cl)
* C++ Compiler:   Microsoft CL.EXE version 16.00.30319.01 (at /cygdrive/c/progra~2/micros~1.0/vc/bin/amd64/cl)

Build performance summary:
* Cores to use:   7
* Memory limit:   16373 MB
* ccache status:  not available for your system

After the configure step you should have a new build directory added.
Mine was located at: /cygdrive/C/Projects/OpenJDK/jdk8_tl/build/windows-x86_64-normal-server-release

Build it

The next thing I did was to build everything, this can take anywhere from 10 to 40 minutes, my build took 20 minutes.

$ cd /cygdrive/C/Projects/OpenJDK/jdk8_tl
$ /cygdrive/C/Projects/OpenJDK/make-3.82/make.exe clean images

After a long wait and a lot of warnings and messages it says it created your very own JDK 8 build. Time to give it a try:

$ cd /cygdrive/C/Projects/OpenJDK/jdk8_tl
$ cd build/windows-x86_64-normal-server-release/images/j2sdk-image
$ bin/java -version

OpenJDK build completed

It worked!

Like I said before, I encountered some problems and errors along the way. Most involved the wrong Cygwin (32 bit), this caused bash to crash (STATUS_ACCESS_VIOLATION) during the configure phase. Also I ran into a problem installing ‘Microsoft SDK for Windows 7.1’ which required me to first uninstall Microsoft VisualC++ 2010. Another problem was not installing ‘diff’ in Cygwin (diffutils) so the build found some other diff.exe (from Git?) which gave differences during the actual build causing it to stop.

I might have forgotten to write down some step, if you encounter any problem (and solve them) please tell me so I can update this post!

Todo

One thing I should add is ‘ccache’. This tool greatly improves the build speed because it caches all unchanged classes/files. It was in Cygwin (32 bit) but it is missing in Cygwin (64 bit)… I’ll have to compile and install it myself. This is one thing I haven’t tried yet, but I probably should do!

Update: I’ve tried compiling and using ccache-3.1.9, but this broke the build. I might try different versions but for now I’ll just skip ‘clean’-ing altogether :-)


To be or not to be... Agile!

To be or not to be... Agile!

In my career I’ve seen a lot of misconceptions about Agile. And I’d like to do a step back and explain what being Agile means to me.

We ‘do’ Agile

This is the biggest misconception of all, and the main reason for my blogpost. Again and again I’m hearing people/companies say: “We do agile”. And this is just wrong… Agile isn’t some system you ‘do’. It isn’t a selection of things you can learn from a book. It is just a mindset and all the other stuff are consequences, let me explain that!

The meaning of ‘Agile’

If you look up the word Agile in the dictionary this is what it comes up with:

ag·ile adjective \ˈa-jəl, -ˌjī(-ə)l\
able to move quickly and easily
quick, smart, and clever

Source: http://www.merriam-webster.com/dictionary/agile

Agile means you’ll move quickly and easily. That is it basically, everything you do when you are agile boils down to:
Start quickly, get something done quickly, expect changes in movement and go with it easily.

For example, in software, expect every bit of code to change (more than once). This will automatically make your code easy to read, extend and refactor. Not only code will change, the requirements will change. Nobody knows what a system will look like in two/three years (those who say they do are wrong), we might have a good idea what to make at this movement, but once you get started people will want other things… this always happens. Stay open to these changes in your movement, be agile!

Agile, Scrum, we need to do stand ups!

Ah, the stand ups, you’ve been reading a book and it told you that you need stand ups? This is a requirement to be ‘agile’? WRONG. Like I said before, it is all just a consequence. Why do we do these stand ups? Because you need to know what everybody around you is doing at this moment. In waterfall projects you already know what people would be doing weeks (even months) in advance. But if you embrace the agile mindset and accept all these changes you might not even know what you’re doing tomorrow. The more you are open to changes and quickly move between tasks, the more often you’ll need to update your colleagues. That is why some projects do stand ups, a consequence of being open for changes.

We use whiteboards and post-its, we do agile!

If your design and the requirements don’t change, write everything down with a pen, or maybe put it in some Software Architect solution (Rational anybody?) or just inscribe it in a stone tablet. Instead, if you are agile and move quickly, designs and requirements change. You don’t want to go into a meeting with a printed piece of paper which has tasks or designs etc, you’d rather want to have the design on a whiteboard. This will allow you to change it during the meeting instead of re-printing. Whiteboards are perfect to erase and redraw shapes and lines! It is just the best tool for somebody who is agile. Except for one problem, if you write down text and you have to move it… rewriting takes a long time. To counter this we use post-its: write once, stick everywhere! It complements the whiteboard very well.

So do we need to use a whiteboard and post-its to ‘be agile’? NO… Whiteboard and post-its are a consequence of being agile, it is the best tool for the job as far as I know.

Retrospective!

If you don’t try new things and you don’t want to change the way you are working (being not agile) you don’t need to evaluate and come up with changes, obviously. But if you are agile, open for change, you need to have meetings on how you can change. As a consequence the retrospective arrives. This meeting is one of the most important meetings if you are agile; you talk about what can be improved in your process: What can make us more agile? What is holding us back?

No agile rules…

I don’t think agile needs a set of rules, you just need people to explain the basic concept of the mindset. The other things most organizations treat as ‘agile rules’ can be considered best practices because they make sense if you are agile; but please don’t just follow these rules.

Learn from others, embrace the agile mindset and come up with your own system… that works for you!


Congratulations to Devoxx4Kids!

Congratulations to Devoxx4Kids!

If you haven’t heard about Devoxx4Kids, first read the following post and watch the video: here!

The people behind Java (Oracle) have recognized the great potential behind Devoxx4Kids and decided to award the ‘concept’ with a Duke’s Choice award!
This was presented to Stephan Janssen (the original mastermind behind the concept) at JavaOne this year. Because I filmed the original event and also our Dutch event in Amsterdam I was asked to make a very short impression video which was shown at the JavaOne Community Keynote.

You can see it here (including a great shout-out to me and my daughter, the ‘mascotte’) here:
(from 10:17 up to 14:55)


Episode #1

Episode #1

Lately I’ve been having a lot of blog ideas, but I never take the time to write it all down in a blogpost.
That’s why I’m now going to try to combine thing I love:

  • Programming
  • Technology
  • Filming/video editing

Here is the first episode of Roy on programming:

Item 1: Oculus Rift
- http://www.oculusvr.com/
- http://www.reddit.com/r/oculus/

Item 2: Rich Hickey
- http://www.infoq.com/presentations/Simple-Made-Easy
- http://www.infoq.com/presentations/Design-Composition-Performance

Item 3: Separation of Concerns
- http://en.wikipedia.org/wiki/Separation_of_concerns

Music by:
- Nasty Goreng: https://soundcloud.com/nastygoreng

Please let me know what you think about it.


No more DSL for acceptance/integration testing

No more DSL for acceptance/integration testing

Everybody seems to be using Fitnesse or Cucumber for integration testing these days. The main idea is that users (testers/business analists and programmers) can write what your program should do in plain text. The programmer then creates a small layer of code which gets interpreted and executes your actual program to see if the specified requirements are met.

Fitnesse example

For example a (hypothetical) piece of code in Fitnesse as it could have been used in the Port of Rotterdam:

!3 Anchorage tests

!|given a ship                                |
|Id  |Name       |Category|CallSign|Shipnumber|
|1001|Maersk Bali|TANKER  |D6DZ8   |7420211   |

|script                                                              |
|When a ship called|MAERSK BALI|reaches the port                     |
|And it reports anchor down                                          |
|ensure            |the ship is shown in the list of anchored vessels|
|check             |the list of anchored vessels now has size|1      |

This is something the testers and product owners can read and understand. Behind this layer of text Fitnesse will try to execute pieces of code. This is left as an exercise for the programmers to complete. They’ll need to write something like:

GivenAShip:
void setId(String id);
void setName(String name);
void setCategory(String category);
void setCallSign(String callSign);
void setShipnumber(String shipnumber);
void execute(); //Stores the ship in the database

AnchorageScenarios:
void whenAShipCalledReachedThePort(String shipName);
void andItReportsAnchorDown();
boolean theShipIsShownInTheListOfAnchoredVessels();
int theListOfAnchoredVesselsNowHasSize();

This is where the problems start, there is a lot of room for typos. Which will send you all over the codebase constantly translating Fitnesse text like “and it reports anchor down” to “andItReportsAnchorDown” during your searches.

Also people always say, because it is ‘plain text’ the testers, analists and product owners can instantly start writing tests. But this just isn’t true… It isn’t just writing down tests/requirements, it is programming with a DSL. Normal human text isn’t good enough, it still needs some DSL structure. They’ll have to learn to program in the Fitnesse DSL language. In almost all project I’ve worked on, it is the programmers writing and maintaining tests, people from the business stay far away from Fitnesse.

If you want to refactor the scenarios/fixtures in Fitnesse you’re in for a treat. Everything is bound by name only, and renaming code for example won’t rename the DSL! Using Fitnesse ‘sounds’ very agile, but it doesn’t really make the code agile at all. Most test code I’ve worked on is fragile, very ridged and not easy to refactor. Why not take advantage of that IDE and statically typed language if the programmers are writing and maintaining the tests?

Using code instead

Today we had a “ShipIt”-day (nautical-pun intended) at work, where we’re free to work on any project for 24 hours straight and we present our findings afterwards. This boosts the creative ideas and in the long run efficiency and innovation.

After using Fitnesse now for three years we’ve become used to it, and don’t ever question its usage. Although we feel the pain of the rigid/fragile test code on a daily basis. I decided to start rethinking the way we do acceptance tests.

My goals was to make our tests:

  • Simpler to write
  • Easier to refactor
  • Maintain the readability
  • Easier to reuse
  • Faster! (no DSL parsing)

This is what the initial design now looks like. The example used above would become the following:

@Jitness        // Annotation that indicates this is a test class
@SuiteSetUp({   // This is called once for all the methods in the class below
	@Execute(
			type = ShipExecutor.class,
			data = { "1001", "Maersk Bali", "TANKER", "D6DZ8", "7420211" })
	})
})
@SetUp({
	       // @Execute annotations here will be called before each of the methods below
	       // The @SetUp annotation can also be placed on a single method in the test class
})
public class AnchorageTests {

	public void givingCommandAnchorDownCausesShipToAppearInList() {

		final AnchorageScenarios scenarios = new AnchorageScenarios();

		scenarios.whenAShipReachedThePortCalled("Maersk Bali");
		scenarios.andItReportsAnchorDown();
		Assert.assertTrue(scenarios.theShipIsShownInTheListOfAnchoredVessels());
		Assert.assertEquals(1, scenarios.theListOfAnchoredVesselsNowHasSize());
	}

	public void anotherTestHere() {
		... etc ...
	}
}

Using Jitness (working title) the scenarios will be implemented in exactly the same way we’ve already been doing. This makes it easier to transition from Fitnesse to this code.
The only bit of ‘magic’ is the execute annotation. You can write executors like this:

public class ShipExecutor implements Executor {

	private ShipRepository repository = new ShipRepository();

	@Override
	public void before(final String[] args) throws Exception {
                repository.store(parse(args));
	}
	
	@Override
	public void after(final String[] args) throws Exception {
                repository.delete(parse(args));
	}

        private Ship parse(String[] args) {
		return new ShipBuilder()
                         .id(Long.parse(args[0])
                         .name(args[1])
                         .category(args[2])
                         .callSign(args[3])
                         .shipNumber(args[4])
                         .build();
        }
}

A (more elaborate) sample of the output it generates, with failure:

Results: 
OK	BunkeringTests.SuiteSetUp
OK	BunkeringTests.addTypeOfFuelToBunker
OK	BunkeringTests.changeTypeOfFuelToBunker
OK	BunkeringTests.removeTypeOfFuelToBunker
OK	BunkeringTests.bunkeringOnIncomingMovement
OK	BunkeringTests.bunkeringOnOutgoingMovement
OK	BunkeringTests.loadBunkeringForOpenVesselVisit
NOK	BunkeringTests.retrieveListOfBunkerBarges
expected:<1> but was:<2>
OK	BunkeringTests.SuiteTearDown
Took: 128 ms

The main advantages of using this are obviously leveraging the full potential of your IDE, including renaming/refactoring. Changing methods and classes in your production code will automatically show up as changes in Jitness. And everything is easy to refactor because it is just code! This will eventually lead to much more agile code, simpler and easier to refactor. Obviously there is one main drawback, it is harder for testers to read and write. But if they pair up with a programmer this should be no problem I think. I’d rather have the testers learn a bit of coding than the programmers learn a text DSL which leads to fragile tests.

Another big advantage is execution time, using compiled code is always faster than parsing plain text or HTML at runtime. Also this code is compiled during the build, so there is no need for additional classpath (Maven) plugins. I’m really fond of the idea to use code to describe the tests, and I’m very curious how my colleagues are going to rate this tomorrow when we demo the “ShipIt” ideas. With a couple of more days work this can be fully tested and made robust enough to start using it and maybe even open source it.

What are your thoughts? Any ideas about improving the API? Should we just keep using Fitnesse/Cucumber or any other alternative DSL instead?


Ethics for programmers

Ethics for programmers

Programmers need more ethics.
And I’m not talking about ‘craftsmanship’ ethics such as “always make clean code”, “be agile” or “coffee should be consumed pure and black”.

No, I’m talking about real ethics:

Ethics, also known as moral philosophy, is a branch of philosophy that involves systematizing, defending and recommending concepts of right and wrong conduct.

Source: http://en.wikipedia.org/wiki/Ethics

PRISM

Almost all the programmers I know are very active online, and all of them are condemning the US government project called PRISM. It is a huge electronic surveillance program, recently revealed by Edward Snowden. This program collects (all) data from a huge list of international websites such as Google, Facebook, AOL, Apple, Yahoo, Microsoft, and more.

One can only imagine the magnitude of this project, it handles a humongous amount of data and it would have to filter and match the data. It will have a lot of (secret) API’s and other methods of collecting the data. And this is all created by…. us programmers!

Just say no…

Why would somebody help the government in making these applications? I ethically wouldn’t think about programming this. And I’m serious about it, I’ve already turned down two projects in my young career because of an ethical standpoint:

  1. Helping to tweak an algorithm for the Dutch government for missile guidance (it would have been a cool project!)
  2. A project for a telecom company doing deep packet inspection, enabling them to ask subscription costs for (free) mobile apps
    (such as WhatsApp)

Both these projects might influence my own, or other people’s life in a negative way, and I refuse to help people do that. It’s like being an arms dealer or gun manufacturer condemning the use of weapons. It would feel like selling a crowbar to a thief, making a bit of money, while actually enabling them to break in.

Ethics!

Maybe programmers should have an oath, something like the Hippocratic Oath or the Physician’s Oath. Here are some of the rules I follow:

  • I won’t write code which helps/enables people to harm others
  • I won’t write code which will limit other people’s freedom
  • I won’t work for companies using software patents to bully/attack other people and companies

If you have any good additions to the list, please drop a comment!


Why are you not organizing a Devoxx4Kids?

Why are you not organizing a Devoxx4Kids?

Last year I attended the first Devoxx4Kids in Gent (Belgium). It is a day where programmers introduce children aged 10 to 14 to programming. That day we used Scratch (a fun online programming framework), Lego Mindstorms and a Mars rover (with emulation). And it was a blast!

Once I returned and edited this video and showed it to my colleagues we quickly came to the conclusion: We need to organize a Dutch version!

Organizing

Edwin (a JPoint colleague) and I tried a couple of contacts and found a willing foe in the form of the University of Amsterdam. They saw the potential and were kind enough to offer us rooms, laptops, Lego Mindstorms and a completely organized lunch! We in turn hired two students to assist with the sessions and took care of subscription, marketing and the technical know-how. It doesn’t take a lot of organizational skills to organize a Devoxx4Kids session.

Devoxx4Kids NL

Last saturday was the moment of truth, in the morning kids and parents started to arrive at the university. We did a quick introduction, flew around with (off the shelf) quadcopter to start off with the wow-factor and then we went into the sessions. In Belgium all the parents left the kids, but this time most parents accompanied the children into the session rooms. In retrospect this wasn’t very good and next time we should banish parents from the sessions. The sessions we organized:

The kids were split up into three groups and rotated over all the sessions. Every session was 90 minutes, in retrospect this is a bit long. Most kids got tired and a bit bored after about 45 to 60 minutes. The last track of the day the kids were a lot more distracted. For future Devoxx4Kids events I’d rather do short fun 45 minute sessions instead of complete in depth 90 minute sessions.

Kids learn quickly… even faster than adults! I tried my Arduino session on a couple of adults and they took about 75 minutes to complete all the tasks. The kids did it in record time, we had to improvise and set them a new challenge on the fly. They first made a blinking LED, then hooked up a piezo buzzer to play some music, then they created a ‘musical instrument’ with a photoresistor and the piezo buzzer. Finally (improvised on-the-fly) we made them create a traffic light with three colored LEDs! The quick learning was also noticed in the other sessions.

Our Devoxx4Kids was completely sold out in just 2 hours, so there is a lot of demand for events involving kids and programming. We are going to brainstorm this Wednesday on how we can do more on this front. Keep organizing Devoxx4Kids for 40 kids at the time? Or are there other options? How can we make it pay for itself instead of being fully voluntary…? All ideas are welcome, leave a comment/tweet/email!

Video impression

Are you not yet convinced you should organize a Devoxx4Kids in your neighbourhood/country? Just watch the video I made during our event:
<iframe type="text/html" src="http://player.vimeo.com/video/67959779" width="640" height="390" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>

p.s. Having pictures and video help to tell the story. This video from Devoxx4Kids 2012 in Belgium helped us to get the University of Amsterdam involved!


Teaching; Programming apprentices

Teaching; Programming apprentices

We have a huge problem.

The knowledge gap

I’ve been a professional programmer now for over a decade, and during this time I’ve met a LOT of excellent programmers. Most of them are partially autodidactic and learned the business side of programming in school. But non of these guru’s/experts use their skills to teach the next generations. Sure, they attend programming conferences, give talks, join user groups etc. But all the other attendees are already good programmers! Companies usually send their brightest programmers to these conferences, not the beginning employees. This means there is already a gap in knowledge and it is increasing. The smart are getting smarter, and nobody is really helping the beginners…

Teacher cleaning chalkboard with duster

Teachers salary

During my time in college I wasn’t very happy with the programming teachers. They had some experience with the business side, they teached us about UML, gantt charts, and (then still hot) waterfall project planning. When it came to real programming, problem solving, algorithms, best practices… they didn’t really teach us much.
Most of the time we had more programming experience and skill than the teachers.

Although I haven’t been in college for a couple of years, except two guest appearances at the University of Amsterdam, I don’t think much has changed. The whole problem is the status and rewards of being a teacher. The average salary of a college teacher (here in the Netherlands) is about half, maybe two thirds of the average professional programmers salary. No wonder why most average or better programmers don’t become teachers! Only the programmers with least experience and skill, or teachers which later on learned programming end up becoming programming teachers.

Mentoring/apprenticeship

In the ancient Greece and Roman times teachers were important professions. Especially the most experienced craftsmen, they did not only master their craft, but earned most money by passing their skills as master/mentor to their protégés. This doesn’t happen anymore, the big names in programming now don’t have apprentices to pass on their knowledge. In the best case they write books, which will take years before the school teachers have read them and adopt the ideas.

Devoxx4Kids?

I’ve already mentioned Devoxx4Kids in previous blogposts. Although this is a great initiative (which I fully support, we’re holding our own version in the Netherlands!). It helps getting kids excited in IT/programming, but it won’t help getting the level of education to a higher level.

Highest status

Being a teacher/mentor should have the highest social status in our modern world, it should be the highest paid profession and all the guru’s/experts should want to become teachers. But it isn’t. It is poorly paid and absolutely not the sought after job. How can we change this?


Agile: Epic Manifesto

Agile: Epic Manifesto

 

At work we’ve had a lot of discussion about epics lately. We seem to have forgotten what they are and how we learned to deal with epics. It became a ‘theme’ or just a big pile of stories which sounded like they belong together. But epics are much much more.

An epic is a goal, and just one goal only. All the stories in the epic should contribute to that epic! Also the epics goal should be a business goal, never a technical goal. The epic should describe a SMART described measurable goal without hinting towards a (technical) solution.

 

Epic manifesto

 

In our project we tend to forget what epics are for and why they are so important after some time.
To remind everybody we decided to write down a manifesto for the epic:

  1. An epic has just one goal, not more and not less.
  2. An epic is always measurable, thou should always ask “why” until the business value is clear.
  3. An epic has a fixed goal, new insights will lead to new epics.
  4. An epic should have a solution conceived with and verified by the stakeholders, users, product owner and the team.
  5. An epic contains only stories that help to achieve the goal.
  6. An epic is ‘done’ when the goal is achieved.
  7. An epic is only estimated in relative complexity, never in units of time.
  8. An epic is always in the correct order on the backlog (by the product owner).
  9. An epic is realized by the team with the minimal viable solution.
  10. Does your epic break one of the rules above? It is an epic failure!

This is going on several walls in our building just to remind us how important correct epics are. If the initial goal is wrong, all the stories are wrong as well. The epic is the base of all the solutions you’ll be creating and must be founded properly.

Any thoughts on this subject, don’t hesitate to reply. Agile projects get better by sharing best practices and trying to improve!


Conference videographer

Conference videographer

A couple of years ago, when our family was planning a trip to South Africa, I bought a nice DSLR. We had friends with excellent cameras, huge lenses and wild life photography experience. So instead of trying to compete with them I decided to buy a camera suited for filming. It was the rise of filming-DSLRs, and even television shows were adapting DSLRs for filming. I settled on a Canon 550D also known as Rebel T2i.

Just before I left for South Africa I decided to take the camera to the Devoxx 2011 conference. I didn’t even have a tripod at that time! After shooting some quick footage I decided to edit everything to make a nice movie and share it with the organiser. He liked it so much he placed it on the main website for a while. This is what I made:

Devoxx 2012

The following year Stephan asked me to be the official videographer for Devoxx 2012! So I went back and made another video, it seems like I’m turning from regular visitor/developer to videographer. I took some more time to analyse the conference to include the main sponsors, show every aspect of the conference and to get across the Devoxx atmosphere (including the James Bond theme, we all watched SkyFall during the conference!):

Devoxx4Kids 2012

The best time I had as videographer was during Devoxx4Kids. It is a separate event first hosted in 2012. The kids were having a great time and that is clearly visible in the video. Kids aren’t shy at all and much easier to approach then adults. That is why I did some quick interviews and I love the result, I might do that in the future for other conferences as well:

Devoxx UK 2013

The latest conference I filmed was Devoxx UK. For the first time the Devoxx conference has crossed the big blue sea to go to the United Kingdom. I’ve tried to make the resulting video look a bit more professional although I’m still just a Java developer with a hobby:

The future…

The last three years I’ve filmed at three weddings, four conferences and edited together our vacations to South Africa, Norway and some trips into the Alps (wintersport). As an amateur videographer (with a growing list of references) there still is a lot to learn, but I’m really enjoying the process! Any tips/hints/tricks are absolutely welcome, and I’m open for invitations!


Test Swing applications in Fitnesse (Nonkey)

Test Swing applications in Fitnesse (Nonkey)

Fitnesse

Fitnesse is a very popular test framework, used in a lot of projects. Using a wiki you describe the tests as plain text, which get translated into ‘fixtures’, code that is called that, in turn, can call your application. An example of a fixture we use:

|when the user logs in with username|roy               |and password   |test123   |
|and asks for the account details                                                 |
|ensure                             |displayed fullname|Roy van Rijn              |
|ensure                             |websites contains |http://www.royvanrijn.com/|
|ensure                             |websites contains |http://www.redcode.nl/    |

The corresponding Java code could be something like:

private Credentials credentials;
private AccountDetails accountDetails;

public void whenTheUserLogsInWithUsernameAndPassword(String username, String password) {
    credentials = new Credentials(username, password);
}

public void andAsksFotTheAccountDetails() {
    accountDetails = mySystem.getAccountDetails(credentials);
}

public boolean displayedFullname(String fullname) {
    return fullname.equals(accountDetails.getFullename());
}

public boolean websitesContains(String url) {
    for(String website:accountDetails.getWebsites()) {
        if(url.equals(website)) {
            return true;
        }
    }
    return false;
}

This is a very simplified example on how you could use Fitnesse to test your application. But the idea is very powerful, there are implementations for Java (initially) and now .NET, Groovy etc. Also, most of the time the system you’re testing doesn’t have a Java API but instead you’d like to test the user interface. With most Java applications being web applications there are numerous plugins to connect Fitnesse with for example Selenium to ease web application testing. The fixtures fill webpages and analyze the resulting HTML response.

At my current client (Port of Rotterdam) we’re building a very rich, partially offline, Swing application. This application has a backend and numerous clients running the Swing application. We’ve used Fitnesse to test the backend code from the start, and the last year we’ve put in some effort to also use Fitnesse for the Swing application. We’ve called this “Nonkey”, being an automated version of our Monkey integration testing we’d been doing up to that moment.

UISpec4J

The first ingredient we need to test Swing GUI’s in Fitnesse is UISpec4J. This is an excellent Swing test framework that allows you to manipulate GUI elements from code. The framework is made with unit-tests in mind, but can also be used in integration testing. In our SetUp page in Fitnesse we call the following code:

private Window mainWindow;
private JFrame mainFrame;
    
//Called from the Suite Set Up page:
public void launchNonkey() {
    mainWindow = WindowInterceptor.run(new Trigger() {
        @Override
        public void run() throws Exception {
            new OurApplication().launch();
            // Get the main JFrame from Spring, used directly in some cases instead of UISpec4J
            mainFrame = BeanFactory.getBean("mainFrame", MainFrame.class);
        }
    });
}

The WindowInterceptor captures the Swing application and allows for manipulation in a later stage. For example:

//In some fixture:
public void loginWithUsernameAndPassword(String username, String password) {
    //UISpec4J uses JComponent.setName() for lookup:
    Panel loginPanel = nonKey.mainWindow.getPanel("loginPanel"); 
    final TextBox usernameTextbox = loginPanel.getTextBox("usernameTextbox");
    usernameTextbox.setText(username);
    final TextBox passwordTextbox = loginPanel.getTextBox("passwordTextbox");
    passwordTextbox.setText(password);
    
    loginPanel.getButton("loginButton").click();
}

This fixture causes the Swing application to fill in the username, password and press the login button. It is easy to read and can be launched from Fitnesse with a single line:

|login with username|roy   |and password   |test123   |

Nice! We are done, we can test Swing GUI’s in Fitnesse now! Well no… not really, unfortunately there are a couple more problem we ran into, regaring the Event Dispatch Thread (EDT).

Event Dispatch Thread (EDT)

The EDT, sometimes called Swing Thread, is a special thread. Everything you do in Swing should always happen on this thread. Setting components to visible, pressing buttons etc, all need to be done in the EDT. If you don’t do this correctly the application gets into a lot of race conditions and it can even cause the application to hang.

When we combined Fitnesse with UISpec4J and started making a lot of tests we noticed that sometimes mysteriously a test would fail without changes. It turned out to be a problem with the EDT. UISpec4J doesn’t worry too much about the EDT, and when we click a button or setText (like the examples above) it does this in the current thread. In Fitnesse this thread is the thread that called the fixture, obviously not the EDT.

We brainstormed a lot about how we could fix this, maybe change UISpec4Js code? This turned out to be a hellish task. How about setting SwingUtilities.invokeAndWait() in every fixture? That could work but is also a massive undertaking. We decided to take a look at how Fitnesse calls the fixtures. This is always done from a ‘StatementExecutor’. So we decided to build a custom StatementExecutor which causes the fixtures to all run in the EDT.

To do this we need a custom SlimService. First I tried to extend the normal SlimService, but this wouldn’t work for some odd reason (still unexplained, should try this again). We decided to copy the SlimService code and make our own:

public class SwingSlimService extends SocketService {

    public static SwingSlimService instance = null;
    public static boolean verbose;
    public static int port;

    public static void main(final String[] args) throws Exception {
        if (parseCommandLine(args)) {
            startWithFactory(args, new SwingJavaSlimFactory()); //<-- custom SwingJavaSlimFactory!
        } else {
            parseCommandLineFailed(args);
        }
    }

    protected static void parseCommandLineFailed(final String[] args) {
        System.err.println("Invalid command line arguments:" + Arrays.asList(args));
    }

    protected static void startWithFactory(final String[] args, 
            final SlimFactory slimFactory) throws Exception {
        new SwingSlimService(port, slimFactory.getSlimServer(verbose));
    }

    protected static boolean parseCommandLine(final String[] args) {
        final CommandLine commandLine = new CommandLine("[-v] port");
        if (commandLine.parse(args)) {
            verbose = commandLine.hasOption("v");
            final String portString = commandLine.getArgument("port");
            port = Integer.parseInt(portString);
            return true;
        }
        return false;
    }

    public SwingSlimService(final int port, final SlimServer slimServer) throws Exception {
        super(port, slimServer);
        instance = this;
    }
}

Next we needed a custom JavaSlimFactory:

public class SwingJavaSlimFactory extends JavaSlimFactory {

    @Override
    public StatementExecutorInterface getStatementExecutor() {
        return new SwingStatementExecutor(); //<-- custom SwingStatementExecutor!
    }
}

And last but not least our custom StatementExecutor which forces all calls into the EDT thread:

package com.portofrotterdam.hamis;

import fitnesse.slim.StatementExecutor;
import fitnesse.slim.StatementExecutorInterface;

/**
 * HaMIS custom StatementExecutor, to call and create in the EDT thread
 */
public class SwingStatementExecutor implements StatementExecutorInterface {

    private final StatementExecutor wrapped = new StatementExecutor();

    private abstract class EDTRunnable implements Runnable {

        private Object returnValue;

        @Override
        public abstract void run();

        public void setReturnValue(final Object returnValue) {
            this.returnValue = returnValue;
        }

        public Object getReturnValue() {
            return returnValue;
        }
    }

    @Override
    public Object call(final String instanceName, final String methodName, final Object... args) {

        final EDTRunnable caller = new EDTRunnable() {

            @Override
            public void run() {
                setReturnValue(wrapped.call(instanceName, methodName, args));
            }
        };
        EDTRunner.run(caller);
        return caller.getReturnValue();
    }

    @Override
    public Object callAndAssign(final String variable,
            final String instanceName,
            final String methodName,
            final Object[] args) {

        final EDTRunnable callAndAssigner = new EDTRunnable() {

            @Override
            public void run() {
                setReturnValue(wrapped.callAndAssign(variable, instanceName, methodName, args));
            }
        };
        EDTRunner.run(callAndAssigner);
        return callAndAssigner.getReturnValue();
    }

    @Override
    public Object create(final String instanceName, final String className, final Object[] args) {

        final EDTRunnable creater = new EDTRunnable() {

            @Override
            public void run() {
                setReturnValue(wrapped.create(instanceName, className, args));
            }
        };
        EDTRunner.run(creater);
        return creater.getReturnValue();
    }

    @Override
    public Object addPath(final String path) {
        return wrapped.addPath(path);
    }

    @Override
    public Object getInstance(final String instanceName) {
        return wrapped.getInstance(instanceName);
    }

    @Override
    public void reset() {
        wrapped.reset();
    }

    @Override
    public void setVariable(final String name, final Object value) {
        wrapped.setVariable(name, value);
    }

    @Override
    public boolean stopHasBeenRequested() {
        return wrapped.stopHasBeenRequested();
    }
}

This uses a small static class we’ve used in our code before to safely run a Runnable on the EDT and invokeAndWait (without waiting IN the EDT):

public class EDTRunner {

    public static void run(final Runnable r) {
        if (SwingUtilities.isEventDispatchThread()) {
            r.run();
        } else {
            try {
                EventQueue.invokeAndWait(r);
            } catch (final InterruptedException e1) {
                throw new RuntimeException(e1);
            } catch (final InvocationTargetException e1) {
                throw new RuntimeException(e1);
            }
        }
    }
}

The only thing left to do is to instruct Fitnesse to use our SlimService instead of the default implementation. This is done using the following parameter in our main page:

!define TEST_SYSTEM {slim}
!define TEST_RUNNER {com.portofrotterdam.hamis.SwingSlimService}

Make sure that the class can be found (for example, package it in a JAR and place it on Fitnesses classpath). And you’re done! Using a ThreadCheckingRepaintManager (workings described here) we checked to see if the tests are now repainting/manipulating Swing from the correct thread, and it works like a charm.

Good luck testing with Fitnesse in the near future, if you have any problems/improvements/ideas please let me know (in the comments below).


Facebook status: Deceased

Facebook status: Deceased

In the next couple of years dead will become an important aspect of the social networks. A time will come that most profiles on Facebook will belong to people who’ll never be able to post another update. Currently due to inactivity most dead profiles will be deleted or become inactive, but I think dead will take a much more important role on the internet.

My parents and even grandparents have a Facebook account now, but in twenty or thirty years, some people I’m connected to won’t be alive anymore. It would be great to be able to browse the lives of people who are no longer around us. I’d love to read about my deceased grandmother for example, just to see how she lived her life. So instead of deleting inactive accounts, death should become more accepted and a part of our social web as well.


Devoxx 2012, the video

Devoxx 2012, the video

Yesterday when I returned home from another amazing Devoxx conference I quickly created another impression video:


Devoxx4Kids

Devoxx4Kids

A couple of weeks ago I went to Devoxx4Kids. Devoxx is a very well known Java conference (the best in the world actually!). This time they organized an event especially for kids ranging from 10 to 14 years old. The goal is twofold, the main goal is to just give them a good time, the second goal is to introduce them to programming and technology in a fun way.

To show the world this is a great idea, Stephan invited me to film the event and capture the happy faces of the kids enjoying themselfs. And now after Devoxx 2012, I’m allowed to show it to everybody:
<iframe type="text/html" src="http://player.vimeo.com/video/51393678?badge=0" width="640" height="390" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>

(more information in Dutch and French: http://www.devoxx.com/display/4KIDS/Home)