Sometimes I let my mind wonder and I get crazy questions. Today was a good example, I encountered a MD5 hash and I started to wonder, would there be a hash which would (when hashed again) be the same?
Thus: MD5(x) = x
This would be a kind of MD5 quine, when fed into the algorithm you get the original value back. This is actually called an MD5 fixed point.
Information I’ve found
So I started investigating, soon I discovered this website about collisions. Its well known that all hashing algorithms must have collisions, you can’t always produce a unique hash for input larger then the output of course.
The output of the MD5 sum is 128 bit (16 byte) long, so the input should also have the same length. But the MD5 algorithm is defined to have 512 bits as input. This isn’t really a problem because the algorithm will extend smaller input with padding. Lets assume the MD5 sum of any input is uniformly distributed over all possible sums, then the probability that a 128-bit string is a fixed point is 1/2^128. This isn’t a crazy assumption because all hashing-algorithms are designed to distribute the output as uniformly as possible to avoid collisions.
So, the probability that no 128-bit string is a real fixed point is (1 - 1/2^128)^(2^128). The probability that there IS a fixed point is 1 - (1 - 1/2^128)^(2^128).
Since the limit as N goes to infinity of (1 - 1/N)^N is 1/e, and 2^128 is most certainly a very large number, this probability is almost exactly 1 - 1/e = 63.21%.
But, of course, there is no randomness involved here - there either is a fixed point or there isn’t. But, we can be 63.21% confident that there is a fixed point. (Also, notice that this number does not depend on the size of the keyspace - if MD5 sums were 32 bits or 1024 bits, the answer would be the same).
Looking for the fixed point
I’ve just implemented a small program to look for these hashes, even though I know it will take millions of years to check all the numbers. But you never know, I might get lucky ;-)
The first algorithm I created took a single random String as input and kept applying the algorithm to the output. Eventually it will:
- Go into a loop
- Find a fixed point (which is a loop of size 1)
The weird thing is, if it ends up in a loop of size 1… I’ve found two things. Not only a md5 fixed point, which creates itself after applying the algorithm. But also an input-value that produces this md5 as output, a collision!
Another interesting thing would be a complete graph of all md5 answers. Which loops can we find, which md5 has most collisions etc. But this would take eternity to calculate, even using all the machines in the world.
- Are there loops? (It is possible there aren’t any loops at all…?)
- Are there loops of size 1, a.k.a. fixed points?
- Which/how many 128 bit combinations can’t be created with the input values?
- Which/how many collisions will you get with all the input values?
More thoughts, errors, solutions..?
In our current project we want to have multiple Spring Web Flow-flows in one WAR-file. But we also want the flows and pages to be inside seperate JAR files, making the application a bit more managable and modulair.
This sounds straightforward but it took quite a bit of code and time…
First I created a single WAR-project with all the basic Spring, JSF and Facelet configuration. Like any Spring Web Flow (SWF) project we have a project-servlet.xml.
The first thing I did was changing our flowRegistry:
The classpath*: allows SWF to search the whole classpath for flow-directories containing a flow definition.
In our case it would be: /flows/module1/module1-flow.xml
When you try to run this, and access a page we got the following exception:
For some reason Spring Web Flow doesn’t want to load the facelet. After browsing around Spring’s forums I came across some solutions. They didn’t do the trick, only when combining several methods I got it working for Spring Web Flow 2.0.7.
This is how I did it, we need to tell Facelets to use our custom ClassPathResourceResolver:
The resolver itself is basic but does the job:
This will help Facelets to translate a given path to a java.net.URL using the current classloader.
Spring Web Flow is currently giving us a FileSystemResource, and this doesn’t work because we want to load the pages with our classloader. For this we have the following wrapper:
To force Spring to use this Resource instead of FileSystemResource we use a post processor:
When the FlowDefinitionRegistry is created we provide it with a new ResourceLoader. When the resources are requested by Spring Web Flow we create our own CustomClassPathContextResource. This consists of our current location plus the defined location (viewId).
To register this post processor add it to your project-servlet.xml:
How to use it
In our project we have the following flow(s) defined in a seperate JAR:
And our pages are in the same directory:
In the flow we can now use the following view-id’s:
Its also possible to have pages in other places, you can define the views as relative paths. For example:
”../../shared_pages/page2.xhtml” turns into “/shared_pages/page2.xhtml”
”../pages/page3.xhtml” turns into “/flows/pages/page3.xhtml”
The only problem we still have using this method is the deployment with RAD/Eclipse WTP and Facelets auto-refresh. For some reason after deploying our application locks the files in the bin-directory. Eclipse then can’t delete this directory and fails to publish. Ending in one big #fail. But a simple clean, clean, republish, clean, rebuild, restart, shout, scream, cry, rebuild and republish will solve this.
The big advantage is the fact that the flows and pages are now defined in their own JAR files, making releases and sharing classes (like shared services/shared menu’s etc) much easier.
Information on Spring forum
Almost all programmers have heard about and used design patterns. And there are a lot of them. Famous design patterns include the Singleton, Observer, Template method. These are all patterns that focus on objects and the relationship between them.
There are more groups of patterns, for example take Dependency Injection (or Inversion of Control). This is an architectural pattern. They tell you things about the design of the whole program. And there are specific pattern groups, like concurrency patterns.
These patterns help you solve common problems in a well defined way, other programmers will recognise the patterns used and it will help produce more maintainable code.
Inversion of logic
The patterns described above all say something about classes/methods and architecture. But I think another class of design patterns exist, which have a smaller granularity. These patterns say something about the code inside a single method. A good example is Inversion of logic.
Let me give an example, lets take the following code:
Its a basic car-rental service and is has some logic before a request is processed. To make this code more readable and maintainable we can do two things. Of course it would be better to factor out the validation-logic into another piece of code. So lets do that first:
Some programmers don’t like the validateRequest() method above because there are multiple exit points. This was more problematic in C code then in Java because in C you needed to do a bit more resource management (freeing mallocs etc), in Java I actually don’t mind having more then one exit-point in my code. In this case it makes it just a bit more readable. It would be easy to add a ‘requestIsValid’ boolean and fill that instead.
Anyway, the next step in refactoring this code is using what I call “Inversion of Logic”. The idea is that you don’t continue nesting if something is true, but instead inverse it, and bail out if something is false. The big advantage is that you’ll lose a lot of nesting, making the code more readable. I’m going to do this in two places, first in the processRequest for-loop, and second in the validateRequest method.
As you can see in the for-loop all the business logic is moved one indent back, this will make your code easier to scan for business rules. And if you look at the validateRequest method the code reads more fluent. Instead of stacking up and remembering the valid cases:
IF form is correct
AND tenant age correct
AND tenant licence is valid
We now eliminate the bad cases first:
IF form is incorrect: deny rental
IF tenant’s age or licence is invalid: deny rental
I’m trying to do this everywhere I can, it realy helps the readability.
Today marks the release of Java EE 6. The reference implementation (Glassfish V3) has been released and the specifications are going into their final state very soon.
About two years ago, while I was attending the JavaOne conference, I first heard about the Servlet 3.0 ideas. As a web developer I’ve worked a lot with these Servlets so I was curious about the ideas. But what I saw wasn’t what I hoped for. On the contrary, what I saw was a huge mistake in my opinion!
So I decided to contact the guys behind this JSR, to ask for some more information and share my views. Some time passed but I never got a reply… My next move was just to write about it, first on my own blog, then on DZone (05/30/2008) and on The Server Side a half year later, december 2008.
The only ‘reply’ I got from the people of the JSR was in the days following my (re-)post on The Server Side. Glassfish has a short re-cap on what happened here.
Now that the release is final I’m actually glad with the result. All the issues I raised from the early draft have been fixed. This is what a simple servlet would have looked like in the early draft:
And this was my suggestion on how to improve it (see second article/link):
And this is from the final release:
That’s very similair, maybe a little too similair..? But anyway, I’m pleased about the result. It’ll be a lot easier in the future to write Servlets and Filters and a lot of cool new features have been added too.
I love the spec, despite the lack of feedback and replies from the expert group, ignoring all positive feedback and constructive advice. If I now look at the way the annotations turned out its almost identical to my first sketch after their JavaOne presentation in March 2008.
Every once in a while head hunters call me. They call you at home, at work, they work out your email adress and mail you… This will probably sound very familiar to most IT professionals.
Anyway, the thing that strikes me is that about 50% of the time a head hunter calls, its Steve Oakley! And it isn’t just me that is getting calls from Steve Oakley, no, all my collegues and friends are getting calls from him as well!
So, are you in or out? Have you been called by Steve Oakley yet?
To improve the quality of your code it is important to thoroughly test your code. Not only using unit tests but also using integration tests. I’ll describe these terms and show examples on how to create tests using JUnit and Spring’s Test library.
A unit-test is an automated test that tests one single unit-of-work. So, what is this unit-of-work? Its the smallest piece of code that can be distinguished. If you are programming in Java the smallest unit you can test is a single method. So all unit-tests should only test this single unit, and no other dependencies.
Here is an example of a unit-test:
By adding the @Test annotation JUnit will know this is a unit test. When running the code we verify that it correctly adds two values.
Test Driven Development
Test Driven Development (TDD) is very important in modern software development. More and more people and businesses are using this technique to improve their code quality.
What TDD consists of, basically, is a very small development cycle for all your code. You just follow these steps:
- Write a unit-test that verifies your usecase
- Implement the interfaces you created for step 1
(can be done quick and dirty, just to get the logic correct)
- Run the tests and confirm your code is logically correct
- Refactor your code into maintainable, readable, reusable code
- Confirm the code still works as designed by running the test(s)
Actually, I don’t write all my tests before the real code, but the process goes hand-in-hand. When writing code I always think about how I’m going to test it, and when the code is done, the test is too, combining step one and two. It speeds the process up a bit, and the code I end up with is almost the same because of the refactor-step.
With unit testing you can verify that the smallest units of work are correct. But this doesn’t give you any guarantees!
A famous example is NASA’s $125 million Mars Orbiter. It crashed when it was about to land on Mars. And why did it crash? Because one component calculated metric units and it feeded this information into another component that was using English units.
Very painfull for NASA, but a great example showing the importance of integration testing. Both NASA-components had been tested a lot, but apparently they neglected to test the components together.
Integration testing with Spring
Most business software in Java uses Spring. This is a framework glueing all your components together and providing dependency injection. This makes it easy to test all the classes and methods, but it makes it a bit harder to test the integration and communication between classes because you depend on the framework for that.
Luckely Spring helps us with a their Spring Test library full of tools that help you creating integration tests.
One thing you need in all integration tests is Springs Context. With Spring Test it is possible to wire some beans and let Spring inject those beans in your unit test. The old(-ish) way to do this is extending org.springframework.test.AbstractDependencyInjectionSpringContextTests or org.springframework.test.AbstractTransactionalSpringContextTests.
If you want to use these classes you need two things:
- Create a seperate Spring configuration with test-beans (or use your normal Spring config)
- Override String getConfigLocations(), provide it with the location of your Spring configuration.
Now, for all the private fields you have in your testclass Spring will attempt to auto-wire beans on it.
A small example:
Testing with Spring 2.5+
With the introduction of Spring 2.5 they improved the above method. For more flexability and a bit more control over the objects they introduced the SpringJUnit4ClassRunner.
This is how you would use it:
It works the same way as before, but instead of having to override the AbstractDependencyInjectionSpringContextTests you now have a POJO. All the information about Spring and JUnit are placed in annotations on top of you class.
Hopefully I’ll find the time to write another post on mocking, including EasyMock and Mockito.
Curiosity is an emotion related to natural inquisitive behaviour such as exploration, investigation, and learning, evident by observation in human and many animal species.
To me, curiosity is one of the greatest things we as humans have. It feeds you with knowledge and sparks your fantasy, its an emotion that tickles your brain and forces you to explore.
In life its vital to search for new things, new limits, new experiences. Always keep asking questions, because with every answer you find, more questions will arise!
At first this sounds a bit depressing, more questions for every answer you get. This doesn’t sound satisfactory, but trust me, it is! Its not the answer that gives you satisfaction, its the new array of questions that give satisfaction! Because these questions feed our curiosity.
It also strikes me that children are very good in asking questions, and they come up with the most amazing questions! I really think this is a good skill, and its a shame we lose this when we get older. Why don’t adults ask these questions? Is it the fear of the “dumb-question”, does it make them look stupid?
Here are some examples of good curious questions:
- Q: How do blind people know when they’re done wiping toiletpaper?
- A: They can feel it from the drag, and then wipe two times extra (just to be sure)
- Q: If you smoke, and go into a coma. When you wake up, do you crave for a cigarette?
- A: It depends, after 8 days all the nicotine would be out of him/her bloodstream. So the physical chemical craving wouldn’t be there when they wake up. They’ll probably still have a habitual craving though.
- Q: Why don’t our eyelashes grow?
- A: All hair follicles have a certain cycle. The hair grows for a certain time, and then it just drops out. Eyelashes have a short cycle-rate.
- Q: Do fish drink?
- A: It actually depends: Saltwater fish drink, and filter the salt and minerals with their gills. Freshwater fish absorb water with their bodies.
- Q: How many neck-vertebrates does a giraffe have?
- A: The same amount as we have, seven. Even a mouse has seven vertebrates. All mammals have seven vertebrates except the manatee and three-toed sloth (They only have six)
- Q: Can Elephants jump?
- A: No, they can’t! They weigh too much. Also they get very uncomfortable if there is more then one feet of the ground at any time.
- Q: If you die from anti-freeze poisoning, would you be able to do cryopreservation?
- A: ?
- Q: A Dutch question: Waarom moet je bevroren eten altijd “ontdooien”? Dooien is het tegenovergestelde van bevriezen! Dus moet je iets toch dooien in plaats van ontdooien?
- A: ?
If you have curious/weird questions, or know answers to unanswered questions, please drop me a comment!
Something other then the usual technical posts…
As you might already know, I am an atheist. Yes, in my opinion there is no God, there never was and (probably) never will be.
Being an Atheist with Christian friends I get a lot of questions. Recently a friend of mine asked: “Why do you celebrate Christmas when you are an Atheist?”.
This question, to me, is quite simple. First of all, because it is a tradition. I think its a wonderful holiday. Ask around, what do people think about when you ask them about Christmas?
People think about:
- Christmas trees
- Santa Claus
- Toys, candy and gifts
- Decorations, lights
- Christmas dinner with relatives
- The poor, and less fortunate
- Christmas songs
- The Grinch
- Christmas cards
- Charles Dickens, A Christmas Carol
All of these things have no connection to the Christian Christmas, except the name “Christ-“.
The birth of Jesus
The date on which we celebrate Christmas is sometimes said to be the birthdate of Christ. But in fact this isn’t true. The date has been borrowed from other holiday’s. Originally people celebrated the Roman winter solstice on the 25th of December. It was probably just convenient to borrow this date.
So instead of saying I celebrate Christmas, I could just say I celebrate midwinter.
Okay, how about Santa Claus, there is nothing more Christmas then Santa Claus. But again this figure has nothing to do with Christianity. The only thing God and Santa Claus have in common is the beard.
I’m from the Netherlands, and on the 5th of december the old Saint Nicolaas (or Sinterklaas) visits our country. At night he visits all the houses and he brings gifts for all the kids! Does this sound familiar? It should be, he and Santa are the same guy..!
When New York City was still called New Amsterdam the Dutch tradition took off in America. Slowly evolving (yes, holidays evolve too!) into the Santa Claus we all know today.
And being Dutch I have a huge advantage, we celebrate Sinterklaas AND Santa Claus, getting gifts and candy twice!
But again, a good reason to celebrate Christmas, it is based on a old Dutch tradition, and Santa Claus has nothing to do with Christianity.
Family, friends and relatives
For me, the single most important reason I celebrate Christmas as an Atheist are my family, relatives and friends. Its the time of year, with its long dark days and cold nights, that is perfectly suited for spending time with friends and family.
Its also the perfect time of year to think about your own situation, about the people you know and love, about people who don’t have it as good as you have. Its the time of year of giving and sharing.
That is the most important reason to celebrate Christmas midwinter, winter solstice, late Sinterklaas.
Finally, I have to get something out of my system about evolution:
Please get over it, evolution exists, there is enough prove. Just because it is called a theory it doesn’t mean you can just throw it out of the Christmas decorated window. For example, did you know gravity is still a theory? It is, first it was known as Newton’s Theory of Gravitation, and after that a guy named Albert Einstein wrote a better version, the General Theory of Relativity. It is accepted that gravity exists and nobody doubts that. Just like gravity, evolution is not just a theory.
Recently VMWare, known from its computer virtualization software, has announced they are developing a dual-boot mobile version. This would allow you to run both Windows Mobile and Android on your phone.
But it isn’t going to be a dual-boot you know from your computer, they are planning on running the OS’es at the same time. This would allow you to switch between OS on-the-fly.
- You want to use a Windows Mobile app? *click*
- Oh, you want to use something on the Andriod platform? *click*.
I’m very curious how this will work out performance-wise, but the idea itself is great!
I’ve just stumbled across a new program: Eureqa
Its a program, developed by the Cornell Computational Synthesis Laboratory, that can detect equations in sets of data. Its primary goal is to identify the simplest mathematical formulas which could describe the underlying mechanisms that produced the data.
Its best described using their instructional video:
I’ve been playing around with it, for example trying to find a good prediction equation for the Son of Darts competition I blogged about earlier.
The algorithm(s) used in this program are based around “symbolic regression”. It is a form of Genetic Programming (GA) where the computer processes a tree of possibilities recursively searching for the best suited building-blocks.
What is #songsincode ?
Last week I discovered a new hype on Twitter. The so-called “#songsincode”. It’s a huge hype under programmers and technical users. The best way to explain is probably by showing an example:
This obviously is Paint It Black by the Rolling Stones (if the door is red, paint it black)
My own #songsincode:
Here is a collection of the #songsincode I created and tweeted:
And here is my all-time favourite (in CSS!!) created by @codepo8:
I hope you liked them!
I’ve been developing with Java 5+ for quite a while now. Not all developers are this lucky, some are still stuck with 1.4… some even with 1.3! But my clients all made the excellent step forward to Java 5 (some even to 6). The problem is, they moved the runtime/JDK but forget to move their developers!
In Java 5 the language brings some good improvements, the for-loop is easy to understand, and almost all the developers are using this by now. The problem starts with generics. There is a part most developers understand, the Collections API. Almost all programmers use lists now as: List
For example the piece of code below:
Of course, there seems to be not much wrong with the code, I see it all the time. Yes, the code breaks if you put something else in the comparator, but hey… the Javadoc says it only accepts LabelPlaceholders! So lets use this code:
Done! Its working and no problems right? Not quite, your IDE (Eclipse in my case) complains about this code.
For example Eclipse says:
Type safety: The expression of type LabelPlaceholderComparator needs unchecked conversion to conform to Comparator<? super T>
At this point, most programmers at the company I work for now will just ignore this warning. They might even add:
What a shame… Lets just examine this warning, what is Eclipse trying to tell us here? The compiler doesn’t know we created the Comparator with only LabelPlaceholders in mind. But the compiler does know (with generics) that the List only contains LabelPlaceholders. So the warning is (in understandable English):
I’ve got a list here of T (LabelPlaceholders) and a Comparator for Objects, this can go wrong! I’d rather have a specific Comparator for this job. Do you have one for me?
The solution to this problem is very simple, but most neglect to use it:
As you can see, the code is much smaller. The interface is now generified, it knows we are going to compare LabelPlaceholders now, nothing more, nothing less. Also, we don’t have to cast anymore, because of the generics you can’t put anything else in there.
So, lets go to the conclusion: Why is the latter code better code?
- As you can see, the code is smaller!
- There are no casts, the code is safer (no ClassCastException or eleborate class checks)
- If somebody uses your code, he/she knows what kind of objects the Comparator can handle. You don’t have to read the Javadoc or the code to see what it does.
Throughout the projects I encounter I keep finding examples of places where generics would have made the code smaller/safer/more understandable. For some reason the programmers still only use generics on collections. So, even though generics aren’t perfect, please use them where/when you can, it’ll always add clarity to the code, and most of the time it’ll also make your code safer, and in some cases the code gets smaller because you can leave away casts and class-checks.
Don’t ever let me see public int compare(Object o1, Object o2); again!
(You see, it is possible for me to have a discussion about Java generics without mentioning reified generics!)
Another thing I’ve been very busy with lately is AZsPCs (Al Zimmermanns Programming Competition). The current contest is called Son of Darts.
The idea behind these contests are that they are easy to grasp, but very hard to master.
Lets take three darts. You have to throw them to a dartboard which is divided into 4 regions. For example, the values on these regions are: 1,2,4,6.
The first question is: What is the lowest value you can’t throw with these three darts?
This is easy to calculate:
Can we throw one? Yes: 1 dart in the 1
Can we throw two? Yes: 1 dart in the 2, or 2 darts in the 1
Can we throw nine? Yes: 6,2,1
Can we throw ten? Yes: 6,4
Can we throw eleven? Yes: 6,4,1
Can we throw twelve? Yes: 6,6
Can we throw thirteen? Yes: 6,6,1
Can we throw fourteen? Yes: 6,6,2
Can we throw fifteen? Err… no, sorry…
So the score is: 15 points.
The main question: Can you think of better values for the regions of the dartboard to get a higher topscore??
This is what the competition is about. But not only for a dartboard consisting of 4 regions, but up to 40 regions. And not only for three darts, but also 4, 5 and even 6 darts.
If you can create a good solver its pretty easy to bruteforce up to a certain point, but the problem is, you quickly get more and more options for which you have to check the scores… It is an exponential function…!
Actually, this is not a new puzzle. Its been around of quite a long time. But its more commenly known as the local postage stamp problem (LPSP). Formulated just a little bit different, instead of a dartboard with regions you have a postcard with room for H stamps. What is the lowest value you can’t create with stamps Nh? Also check out Wolfram’s description of the problem.
This problem has been proven to be NP-hard, so bruteforcing won’t be an option, you’ll need to use something different. Put on your thinking-caps and create some good innovative heuristics.
About a week ago I had a discussion with a fellow programmer about some boolean logic. We had three parameters, something like:
- personHasInsurence (A)
- personNeedsInsurence (B)
- personIsKnownAtThisAgency (C)
We also had two particulair cases for an insurance page:
Person has insurence and isn’t yet known at this agency
Person doesn’t have insurence, needs insurence and is known at this agency
Person doesn’t have insurence, doesn’t need insurence and is known at this agency
So the view-logic was a bit complex:
Then I remebered something I learned at school some time ago. So called karnaugh maps. I’ve completely forgotten how to use them, but I knew it was possible to calculate the shortest form to comply to the logic rules. When looking further I found the so called “Quine McCluskey“-algorithm, and I decided to implement it (just to learn how it works).
Quine - McCluskey algorithm
First of, lets go through a couple of terms.
Minterm: A small boolean function which has all the different input variables, once. So for example, a minterm using the above variables would be: ABC (A and B and C), or A’BC (not A and B and C).
The first thing you do using this algorithm is that you find so called “prime implicants”. An implicant is a combination of one (or more) minterms, and a prime implicant is a implicant which can’t be combined with other implicants (for more details, read the wikipage with examples)!
After combining all the minterm of your case(s), you’ll end up with a “prime implicant chart”. This is a chart with all the prime implicants and the fields they cover. Sometimes its easy to spot “essential prime implicants”. That is, implicants which are unique in covering a field. You have to use these implicants in the final logic.
When you have multiple options left to combine to cover all the fields, you can use Petrick’s Method to select the best/smallest option.
Using the above example, if you minimize, you’ll come down to:
The algorithm is pretty fun to program, and its a bit different from most algorithms I’ve seen lately!
And if you need something even faster, try out Espresso!
ps. I love browsing this page: List of algorithms
But OTOH, its kind of depressing I still have soo much to learn.
I’ve implemented a Brainfuck interpreter!
The language itself is pretty simple, and most of it was implemented rather quickly in Redcode. The only big problem was navigating back to the correct closing brackets in a loop. This is now solved by counting the amount of open/close brackets in between.
So here is the code: