DSLs: Introducing Slang

Did you ever ask for a more compact way to express something in your favorite programming language? Say hello to DSL (Domain Specific Language). A DSL is a slang, a compact way to say what you want. When two astronauts talk, they use slang. They need to get information across and presto. "Over" instead of "I'll now clear the frequency so you can start talking." And when these guys do it, there's no reason for us not to.

Here is an artical on Java World which gives some nice examples how to create a slang in Java and in Groovy. Pizza-lovers of the world, eat your heart out.

Jazoon: One Year Of Groovy

That was my talk. Attendance could have been better but I guess with Joshua Bloch presenting Java Puzzlers, Cédric Beust with "A quick guide to modern languages and interesting concepts for the busy Java programmer" and Adam Bien with Glassfish at the same time, it wasn't so bad.

Here is the link to the presentation as OpenOffice document. Feel free to post any comments or questions below.

Jazon: Building DSL's

Neal Ford's talk about DSLs and how to create them in static and dynamic languages (Java and Groovy in this case) was very interesting because he clear up the fog around DSL's a but, why they are great and when they help. Basically a DSL is "jargon". It is compact code that allows you to express much more tightly what you want than any "general purpose" language ("one site fits all" actually makes everyone look ridiculous).

Remember that old OO promise to allow to mirror the world in a programming language, to allow to model the code after what the customer had in mind? Turns out, that OO was ... nah, not lying ... it was just overselling itself because customers don't think in objects or models. They think jargon. And DSL's allow you to make a compiler understand jargon. That's even possible with Java even though "Java is like taking to a retard": You always have to repeat yourself to make sure that the stupid compiler gets what you want (pun intended).

A DSL is not to be confused with an API. An API uses explicit context like "obj.setThis(...); obj.setThat(...);" etc. Here "obj" is the context and "setXyz()" is not meaningful without knowing the object we're talking about here. A DSL, on the other hand, is all about context and context is implicit. It goes "obj.should(...).allow(...).this(...)". An example is hamcrest: "assertThat(theBiscuit, is(myBiscuit));" Reads fluently, compact code, and it's obvious that "is(myBiscuit)" should be compared to "theBiscuit" in some way.

The handouts of his presentation are available from his homepage but without his witty remarks. A pity but still worth to check out if you want to see just how far you can push Java to get code that your customer might understand.

Jazoon 2008, Day 0

Just back from the Jazoon 2008, Tutorial day (the day before the actual conference starts). I attended Ted Neward's talk "The Busy Java Developer's Guide to Groovy". As always, the talk was quite a lot of fun even for someone like me, who already knows a lot about Groovy. If you get a chance to attend a talk given by Ted: Go. It's worth it.

In his talk, Ted crammed a pretty complete introduction in the key features of Groovy into three hours. For me, the key points of the talk were these: Productivity and ceremony.

Research shows that all developers roughly write the same amount of code lines per day. The difference between guru developers and the greenhorns is that the gurus write a line once and it works first try and each line achieves much more in respect to reaching the goal than any line of a novice. So in order to boost productivity of a guru, the only way is to use a programming language which can achieve more in a single line of code (and we're talking one statement per line here, obviously). So it's not that the guru writes ten times as much code per day as the novice, the novice just writes every line ten times because every time, (s)he gets something wrong and has to fix it.

The other aspect is ceremony. A lot of code in Java is ceremony. In most cases, the Java compiler knows where a statement ends and it wouldn't need the semicolon. Still it insists that I press the key anyway (and it's two keys on the German or Swiss keyboard). The same goes for many other things in Java which are just there for tradition. It would be simple for the Java compiler to infer the type of an object after if (obj instanceof ...). There really is no need for the cast in the next line. I could find another 10 examples easily.

Groovy does away with all that. It comes at a price, though. If you like to rely on the compiler creating a cozy padded cell for you, checking types and syntax as much as possible, then Groovy is not for you. In Groovy, the idea is that the developer knows exactly what (s)he's doing and all (s)he needs is the fastest path to get there. You can do all the dirty things you wished for in Java but you don't have to. Groovy is an "also" language, not an "either-or" language. If you feel that Java is dirty enough, already, try Scala.

Thanks, Ted.

If you want to know more about Groovy and you're in Zurich, why not attend my talk on Thursday?

Complexity Budget

Are you a human? If not, then this is probably not for you.

If you are, then you have a "complexity budget". I define it a little bit differently than the author of the article. My definition is that you can spend only so much on understanding something. Example. What does this little C program do?

_(__,___,____){___/__<=1?_(__,___+1,____):!(___%__)?_(__,___+1,0):___%__==___/
__&&!____?(printf("%d\t",___/__),_(__,___+1,0)):___%__>1&&___%__<___/__?_(__,1+
___,____+!(___/__%(___%__))):___<__*__?_(__,___+1,____):0;}main(){_(100,0,0);}

It prints the primes below 100. And it blows the complexity budget. C++ also often blows the complexity budget. A friend of mine once said: "To understand C++, you have to be a C++ compiler."

Java once was a simple language but the wise guys with a sun-burn (from being exposed too long to the sun, get it?) decided to do something about it and came up with Generics. And since they weren't sure that this would indeed make the language too complex, they added annotations on top of that. Excellent move!

But it didn't work. There are still too many people who use and understand Java. So they came up with JSR 308 which allows you to use annotations in even more places to write code like this (from this article):

@NotEmpty List<@NonNull String> strings
        = new ArrayList<@NonNull String>();

If that doesn't do it, nothing will.

On a positive side note, in Groovy, I can not only register an annotation processor (AP) in the compiler, I can even manipulate the AST from the AP, allowing to create code like this:

@GroovySQL(type=DemoType.class)
def load(long id) {
    return """select * from demo_table where id = ${id}""";
}

That doesn't return a string but an object of type DemoType filled with the data from the database. Life can be so simple.

And to the guys from Sun: Thanks a lot! We really appreciate your help! :-)

Links: Article by Michael Nygard about JSR 308

What's Wrong With Java, Part 1

As I promised at the Jazoon, I'm starting a series of posts here to present the reasons behind my thoughts on the future on Java. To do this, I'll develop a small knowledge management application with the name Sensei, the Japanese word for "teacher". The final goal is to have three versions in Java, Python an Groovy at the same (or at least a similar) level.

The application will sport the usual features: A database layer, a model and a user interface. Let's start with the model which is fairly simple.

We have knowledge nodes that contain the knowledge (a short name and a longer description), keywords to mark knowledge nodes and relations to connect knowledge nodes. Additionally, we want to be able to organize knowledge nodes in a tree. Here is the UML:

To make it easier to search the model for names, I collect them in the keyword class. So a knowledge node has no String field with the name but a keyword field instead. The same applies to the relations. Here is what the Java model looks like:

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26:

class Keyword { public enum Type { KEYWORD, KNOWLEDGE, RELATION }; static Set<Keyword> allKeywords; Type type; String name; } class Knowledge { Keyword name; String knowledge; Knowledge parent; List<Knowledge> children; Set<Keyword> keywords; } class Relation { Keyword name; Knowledge from; Knowledge to; }

UML of Sensei Model

Note that I omitted all the usual cruft (private fields, getters/setters, equals/hashCode).

This model has some interesting properties:

1. There is a recursive element. Many OR mappers show lots of example how to map a String to a field but for some reason, examples with tree-like structures are scarce.
2. It contains 1:N mappings where N can be 0 (keywords can be attached to knowledge nodes but don't have to be), 1:N mappings where N is always >= 1 (names of knowledge nodes and relations) and N:M mappings (relations between knowledge nodes).
3. It contains a field called "from" which is an SQL keyword.
4. There are sorted and unsorted mappings (children and keywords of a knowledge node).
5. Instances of Keyword must be garbage collected somehow. When I delete the last knowledge node or relation with a certain name, I want the keyword deleted with it. This is not true for pure keywords, though.

To summarize, this is probably the smallest meaningful model you can come up with to test all possible features an OR mapper must support.

Let's have a look at the features we want to support:

1. Searching knowledge nodes by name or keyword or relation
2. Searches should by case insensitive and allowing to search for substrings
3. Adding child nodes to an existing knowledge node
4. Reordering children
5. Moving a child from one parent to another
6. Adding and removing relations between nodes
7. Adding and removing keywords to a knowledge node

In the next installment, we will have a look how Java and Hibernate can cope with these demands.

Building HTML

There are two ways to generate HTML: The right way and the JSP way. Enough has been said about JSP, what's the right way?

The right way is to have a programming language that is flexible enough to merge HTML and code painlessly. Groovy does it this way:

 def builder = new MarkupBuilder ();
 builder.html {
    head {
        title getTitle()
    }
    body {
        genBody (builder)
    }
 }

What is going on here?

First of all, you must know that you can omit several things in Groovy: parentheses or semi-colon, for example. If we add these, the example becomes less readable but better understandable for Java developers:

 def builder = new MarkupBuilder ();
 builder.html () {
    head () {
        title (getTitle());
    };
    body () {
        genBody (builder);
    };
 };

So obviously, in the first line, a method html() is called. Now, you need to know that the code in curly brackets is a closure and that closure is added as the last parameter to the call of the method in front of it. This means the code in the curly brackets is passed into html() and can be executed any time html() wants to do it. It can even invoke the closure several times. In Java, the definition of html would be: html(Closure c).

The same happens with head(), title() (which takes a String argument) and body(). Now where are these methods defined?

Nowhere. MarkupBuilder() is a class which defines a "catch all" method called invokeMethod. Whenever Groovy cannot find the right method to invoke, it will check if a class defines

 Object invokeMethod(String methodName, Object args)

and invoke this method instead. The method gets the name of the original method and all the original parameters in a list.

In our case, MarkupBuilder.invokeMethod() will use the method name as the HTML element name. That's it. A little bit of flexibility in the parser got us a long way towards HTML without making the code unreadable.

Next, we need a flexible way to pass HTML attributes. In Groovy, named arguments are supported:

 void genBody (MarkupBuilder builder) {
     builder.p (style:'font-weight: bold;', 'Impressive.')
 }

This will call MarkupBuilder.invokeMethod() with "p" as the method name and a list consisting of the map with the style and a string. This information will be used to build the element and it's content.

Of course, the builder will stream the output (which is very simple since it can wrap your code when it needs to: essentially it will just warp all the virtual methods you call), there is no way to forget a closing element (your program won't compile anymore) and splitting complex HTML into several methods is a cinch.

Life can be so simple with a little bit of flexibility.

Further reading: Using MarkupBuilder for Agile XML creation

Back from JaZOOn, First Day

During the keynote speeches, someone mentioned that the streetcar no. 5, which goes to Jazoon'07, also goes to the zoo of Zurich. The good news was that the zoo was at the other end of the line, so JaZOOn (which apparently doesn't mean anything) is either not related to the zoo or it's related in such a way that it's the opposite.

I kind of disagree. A zoo is a place where you can see things that you normally can't and in a safe way. In this regard, Jazoon is a zoo full of Java animals and I'm a happy part of it.

Yesterday, I attended the keynote given by Ted Neward which was "Why the Next Five Years Will Be About Languages". He mentioned a lot of interesting things, like:

• Tools to build custom languages (a.k.a DSL's) become more simple, more powerful, more widespread
• The need to use there tools grows because it takes so much to formulate some things in general purpose languages like Java. What was great ten years ago seems clumsy today.
• There is a plethora of languages that run on the Java VM (which is not Java(TM)) like Groovy, JRuby, Jython, Nice, Rhino (see here for a more complete list). He mentioned something like 200 languages using the VM besides Java but my memory could fail me here.

For me, this means that my own talk What's Wrong With Java?, where I compare Java, Groovy and Python, probably wasn't that far off. Of course, I was pretty nervous how people at a Java conference would take it (plus I got sick on the weekend, so I had to take so many drugs to be able to give the talk that, if I had fallen into the Lake of Zurich, they would have had to pump it dry and deposit the water as medical waste ;-) ... anyway ...).

Moreover, I had to rush through my talk because the time limits were really tight. All in all, I felt my performance could have been better but the critics seem fair. (see Fabrizio Gianneschi's comment, thomas and another blog). Also, the room in my talk was full of people; something I haven't seen for any other talk since (which probably means that I attend the wrong ones ;-)) and comments at the show were good, too (but I can't prove it).

I was thinking about registering a BOF but I just don't feel well enough, so I'll expand my thoughts and ideas a bit more here where space and time are only limited by your and my endurance. And you can think about your comments before sharing them with the world. Win-win, I'd say.

Back to Jazoon. After my own talk, I attended Space Based Architecture - Scalable as Google, Simple as Spring. The talk itself was interesting and made sense; unfortunately, my body demanded sleep and it takes what it can't get. So if you want to know any details, ask the person who sat next to me. I can only pray that I didn't snore. Not the fault of the speaker, I swear!

After having restored some of my energy, I went to see Impossible Possibilities - Programming Java an Unusual Way. The presenter, Michael Wiedeking, has the same problem with English as I: Great pronunciation but small built-in dictionary. Still, I could get what he talked about. He presented a way to implement a generator/corouting in Java 5. The basic idea here is that you have a piece of code which can return something to the caller and then continue it's work when the caller calls it again. Confused? Here is an example:

def parse(filename):
    handle = file(filename, 'r')
    while True:
        line = handle.readline()
        if line[0] == '#':
          continue
        yield line
        # <-- If you call parse() a second time, you will be here.
    handle.close()

parse() throws away every line that starts with a hash (#) and returns ("yields") all the rest. The interesting part is when you call parse a second time: It will not start with the line where a new file is opened but it will continue with the next statement after "yield", therefore reading the next line in the already opened file.

In Python, you can have as many yields in one method or function as you like. They work with recursion and exceptions. This way, you can run a complex algorithm until a certain point (when you have a first result to return), return it and then go on as if the return had never happened. You don't have to worry about local variables, closing the file handle, control flow. The language all handles it for you.

If Java, you achieve the same thing with two threads and about four or five hoops to jump through. This is the difference between a modern dynamic language like Python or Ruby: There are completely new ways to do things that are very powerful, simple to understand and (almost) impossible to do in Java.

I spend the rest of the day with Michael Wiedeking and Neil M. Gafter, arguing about checked exceptions until I was to tired that I crept home and went to bed.