Dark Views

Goodbye

2009-02-25T11:24:00.000+01:00

I've moved. You can find my new blog under http://blog.pdark.de/

I'm Moving

2009-02-17T21:48:00.003+01:00

Blogger was nice for the time being. It's a simple interface, easy to use but ... well ... I need a bit more. I'd especially like to know if anyone is ever reading my blog. So I'm moving my blog over to WordPress. They have a nice dashboard with daily access figures. I'll post the new URL here as soon as the move is completed.

A Different View on Exceptions

2009-01-30T10:32:00.002+01:00

The discussion about checked/unchecked exceptions is almost as old as Java. While we all have a point in your stance towards this, maybe we are looking at the problem from the wrong angle. Manuel Woelker wrote an article which concentrates on the receiver of the exception, the user, and how exceptions should behave to help the user: Exceptions From a User’s Perspective

How to Hide a Virus in Source Code

2009-01-28T10:15:00.001+01:00

I've been looking for quite some time for this article: How can you hide a virus in the source code? Basically, you create a binary of a compiler which contains the virus and which is patched to infect other programs as it compiles them.

Reflections on Trusting Trust by Ken Thompson.

One Word: Cute

2009-01-27T15:53:00.000+01:00

Danc's Miraculously Flexible Game Prototyping Tiles Enjoy!

Another Lesson on Performance

2009-01-23T14:16:00.001+01:00

Just another story you can tell someone who fears that "XYZ might be too slow":

I'm toying with the idea to write a new text editor. I mean, I've written my own OS, my own XML parser and I once maintained XDME, an editor written originally by Matthew Dillon. XDME has a couple of bugs and major design flaws that I always wanted to fix but never really got to it. Anyway.

So what is the best data structure for a text editor in 2008? List of lines? Gap-Buffer? Multi-Gap-Buffer?

XDME would keep the text in a list of lines and each line would point to a character array with the actual data. When editing, the characters would be copied into an edit buffer, the changes made and after the edit, the changed characters would be copied back, allocation a new character array if necessary.

This worked, it was a simple design but it had a flaw: it didn't scale. The length of a line was limited to the max size of the edit buffer and loading a huge file took ages because each line was read, analyzed, memory was allocated ... you get the idea.

So I wanted to make it better. Much better. I'd start with reading the file into a big buffer, chopped into evenly sized chunks to make reading both fast and memory efficient (imagine loading a 46MB file into a single memory chunk - after a couple of changes, I'd need to allocate a second 46MB chunk, copy the whole stuff over, etc, needing twice the amount of RAM for a short time).

During the weekend, I mulled the various ideas over, planned, started with a complex red-black tree structure for markers (positions in the text that move when you insert before them). It's huge, complex. It screams "wrong way!"

So today, I sat back and did what I should have done first: Get some figures. How much does it really cost to copy 4MB of RAM? Make a guess. Here is the code to check:

    public static void main (String[] args)
    {
        long start = System.currentTimeMillis ();
        
        int N = 10000;
        for (int i=0; i<N; i++)
        {
            int[] buffer = new int[1024*1024];
            System.arraycopy (buffer, 0, buffer, 1, buffer.length-1);
        }
        
        long duration = System.currentTimeMillis () - start;
        System.out.println (duration);
        System.out.println (duration / N);
    }

On my machine, this prints "135223" and "13". That's thirteen milliseconds to copy 4MB of RAM. Okay. It's obviously not worth to spend a second to think about the cost of moving data around in a big block of bytes.

That leaves the memory issue. I would really like to be able to load and edit a 40MB file in a VM which has 64MB heap. Also, I would like to be effective loading a file with 40MB worth of line-feeds as well as a file which contains just a single line with 40MB data in it.

But this simple test has solved one problem for me: I can keep the lines in an ArrayList for fast access and need not worry too much about performance. The actual character data needs to go into a chunked memory structure, though.

Morale: There is no way to tell the performance of a piece of code by looking at it.

25 Most Dangerous Programming Errors

2009-01-23T14:11:00.000+01:00

If you want to improve your l33t [0d|ng skillz, especially keeping script kiddies off your back, here is a list of the 25 most common coding errors: http://www.sans.org/top25errors/

Sorting for Humans: Natural Sort Order

2009-01-22T13:58:00.002+01:00

Kate Rhodes sums it up nicely:

Silly me, I just figured that alphabetical sorting was such a common need (judging by the number of people asking how to do it I'm not wrong either) that I wouldn't have to write the damn thing. But I didn't count on the stupid factor. Jesus Christ people. You're programmers. You're almost all college graduates and none of you know what the f**k "Alphabetical" means. You should all be ashamed. If any of you are using your language's default sort algorithm, which is almost guaranteed to be ASCIIbetical (for good reason) to get alphabetical sorting you proceed to the nearest mirror and slap yourself repeatedly before returning to your desks and fixing your unit tests that didn't catch this problem.

So if you want to sort your lists the right way (instead of the ASCII way), read this.

The Temporal Void

2009-01-03T14:49:00.003+01:00

Holidays. The only time where I can read or "dream with open eyes" (text from a bookmark). This year, it was "The Temporal Void" by Peter F. Hamilton. It's the sequel to "The Dreaming Void" (my review).

Again, the series is coming along great (which Peter can probably see on your bank account :) Well deserved if you ask me). I like the rich characters, the story is sound and believable. Recommendation: Buy. Now.

There were three spots which I didn't "buy" in "The Temporal Void", places where I dropped from the story and thought "WTF?" Note: Only mild spoilers below; you can read on even if you haven't read the book, yet.

So Aaron is stranded on Hanko, the planet is about to blow up and the Navy scout is about to pick him up. After being warned that he's dangerous, having the best sensors military money can buy, they let him simply walk on their ship battle ready and kill them. I mean, OK, shit happens and maybe these was the Omega ship with the best morons the Navy could find and such ... but ... nah, really :) With instant comm available at all times, no one is watching this important operation? There isn't even a recording? Didn't buy that one.

The same happened in the first part when Aaron broke into the storage vault to claim Inigos memories. Why did you place the guards *inside* (where all that delicate stuff will break if they ever would have to engage someone)? Why not place them on the other side of the vault door where they can pummel any intruder against a foot or two of solid steel, without any cover?
Edeard finds his childhood friend Salrana in the clutches of Ranalee and leaves her there. I never thought he would be the character to leave someone behind. He knows only bad can come from this; I mean it's only the tenth time this happens, he got to learn something, right? If he dragged Salrana away, the girl would be mad but he could leave her with the Pythia and look for a solution if she doesn't know one. If all else fails, he could simply blackmail Ranalee into fixing what she did. So I accept that he's tired and worn out and all that but this just didn't fit.
Paula and the quantumbuster. So this thing really distorts spacetime to wreak havoc with matter. How can she get away when space is so twisted? How about just nailing her in place using the ships in orbit and blowing up the station the traditional way?

Other than that, the story is the usual perfect piece of work from Peter. I've posted the text above in Peter's inbox; should I get a reply, I'll post it here.

Hit Them Harder Until They Learn

2008-12-23T11:20:00.005+01:00

Do you believe that hitting people harder will make them learn? No? Do you believe that prison will turn a bad person into a good citizen? If not, why don't you do something about it?

People like simple solutions. Out of the eyes, problem solved. Lock someone up, it will turn them into a nice fellow. Because you don't like to be locked up, you believe that others feel the same. The question remains: Why should locking up someone turn them into a better person?

Well, it does work ... for monks. They willingly lock themselves up somewhere to be able to concentrate on those things that are dear to them and where they want to improve. But there is a big difference between locking up yourself (like a hermit) and locking up someone with a bunch of criminals and guards. Just make a wild guess where most criminals learn their trade ...

The relapse rates vary a lot (by age, type of crime, prison, treatment). Still, I'm more astonished by the fact that not all of the inmates become criminal again rather than the fact that some do.

That said, it is refreshing when you meet people who don't fall for the temptation of retribution. The guys at stackoverflow.com recently discovered that some people gamed the system to hurt other members or to increase their status. Instead of punishing them, they decided to just take away any advantage of trying to game the system. Problem really solved. Carry on, commander!

Links: Vote Fraud And You
Ein Gefängnisersatz mit Rückfallquote null (German)
GEFÄNGNIS - LEBEN HINTER GITTERN (German)

Holding a Program in One's Head

2008-12-18T11:50:00.001+01:00

In my perpetual search for brain food for programmers, I've found this article: Holding a Program in One's Head

Obama Goes Creative Commons

2008-12-02T17:37:00.004+01:00

If you've been living under a stone for the past few months, the next president of the USA, Barack Obama, had his team build a web site where he shares his thoughts, ideas and plans. That is itself probably constitutes a revolution but it gets better: You can talk to these guys. Or rather send them your ideas, hopes and worries. And it seems these really count. I mean, how much better can it get?

It can: The site and all content is under the Create Commons license CC-BY which basically means the content is free as long as you say where you got it from (read the license for details). Amazing :)

Thanks for the nice Christmas present, Mr. Soon-President! It really makes me happy to know that there is finally someone who gets the Internet.

Links: Lawrence Lessig's Blob (he's the inventor of CC, just in case).

Stackoverflow: Reputation over 1000 :)

2008-12-02T14:36:00.001+01:00

Just a tiny post to cheer the fact that my reputation on Stackoverflow.com has transcended 1,000. Yay!

Writing Testable Code

2008-12-01T17:21:00.002+01:00

Just stumbled over this article: "Writing Testable Code". Apparently, it's a set of rules which Google uses. While I'm not a 100% fan of Google, this is something every developer should read and understand.

How To Be Agile

2008-11-29T10:40:00.000+01:00

The article "When Agile Projects Go Bad" got me thinking. I've talked to many people about XP and Agile Development and TDD and the usual question is: "How do we make it work?" And the next sentence is: "This won't work with us because we can't do this or that.".

This is a general misconception which comes from the ... uh ... "great" methodologies which you were taught in school: the waterfall model, the V model, the old dinosaurs. They told you: "You must follow the rules to the letter or doom will rain on your head!" Since you could never follow all the rules, they could easily say "Told you so!" when things didn't work out.

Agile development is quite different in this respect. First of all, it assumes that you're an adult. That you have a brain and can actually use it. It also assumes that you want to improve your situation. It also assumes nothing else.

When a company is in trouble, it will call for help. Expensive external advisers will be called, they will think about the situation for a long time (= more money for them). After a while (when the new yacht is in the dry), they will come up with what's wrong and how to fix it. Did you know that in most companies in trouble, the external advisers will just repeat what they heard form the people working there?

It's not that people don't know what's wrong, it's just not healthy to mention it ... at least if you want to work there. So people walk around, with the anger in their hearts and the fist in the pocket and nothing will happen until someone from the outside comes in and states the obvious. Can't happen any other way because if it could, you wouldn't be in this situation in the first place.

Agile Development is similar. It acknowledges that you're smart and that you know what's wrong and that you don't have the power to call in help. What it does is it offers you a set of tools, things that have worked for other people in the past and some of them might apply to you. Maybe all. Probably not. Most likely, you will be able to use one or two. That doesn't sound like much but the old methodologies are pretty useless if you can't implement 90%+. Agile is agile. It can bend and twist and fit in your routine.

So you're thinking about doing TDD. Do you have to ask your boss? No. Do you have to get permission from anyone? No. Do you have to tell anyone? No. Can you do it any time you like, as often as you like, stop at will? Yes. If it doesn't work for you in your situation, for the current project, then don't use it. No harm done, nothing gained either.

But if you can use it, every little bit will help. Suddenly, you will find yourself to be able to deliver on time. Your code will work and it will be much more solid than before. You will be able to do more work in less time. People will notice. Your reputation will increase. And eventually, they will be curious: How do you do it? "TDD." What's that?

You win.

Be agile. Pick and choose. Pick what you think will work, try it, drop it if it doesn't deliver. And if it works, try the next thing. Evolve. Become the better you.

Agile is not a silver bullet. It won't miraculously solve all your issues. You still have to think and be an adult about your work. It's meant to be that way. I don't do every Agile practice every day. Sometimes, I don't even TDD (and I regret every time). But I always return because life is just so much more simple.

Space: Not So Black And Empty After All

2008-11-28T14:41:00.001+01:00

If you always wanted to know what NASA does with all the billions of dollars spent, here are some images.

Navigating SharePoint Folders With Axis2

2008-11-26T15:02:00.006+01:00

I've just written some test code to get a list of items in a SharePoint folder with Apache Axis2 and since this was "not so easy", I'll share my insights here.

First, you need Axis2. If you're using Maven2, put this in your pom.xml:

    <dependency>
        <groupId>org.apache.axis2</groupId>
        <artifactId>axis2-kernel</artifactId>
        <version>1.4.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.axis2</groupId>
        <artifactId>axis2-adb</artifactId>
        <version>1.4.1</version>
    </dependency>

Next stop: Setting up NTLM authorization.

import org.apache.axis2.transport.http.HttpTransportProperties;
import org.apache.commons.httpclient.auth.AuthPolicy;

        HttpTransportProperties.Authenticator auth = new
            HttpTransportProperties.Authenticator();
        auth.setUsername ("username");
        auth.setPassword "password");
        auth.setDomain ("ntdom");
        auth.setHost ("host.domain.com");

        List authPrefs = new ArrayList (1);
        authPrefs.add (AuthPolicy.NTLM);
        auth.setAuthSchemes (authPrefs);

This should be the username/password you're using to login to the NT domain "ntdom" on the NT domain server "host.domain.com". Often, this server is the same as the SharePoint server you want to connect to.

If the SharePoint server is somewhere outside your intranet, you may need to specify a proxy:

        HttpTransportProperties.ProxyProperties proxyProperties =
            new HttpTransportProperties.ProxyProperties();
        proxyProperties.setProxyName ("your.proxy.com");
        proxyProperties.setProxyPort (8888);

You can get these values from your Internet browser.

If there are several SharePoint "sites" on the server, set site to the relative URL of the site you want to connect to. Otherwise, leave site empty. If you have no idea what I'm talking about, browse the SharePoint server in Internet Explorer. In the location bar, you'll see an URL like this: https://sp.company.com/projects/demo/Documents2/Forms/AllItems.aspx?RootFolder=%2fprojects%2fdemo%2fDocument2%2f&FolderCTID=&View=%7b18698D80%2dE081%2d4BBE%2d96EB%2d73BA839230B9%7d. Scary, huh? Let's take it apart:

https:// = the protocol,
sp.company.com = The server name (with domain),
projects/demo = The "site" name
Documents2 = A "list" stored on the site "projects/demo"
/Forms/AllItems.aspx?RootFolder=... is stuff to make IE happy. Ignore it.

So in out example, we have to set site to:

        String site = "/projects/demo";

Mind the leading slash!

To verify that this is correct, replace "/Documents2/Forms/" and anything beyond with "/_vti_bin/Lists.asmx?WSDL". That should return the WSDL definition for this site. Save the result as "sharepoint.wsdl" (File menu, "Save as..."). Install Axis2, open a command prompt in the directory where you saved the WSDL file and run this command (don't forget to replace the Java package name):

%AXIS2_HOME%\bin\WSDL2Java -uri sharepoint.wsdl -p java.package.name -d adb -s

This will create a "src" directory with the Java package and a single file "ListsStub.java". Copy it into your Maven2 project.

Now, we can get a list of the lists on the site:

        ListsStub lists = new ListsStub
            ("https://sp.company.com"+site+"/_vti_bin/Lists.asmx");
        lists._getServiceClient ().getOptions ()
            .setProperty (HTTPConstants.AUTHENTICATE, auth);

If you need a proxy, specify it here:

        options.setProperty (HTTPConstants.HTTP_PROTOCOL_VERSION,
            HTTPConstants.HEADER_PROTOCOL_10);
        options.setProperty (HTTPConstants.PROXY, proxyProperties);

We need to reduce the HTTP protocol version to 1.0 because most proxies don't allow to send multiple requests over a single connection. If you want to speed things up, you can try to comment out this line but be prepared to see it fail afterwards.

Okay. The plumbing is in place. Now we query the server for the lists it has:

        String liste = "Documents2";
        String document2ID;
        {
            ListsStub.GetListCollection req = new ListsStub.GetListCollection();
            ListsStub.GetListCollectionResponse res = lists.GetListCollection (req);
            displayResult (req, res);
            
            document2ID = getIDByTitle (res, liste);
        }

This downloads all lists defined on the server and searches for the one we need. If you're in doubt what the name of the list might be: Check the bread crumbs in the blue part in the intern explorer. The first two items are the title of the site and the list you're currently in.

displayResult() is the usual XML dump code:

    private void displayResult (GetListCollection req,
            GetListCollectionResponse res)
    {
        System.out.println ("Result OK: "
                +res.localGetListCollectionResultTracker);
        OMElement root = res.getGetListCollectionResult ()
                .getExtraElement ();
        dump (System.out, root, 0);
    }

    private void dump (PrintStream out, OMElement e, int indent)
    {
        indent(out, indent);
        out.print (e.getLocalName ());
        for (Iterator iter = e.getAllAttributes (); iter.hasNext (); )
        {
            OMAttribute attr = (OMAttribute)iter.next ();
            out.print (" ");
            out.print (attr.getLocalName ());
            out.print ("=\"");
            out.print (attr.getAttributeValue ());
            out.print ("\"");
        }
        out.println ();
        
        for (Iterator iter = e.getChildElements (); iter.hasNext (); )
        {
            OMElement child = (OMElement)iter.next ();
            dump (out, child, indent+1);
        }
    }

    private void indent (PrintStream out, int indent)
    {
        for (int i=0; i<indent; i++)
            out.print ("    ");
    }

We also need getIDByTitle() to search for the ID of a SparePoint list:

    private String getIDByTitle (GetListCollectionResponse res, String title)
    {
        OMElement root = res.getGetListCollectionResult ().getExtraElement ();
        QName qnameTitle = new QName ("Title");
        QName qnameID = new QName ("ID");
        for (Iterator iter = root.getChildrenWithLocalName ("List"); iter.hasNext (); )
        {
            OMElement list = (OMElement)iter.next ();
            if (title.equals (list.getAttributeValue (qnameTitle)))
                return list.getAttributeValue (qnameID);
        }
        return null;
    }

With that, we can finally list the items in a folder:

        {
            String dir = "folder/subfolder";

            ListsStub.GetListItems req
                = new ListsStub.GetListItems ();
            req.setListName (document2ID);
            QueryOptions_type1 query
                = new QueryOptions_type1 ();
            OMFactory fac = OMAbstractFactory.getOMFactory();
            OMElement root = fac.createOMElement (
                new QName("", "QueryOptions"));
            query.setExtraElement (root);

            OMElement folder = fac.createOMElement (
                new QName("", "Folder"));
            root.addChild (folder);
            folder.setText (liste+"/"+dir); // <--!!

            req.setQueryOptions (query);
            GetListItemsResponse res = lists.GetListItems (req);
            displayResult (req, res);
        }

The important bits here are: To list the items in a folder, you must include the name of the list in the "Folder" element! For reference, this is the XML which actually sent to the server:

<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
    <soapenv:Body>
        <ns1:GetListItems xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/">
            <ns1:listName>{12AF2346-CCA1-486D-BE3C-82223DEC3F42}</ns1:listName>
            <ns1:queryOptions>
                <QueryOptions>
                    <Folder>Documents2/folder/subfolder</Folder>
                </QueryOptions>
            </ns1:queryOptions>
        </ns1:GetListItems>
    </soapenv:Body>
</soapenv:Envelope>

If the folder name is not correct, you'll get a list of all files and folders that the SharePoint server can find anywhere. The folder names can be found in the bread crumbs. The first two items are the site and the list name, respectively, followed by the folder names.

The last missing piece is displayResult() for the items:

    private void displayResult (GetListItems req,
         GetListItemsResponse res)
    {
        System.out.println ("Result OK: "
                +res.localGetListItemsResultTracker);
        OMElement root = res.getGetListItemsResult ()
                .getExtraElement ();
        dump (System.out, root, 0);
    }

If you run this code and you see the exception "unable to find valid certification path to requested target", this article will help.

If the SharePoint server returns an error, you'll see "detail unsupported element in SOAPFault element". I haven't found a way to work around this bug in Axis2. Try to set the log level of "org.apache.axis2" to "DEBUG" and you'll see what the SharePoint server sent back (not that it will help in most of the cases ...)

Links: GetListItems on MSDN, How to configure Axis2 to support Basic, NTLM and Proxy authentication?, Java to SharePoint Integration - Part I (old, for Java 1.4)

Good luck!

"Hunderte von Milliarden" auf Perry-Rhodan.net

2008-11-19T21:38:00.003+01:00

I've published a story :) Since the story is in German, this post is, too.

Ich gebe es zu, ich bin ein Perry Rhodan Fan. Nicht nur, weil es die grösste SciFi-Serie der Welt ist (mit inzwischen 2466 Heften à 64 Seiten jede Woche, seit nunmehr fast 50 Jahren! Die aktuellen Ereignisse um Roi Danton und Dantyren haben mich so lange beschäftigt, bis ich eine Geschichte zu Papier (oder in diesem Fall zu PDF) bringen musste.

Arndt Ellmer war so freundlich sie in der LKS Galerie auf der Homepage von Perry Rhodan zu platzieren. Der Titel ist "Hunderte von Milliarden" und enthält meine Interpretation von Aussagen wie "Der Erbe des Universums".

Viel Vergnügen!

Feedback ist erwünscht. Entweder als Kommentar anhängen oder per eine Mail (digulla at hepe dot com bzw. dark at pdark dot de).

Stuck? Ask Stack Overflow

2008-11-19T17:22:00.002+01:00

Stuck with a hard programming problem? Just solved an impossible problem and want to show the world your genius? Don't know how to solve a problem with your favorite OS or programming language? Check out stackoverflow.com.

Testing the Impossible: Rules of Thumb

2008-11-19T14:46:00.003+01:00

When people say "we can't test that", they usually mean "... with a reasonable effort". They say "we can't test that because it's using a database" or "we can't test the layout of the UI" or "to test this, we need information which is buried in private fields of that class".

And they are always wrong. You can test everything. Usually with a reasonable effort. But often, you need to take a step back and do the unusual. Some examples.

So your app is pumping lots of data into a database. You can't test the database. You'd need to scrap it for every test run and build it from scratch which would take hours or at least ages. Okay. Don't test the database. Test how you use it. You're not looking for bugs in the database, you're looking for bugs in your code. Saying "but some bugs might get away" is just a lame excuse.

Here is what you need to do: Identify independent objects (which need no other objects stored in the database). Write tests for those. Put the test data for them in an in-memory database. HSQLDB and Derby are your friends. If you must, use your production database but make the schema configurable. Scrap the tables before the test and load them from clean template tables.

So you need some really spiffy SQL extensions? Put them in an isolated place and test them without everything else against the real database. You need to test that searching a huge amount of data works? Put that data in a static test database. Switch database connections during the tests. Can't? Make that damn connection provider configurable at runtime! Can't? Sure you can. If everything else fails, get the source with JAD, compile that into an independent jar and force that as the first thing into the classpath when you run your tests. Use a custom classloader if you must.

While this is not perfect, it will allow you to learn how to test. How to test your work. Testing is always different just like every program is different. Allow yourself to make mistakes and to learn from them. Tackle the harder problems after the easier ones. Make the tests help you learn.

So you have this very complex user interface. Which you can't test. Let alone starting the app takes ten minutes and the UI changes all the time and ... Okay. Stop the whining. Your program is running on a computer and for same inputs, a computer should return the same outputs, right? Or did you just build a big random number generator? Something to challenge the Infinite Improbability Drive? No? Then you can test it. Follow me.

First, cut the code that does something from the code that connects said code to the UI. As a first simple step, we'll just assume that pressing a button will actually invoke your method. If this fails for some reason, that reason can't be very hard to find, so we can safely ignore these simple bugs for now.

After this change, you have the code that does stuff at the scruff. Now, you can write tests for it. Reduce entanglement. Keep separate issues separate. A friend of mine builds all his code around a central event service. Service providers register themselves and other parts of the code send events to do stuff. It costs a bit performance but it makes testing as easy as overwriting an existing service provider with a mock up.

Your software needs an insanely complex remote server? How about replacing this with a small proxy that always returns the same answers? Or at least fakes something that looks close enough to a real answer to make your code work (or fail when you're testing the error handling).

And if you need data that some stubborn object won't reveal, use the source, Luke (download the source and edit the offender to make the field public, remove "final" from all files, add a getter or make it protected and extend the class in the tests). If everything else fails, turn to java.lang.reflect.Field.setAccessible(true).

If you're using C/C++, always invoke methods via a trampoline: Put a pointer somewhere which contains the function to call and always use that pointer instead of the real function. Use header files and macros so no human can tell the difference. In your tests, bend those pointers. The Amiga did it in 1985. #ifdef is your friend.

If you're using some other language, put the test code in comments and have a self-written preprocessor create two versions that you can compile and run.

If all else fails, switch to Python.

Testing the Impossible: JavaScript in a Web Page

2008-11-18T11:15:00.006+01:00

How do you run JUnit tests on JavaScript in a web page? Impossible?

Here is what you need: First, get a copy of Rhino (at least 1.6R7). Then, save a copy of the JavaScript code at the bottom as "env.js". And here is the setup code for the JUnit test:

    Context cx;
    Global scope;

    public void setupContext () throws IllegalAccessException,
            InstantiationException, InvocationTargetException
    {
        cx = Context.enter();
        scope = new Global();
        scope.init (cx);

        addScript(cx, scope, new File ("html/env.js"));

        File f = new File ("html/demo.html");
        cx.evaluateString(scope, 
                "window.location = '"+f.toURL()+"';\n" +
      "", "<"+getName ()+">", 1, null);
    }

    public void addScript (Context cx, Scriptable scope, File file) throws IOException
    {
        Reader in = new FileReader (file);
        cx.evaluateReader(scope, in, file.getAbsolutePath(), 1, null);
    }

This will load "demo.html" into the browser simulation. The problem here: The loading is asynchronous (just like in a real browser). Now what? We need synchronization:

import org.mozilla.javascript.ScriptableObject;

public class JSJSynchronize extends ScriptableObject
{
    public Object data;
    public Object lock = new Object ();
    
    public JSJSynchronize()
    {
    }
    
    @Override
    public String getClassName ()
    {
        return "JSJSynchronize";
    }
    
    public Object jsGet_data()
    {
        synchronized (lock)
        {
            try
            {
                lock.wait ();
            }
            catch (InterruptedException e)
            {
                throw new RuntimeException ("Should not happen", e);
            }
            
            return data;
        }
    }

    public void jsSet_data(Object data)
    {
        synchronized (lock)
        {
            this.data = data;
            lock.notify ();
        }
    }
    
    public Object getData()
    {
        synchronized (lock)
        {
            try
            {
                lock.wait ();
            }
            catch (InterruptedException e)
            {
                throw new RuntimeException ("Should not happen", e);
            }
            
            return data;
        }
    }

    public void setData(Object data)
    {
        synchronized (lock)
        {
            this.data = data;
            lock.notify ();
        }
    }
    
}

With this code and "window.onload", we can wait for the html to load:

        JSJSynchronize jsjSynchronize;
        ScriptableObject.defineClass(scope, JSJSynchronize.class);
        
        jsjSynchronize = (JSJSynchronize)cx.newObject (scope, "JSJSynchronize");
        scope.put("jsjSynchronize", scope, jsjSynchronize);

        cx.evaluateString(scope, 
                "window.location = '"+f.toURL()+"';\n" +
      "window.onload = function(){\n" +
      "    print('Window loaded');\n" +
      "    jsjSynchronize.data = window;\n" +
      "};\n" +
      "", "<"+getName ()+">", 1, null);

        ScriptableObject window = (ScriptableObject)jsjSynchronize.getData();
        System.out.println ("window="+window);
        ScriptableObject document = (ScriptableObject)scope.get ("document", scope);
        System.out.println ("document="+document);
        System.out.println ("document.forms="+document.get ("forms", document));
        ScriptableObject navigator = (ScriptableObject)scope.get ("navigator", scope);
        System.out.println ("navigator="+navigator);
        System.out.println ("navigator.location="+navigator.get ("location", navigator));

        // I've been too lazy to parse the HTML for the scripts:
        addScript(cx, scope, new File ("src/main/webapp/script/prototype.js"));

Slightly modified version of env.js, original by John Resig (original code):

/*
 * Simulated browser environment for Rhino
 *   By John Resig <http://ejohn.org/>
 * Copyright 2007 John Resig, under the MIT License
 * http://jqueryjs.googlecode.com/svn/trunk/jquery/build/runtest/
 * Revision 5251
 */

// The window Object
var window = this;

// generic enumeration
Function.prototype.forEach = function(object, block, context) {
 for (var key in object) {
  if (typeof this.prototype[key] == "undefined") {
   block.call(context, object[key], key, object);
  }
 }
};

// globally resolve forEach enumeration
var forEach = function(object, block, context) {
 if (object) {
  var resolve = Object; // default
  if (object instanceof Function) {
   // functions have a "length" property
   resolve = Function;
  } else if (object.forEach instanceof Function) {
   // the object implements a custom forEach method so use that
   object.forEach(block, context);
   return;
  } else if (typeof object.length == "number") {
   // the object is array-like
   resolve = Array;
  }
  resolve.forEach(object, block, context);
 }
};

function collectForms(document) {
 var result = document.body.getElementsByTagName('form');
 //print('collectForms');
 document.forms = result;
  
 for (var i=0; i<result.length; i++) {
     var f = result[i];
     f.name = f.attributes['name'];
     //print('Form '+f.name);
     document[f.name] = f;
     f.elements = f.getElementsByTagName('input');
     
     for(var j=0; j<f.elements.length; j++) {
         var e = f.elements[j];
         var attr = e.attributes;
         
         //forEach(attr, print);
         e.type = attr['type'];
         e.name = attr['name'];
         e.className = attr['class'];
         
         f[e.name] = e;
  //print('    Input '+e.name);
     }
 }
}

(function(){

 // Browser Navigator

 window.navigator = {
  get userAgent(){
   return "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3";
  },
  get appVersion(){
   return "Mozilla/5.0";
  }
 };
 
 var curLocation = (new java.io.File("./")).toURL();
 
 window.__defineSetter__("location", function(url){
  var xhr = new XMLHttpRequest();
  xhr.open("GET", url);
  xhr.onreadystatechange = function(){
   curLocation = new java.net.URL( curLocation, url );
   window.document = xhr.responseXML;
   collectForms(window.document);

   var event = document.createEvent();
   event.initEvent("load");
   window.dispatchEvent( event );
  };
  xhr.send();
 });
 
 window.__defineGetter__("location", function(url){
  return {
   get protocol(){
    return curLocation.getProtocol() + ":";
   },
   get href(){
    return curLocation.toString();
   },
   toString: function(){
    return this.href;
   }
  };
 });
 
 // Timers

 var timers = [];
 
 window.setTimeout = function(fn, time){
  var num;
  return num = setInterval(function(){
   fn();
   clearInterval(num);
  }, time);
 };
 
 window.setInterval = function(fn, time){
  var num = timers.length;
  
  timers[num] = new java.lang.Thread(new java.lang.Runnable({
   run: function(){
    while (true){
     java.lang.Thread.currentThread().sleep(time);
     fn();
    }
   }
  }));
  
  timers[num].start();
 
  return num;
 };
 
 window.clearInterval = function(num){
  if ( timers[num] ) {
   timers[num].stop();
   delete timers[num];
  }
 };
 
 // Window Events
 
 var events = [{}];

 window.addEventListener = function(type, fn){
  if ( !this.uuid || this == window ) {
   this.uuid = events.length;
   events[this.uuid] = {};
  }
    
  if ( !events[this.uuid][type] )
   events[this.uuid][type] = [];
  
  if ( events[this.uuid][type].indexOf( fn ) < 0 )
   events[this.uuid][type].push( fn );
 };
 
 window.removeEventListener = function(type, fn){
    if ( !this.uuid || this == window ) {
        this.uuid = events.length;
        events[this.uuid] = {};
    }
    
    if ( !events[this.uuid][type] )
   events[this.uuid][type] = [];
   
  events[this.uuid][type] =
   events[this.uuid][type].filter(function(f){
    return f != fn;
   });
 };
 
 window.dispatchEvent = function(event){
  if ( event.type ) {
   if ( this.uuid && events[this.uuid][event.type] ) {
    var self = this;
   
    events[this.uuid][event.type].forEach(function(fn){
     fn.call( self, event );
    });
   }
   
   if ( this["on" + event.type] )
    this["on" + event.type].call( self, event );
  }
 };
 
 // DOM Document
 
 window.DOMDocument = function(file){
  this._file = file;
  var factory = Packages.javax.xml.parsers.DocumentBuilderFactory.newInstance();
  factory.setValidating(false);
  this._dom = factory.newDocumentBuilder().parse(file);
  
  if ( !obj_nodes.containsKey( this._dom ) )
   obj_nodes.put( this._dom, this );
 };
 
 DOMDocument.prototype = {
  createTextNode: function(text){
   return makeNode( this._dom.createTextNode(
    text.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;")) );
  },
  createElement: function(name){
   return makeNode( this._dom.createElement(name.toLowerCase()) );
  },
  getElementsByTagName: function(name){
   return new DOMNodeList( this._dom.getElementsByTagName(
    name.toLowerCase()) );
  },
  getElementById: function(id){
   var elems = this._dom.getElementsByTagName("*");
   
   for ( var i = 0; i < elems.length; i++ ) {
    var elem = elems.item(i);
    if ( elem.getAttribute("id") == id )
     return makeNode(elem);
   }
   
   return null;
  },
  get body(){
   return this.getElementsByTagName("body")[0];
  },
  get documentElement(){
   return makeNode( this._dom.getDocumentElement() );
  },
  get ownerDocument(){
   return null;
  },
  addEventListener: window.addEventListener,
  removeEventListener: window.removeEventListener,
  dispatchEvent: window.dispatchEvent,
  get nodeName() {
   return "#document";
  },
  importNode: function(node, deep){
   return makeNode( this._dom.importNode(node._dom, deep) );
  },
  toString: function(){
   return "Document" + (typeof this._file == "string" ?
    ": " + this._file : "");
  },
  get innerHTML(){
   return this.documentElement.outerHTML;
  },
  
  get defaultView(){
   return {
    getComputedStyle: function(elem){
     return {
      getPropertyValue: function(prop){
       prop = prop.replace(/\-(\w)/g,function(m,c){
        return c.toUpperCase();
       });
       var val = elem.style[prop];
       
       if ( prop == "opacity" && val == "" )
        val = "1";
        
       return val;
      }
     };
    }
   };
  },
  
  createEvent: function(){
   return {
    type: "",
    initEvent: function(type){
     this.type = type;
    }
   };
  }
 };
 
 function getDocument(node){
  return obj_nodes.get(node);
 }
 
 // DOM NodeList
 
 window.DOMNodeList = function(list){
  this._dom = list;
  this.length = list.getLength();
  
  for ( var i = 0; i < this.length; i++ ) {
   var node = list.item(i);
   this[i] = makeNode( node );
  }
 };
 
 DOMNodeList.prototype = {
  toString: function(){
   return "[ " +
    Array.prototype.join.call( this, ", " ) + " ]";
  },
  get outerHTML(){
   return Array.prototype.map.call(
    this, function(node){return node.outerHTML;}).join('');
  }
 };
 
 // DOM Node
 
 window.DOMNode = function(node){
  this._dom = node;
 };
 
 DOMNode.prototype = {
  get nodeType(){
   return this._dom.getNodeType();
  },
  get nodeValue(){
   return this._dom.getNodeValue();
  },
  get nodeName() {
   return this._dom.getNodeName();
  },
  cloneNode: function(deep){
   return makeNode( this._dom.cloneNode(deep) );
  },
  get ownerDocument(){
   return getDocument( this._dom.ownerDocument );
  },
  get documentElement(){
   return makeNode( this._dom.documentElement );
  },
  get parentNode() {
   return makeNode( this._dom.getParentNode() );
  },
  get nextSibling() {
   return makeNode( this._dom.getNextSibling() );
  },
  get previousSibling() {
   return makeNode( this._dom.getPreviousSibling() );
  },
  toString: function(){
   return '"' + this.nodeValue + '"';
  },
  get outerHTML(){
   return this.nodeValue;
  }
 };

 // DOM Element

 window.DOMElement = function(elem){
  this._dom = elem;
  this.style = {
   get opacity(){ return this._opacity; },
   set opacity(val){ this._opacity = val + ""; }
  };
  
  // Load CSS info
  var styles = (this.getAttribute("style") || "").split(/\s*;\s*/);
  
  for ( var i = 0; i < styles.length; i++ ) {
   var style = styles[i].split(/\s*:\s*/);
   if ( style.length == 2 )
    this.style[ style[0] ] = style[1];
  }
 };
 
 DOMElement.prototype = extend( new DOMNode(), {
  get nodeName(){
   return this.tagName.toUpperCase();
  },
  get tagName(){
   return this._dom.getTagName();
  },
  toString: function(){
   return "<" + this.tagName + (this.id ? "#" + this.id : "" ) + ">";
  },
  get outerHTML(){
   var ret = "<" + this.tagName, attr = this.attributes;
   
   for ( var i in attr )
    ret += " " + i + "='" + attr[i] + "'";
    
   if ( this.childNodes.length || this.nodeName == "SCRIPT" )
    ret += ">" + this.childNodes.outerHTML + 
     "</" + this.tagName + ">";
   else
    ret += "/>";
   
   return ret;
  },
  
  get attributes(){
   var attr = {}, attrs = this._dom.getAttributes();
   
   for ( var i = 0; i < attrs.getLength(); i++ )
    attr[ attrs.item(i).nodeName ] = attrs.item(i).nodeValue;
    
   return attr;
  },
  
  get innerHTML(){
   return this.childNodes.outerHTML; 
  },
  set innerHTML(html){
   html = html.replace(/<\/?([A-Z]+)/g, function(m){
    return m.toLowerCase();
   });
   
   var nodes = this.ownerDocument.importNode(
    new DOMDocument( new java.io.ByteArrayInputStream(
     (new java.lang.String("<wrap>" + html + "</wrap>"))
      .getBytes("UTF8"))).documentElement, true).childNodes;
    
   while (this.firstChild)
    this.removeChild( this.firstChild );
   
   for ( var i = 0; i < nodes.length; i++ )
    this.appendChild( nodes[i] );
  },
  
  get textContent(){
   return nav(this.childNodes);
   
   function nav(nodes){
    var str = "";
    for ( var i = 0; i < nodes.length; i++ )
     if ( nodes[i].nodeType == 3 )
      str += nodes[i].nodeValue;
     else if ( nodes[i].nodeType == 1 )
      str += nav(nodes[i].childNodes);
    return str;
   }
  },
  set textContent(text){
   while (this.firstChild)
    this.removeChild( this.firstChild );
   this.appendChild( this.ownerDocument.createTextNode(text));
  },
  
  style: {},
  clientHeight: 0,
  clientWidth: 0,
  offsetHeight: 0,
  offsetWidth: 0,
  
  get disabled() {
   var val = this.getAttribute("disabled");
   return val != "false" && !!val;
  },
  set disabled(val) { return this.setAttribute("disabled",val); },
  
  get checked() {
   var val = this.getAttribute("checked");
   return val != "false" && !!val;
  },
  set checked(val) { return this.setAttribute("checked",val); },
  
  get selected() {
   if ( !this._selectDone ) {
    this._selectDone = true;
    
    if ( this.nodeName == "OPTION" && !this.parentNode.getAttribute("multiple") ) {
     var opt = this.parentNode.getElementsByTagName("option");
     
     if ( this == opt[0] ) {
      var select = true;
      
      for ( var i = 1; i < opt.length; i++ )
       if ( opt[i].selected ) {
        select = false;
        break;
       }
       
      if ( select )
       this.selected = true;
     }
    }
   }
   
   var val = this.getAttribute("selected");
   return val != "false" && !!val;
  },
  set selected(val) { return this.setAttribute("selected",val); },

  get className() { return this.getAttribute("class") || ""; },
  set className(val) {
   if (typeof val != 'string') { val = "" + val; }
   return this.setAttribute("class",
    val.replace(/(^\s*|\s*$)/g,""));
  },
  
  get type() { return this.getAttribute("type") || ""; },
  set type(val) { return this.setAttribute("type",val); },
  
  get value() { return this.getAttribute("value") || ""; },
  set value(val) { return this.setAttribute("value",val); },
  
  get src() { return this.getAttribute("src") || ""; },
  set src(val) { return this.setAttribute("src",val); },
  
  get id() { return this.getAttribute("id") || ""; },
  set id(val) { return this.setAttribute("id",val); },
  
  getAttribute: function(name){
   return this._dom.hasAttribute(name) ?
    new String( this._dom.getAttribute(name) ) :
    null;
  },
  setAttribute: function(name,value){
   this._dom.setAttribute(name,value);
  },
  removeAttribute: function(name){
   this._dom.removeAttribute(name);
  },
  
  get childNodes(){
   return new DOMNodeList( this._dom.getChildNodes() );
  },
  get firstChild(){
   return makeNode( this._dom.getFirstChild() );
  },
  get lastChild(){
   return makeNode( this._dom.getLastChild() );
  },
  appendChild: function(node){
   this._dom.appendChild( node._dom );
  },
  insertBefore: function(node,before){
   this._dom.insertBefore( node._dom, before ? before._dom : before );
  },
  removeChild: function(node){
   this._dom.removeChild( node._dom );
  },

  getElementsByTagName: DOMDocument.prototype.getElementsByTagName,
  
  addEventListener: window.addEventListener,
  removeEventListener: window.removeEventListener,
  dispatchEvent: window.dispatchEvent,
  
  click: function(){
   var event = document.createEvent();
   event.initEvent("click");
   this.dispatchEvent(event);
  },
  submit: function(){
   var event = document.createEvent();
   event.initEvent("submit");
   this.dispatchEvent(event);
  },
  focus: function(){
   var event = document.createEvent();
   event.initEvent("focus");
   this.dispatchEvent(event);
  },
  blur: function(){
   var event = document.createEvent();
   event.initEvent("blur");
   this.dispatchEvent(event);
  },
  get elements(){
   return this.getElementsByTagName("*");
  },
  get contentWindow(){
   return this.nodeName == "IFRAME" ? {
    document: this.contentDocument
   } : null;
  },
  get contentDocument(){
   if ( this.nodeName == "IFRAME" ) {
    if ( !this._doc )
     this._doc = new DOMDocument(
      new java.io.ByteArrayInputStream((new java.lang.String(
      "<html><head><title></title></head><body></body></html>"))
      .getBytes("UTF8")));
    return this._doc;
   } else
    return null;
  }
 });
 
 // Helper method for extending one object with another
 
 function extend(a,b) {
  for ( var i in b ) {
   var g = b.__lookupGetter__(i), s = b.__lookupSetter__(i);
   
   if ( g || s ) {
    if ( g )
     a.__defineGetter__(i, g);
    if ( s )
     a.__defineSetter__(i, s);
   } else
    a[i] = b[i];
  }
  return a;
 }
 
 // Helper method for generating the right
 // DOM objects based upon the type
 
 var obj_nodes = new java.util.HashMap();
 
 function makeNode(node){
  if ( node ) {
   if ( !obj_nodes.containsKey( node ) )
    obj_nodes.put( node, node.getNodeType() == 
     Packages.org.w3c.dom.Node.ELEMENT_NODE ?
      new DOMElement( node ) : new DOMNode( node ) );
   
   return obj_nodes.get(node);
  } else
   return null;
 }
 
 // XMLHttpRequest
 // Originally implemented by Yehuda Katz

 window.XMLHttpRequest = function(){
  this.headers = {};
  this.responseHeaders = {};
 };
 
 XMLHttpRequest.prototype = {
  open: function(method, url, async, user, password){ 
   this.readyState = 1;
   if (async)
    this.async = true;
   this.method = method || "GET";
   this.url = url;
   this.onreadystatechange();
  },
  setRequestHeader: function(header, value){
   this.headers[header] = value;
  },
  getResponseHeader: function(header){ },
  send: function(data){
   var self = this;
   
   function makeRequest(){
    var url = new java.net.URL(curLocation, self.url);
    
    if ( url.getProtocol() == "file" ) {
     if ( self.method == "PUT" ) {
      var out = new java.io.FileWriter( 
        new java.io.File( new java.net.URI( url.toString() ) ) ),
       text = new java.lang.String( data || "" );
      
      out.write( text, 0, text.length() );
      out.flush();
      out.close();
     } else if ( self.method == "DELETE" ) {
      var file = new java.io.File( new java.net.URI( url.toString() ) );
      file["delete"]();
     } else {
      var connection = url.openConnection();
      connection.connect();
      handleResponse();
     }
    } else { 
     var connection = url.openConnection();
     
     connection.setRequestMethod( self.method );
     
     // Add headers to Java connection
     for (var header in self.headers)
      connection.addRequestProperty(header, self.headers[header]);
    
     connection.connect();
     
     // Stick the response headers into responseHeaders
     for (var i = 0; ; i++) { 
      var headerName = connection.getHeaderFieldKey(i); 
      var headerValue = connection.getHeaderField(i); 
      if (!headerName && !headerValue) break; 
      if (headerName)
       self.responseHeaders[headerName] = headerValue;
     }
     
     handleResponse();
    }
    
    function handleResponse(){
     self.readyState = 4;
     self.status = parseInt(connection.responseCode) || undefined;
     self.statusText = connection.responseMessage || "";
     
     var stream = new java.io.InputStreamReader(connection.getInputStream()),
      buffer = new java.io.BufferedReader(stream), line;
     
     while ((line = buffer.readLine()) != null)
      self.responseText += line;
      
     self.responseXML = null;
     
     if ( self.responseText.match(/^\s*</) ) {
      //try {
       self.responseXML = new DOMDocument(
        new java.io.ByteArrayInputStream(
         (new java.lang.String(
          self.responseText)).getBytes("UTF8")));
      //} catch(e) {
      //}
     }
    }
    
    self.onreadystatechange();
   }

   if (this.async)
    (new java.lang.Thread(new java.lang.Runnable({
     run: makeRequest
    }))).start();
   else
    makeRequest();
  },
  abort: function(){},
  onreadystatechange: function(){},
  getResponseHeader: function(header){
   if (this.readyState < 3)
    throw new Error("INVALID_STATE_ERR");
   else {
    var returnedHeaders = [];
    for (var rHeader in this.responseHeaders) {
     if (rHeader.match(new Regexp(header, "i")))
      returnedHeaders.push(this.responseHeaders[rHeader]);
    }
   
    if (returnedHeaders.length)
     return returnedHeaders.join(", ");
   }
   
   return null;
  },
  getAllResponseHeaders: function(header){
   if (this.readyState < 3)
    throw new Error("INVALID_STATE_ERR");
   else {
    var returnedHeaders = [];
    
    for (var header in this.responseHeaders)
     returnedHeaders.push( header + ": " + this.responseHeaders[header] );
    
    return returnedHeaders.join("\r\n");
   }
  },
  async: true,
  readyState: 0,
  responseText: "",
  status: 0
 };
})();

UPCScan 0.7: Where is my stuff?

2008-11-16T20:39:00.003+01:00

UPCScan 0.7 is released. New features:

UPCScan can now find music CDs
If UPCScan can't find something on Amazon, it will still create an entry which you can then edit to fill in the details.
Entries can be deleted.
I've added lending information so you can quickly figure out who your new "ex-friends" should be.
I'm working on a series/issue information system to make it more simple to complete your collection. With this version, you'll need to edit the database directly to add series/issue information but the user interface can already display this data.
I'm working on a feature to create an OpenOffice document with the locations. This would allow you to print this out and then scan the locations in as you scan your collection to tell UPCScan under which location to file the items. If you can't wait, then you can use the barcode.py script to generate PNG images with barcodes which you can import in OpenOffice to achieve the same effect.

Download: upcscan-0.7.tar.gz (26,921 Bytes, MD5)

Testing the Impossible: User Dialogs

2008-11-11T11:26:00.004+01:00

How do you test a user dialog like "Do you really want to quit?"

This code usually looks like this:

    public void quit () {
        if (!MessageDialog.ask (getShell(),
            "Really quit?",
            "Do you really want to quit?"
        ))
            return;

        ... quit ...
    }

The solution is simple:

    public void quit () {
        if (askToQuit ())
            return;

        ... quit ...
    }

    /** For tests */
    protected boolean askToQuit () {
        ... ask your question here ...
    }

In test cases, you can now extend the class, and override askToQuit:

    public boolean askToQuitWasCalled = false;
    public boolean askToQuitResult = true;

    protected boolean askToQuit () {
        askToQuitWasCalled = true;
        return askToQuitResult;
    }

Now, you can find out if the question would be asked and you can verify that the code behaves correctly depending on the answer. Tests that just want to quit won't need to do anything special to get the desired behavior.

The same applies to more complex dialogs: Refactor them to put their data into an intermediate structure which you can mock during the tests. That means to copy the data if the dialog is a black box but that's a small price to be paid for being able to test modal user dialogs.

Lesson: You don't want to test the dialog, you want to test whether it is opened at the right place, under the right circumstances and if the result is processes correctly.

I Have Nothing to Hide ... I Think

2008-11-10T16:54:00.002+01:00

So it has happened again. Someone put a nice web site online and when it came to pick and chose between security and comfort, guess who won. Alas, those who do as you shouldn't still server as a bad example. What has happened?

DHL, a German parcel delivery service, offers a web site where you can track where your brand new gadget is now so you can guess how long it will take until you rip the wrapping off it. That good.

Not so good is that all customers of DHL get the same default password.

Bad is that DHL reuses the tracking numbers after roughly six months (depending on the amount of parcels that go through the system; if there are less, you can look further into the past).

Really bad is that part of DHL's tracking number of fixed. It's based on the DHL customer number. That's not you, this "customer" is the guy or company you ordered from (DHL renders a service for them).

So this leaves us with a convenient way to check who else has ordered anything from those that shop.

Now imagine you ordered something innocent ... oh, maybe porn or "adult toys" or something from company B which is the arch enemy of company A which incidentally pays your wage. All of a sudden, a couple of innocent bits of information have turned ugly.

Whenever you put something out to the world, step away for a few moments from your dreams how much good someone could do with your service and think how much bad someone could do with it. And if you can't think of anything, you should be very, very worried.

Multi-line String Literals in Java

2008-10-30T14:47:00.001+01:00

You want multi-line string literals in Java which work in Java 1.4 and up? Can't be done, you say?

Sven Efftinge found a way.

Compile VMware tools on openSUSE 10.3

2008-10-27T11:09:00.002+01:00

I just tried to install the VMware tools on openSUSE 10.3 (Linux kernel 2.6.22.18) so the virtual machine would survive more than 10 days on an ESX server and failed. If you have the same problem, the solution is here.

Errors you'll see during the installation otherwise:

The directory of kernel headers (version @@VMWARE@@ UTS_RELEASE) does not match your running kernel (version 2.6.22.18-0.2-default). Even if the module were to compile successfully, it would not load into the running kernel.

Failure is not an Option

2008-10-22T20:01:00.003+02:00

Everyone loves war stories. Here is one of mine. I need a special diet, especially bread. So one Friday evening, I was taking the train home after buying a couple of custom made loafs of bread. In Dübendorf, I left the train and walked home.

About halfway home, I noticed that I had my head, my arms, my bag ... but not my bread! ARGH! Stupid, stupid, stupid! I knew I should have stuffed them in my rucksack but didn't because it was so full and ... yeah ... okay. My baker needs three days to make these breads so that meant about a week without any for me.

Arriving home, an idea struck me and I fired up the VBZ online service to find out where the train was and when the driver would make the next break. A few moments later, the SBB Train Police got a really strange call by me: "I need my bread. It's in this train and can you please, please ask the driver to check if the white plastic bag with the bread is still there?"

The woman on the other end was surprised and promised to call me back.

Ten minutes can be sooo long.

From the timetable, I knew that my train would probably come through my town in about twenty minutes when I got the call. Yes, they found it and the driver would take the plastic bag into his cabin and she told me where to wait on the platform so he could hand it over.

Try train was on time (as usual), the driver handed me my bag (and it really was mine and all the bread was still there) and I was really relieved. After thanking him, I went home to have my dinner. Thanks to the SBB train police, a train driver and an unknown person who put my bread in the overhead compartment when I left it behind, I didn't go hungry that weekend.

Lesson: If all seems lost, take a step back, do something else and you might have the idea which will save the day.

Train related joke: The SBB (Schweizer Bundesbahn - Swiss federation train company) and the German Bundesbahn (the counterpart of the SBB in Germany) wanted to save some money and decided to buy the same information system to inform about arriving trains on the platform. After a longer evaluation, the plan was dropped. The SBB needed signs which said "Train is 1, 2, 3, 4, 5 minutes late" and the German Bundesbahn needed "Train is half an hour, 1 hour, 2 hours, 3 hours, 4 hours, Train Cancelled."

So... You want your code to be maintainable.

2008-10-14T17:56:00.003+02:00

A great post if you're interested in TDD or testing in general: So... You want your code to be maintainable. by Uncle Bob. Thanks, Bob!

Good Games for the PS3

2008-10-11T22:58:00.002+02:00

I've recently upgraded to a PlayStation 3. I kept my old PS2, though, since the new PS3 can't emulate the PS2. I wonder why that is ... maybe it's because Sony is still selling so many PS2's? Ah, rumors :) Easy to create and hard to kill.

So what good games are there? Here is my list:

Burnout Paradise City
PixelJunk Eden
Ratchet & Clank - Tools of Destruction
Flow

Burnout Paradise City

Mindless street racing with a high adrenaline level. Ideal to waste a couple of minutes or an hour. Great graphics, no blood, no violence (it's more like auto scooter) and nice ideas like smashing ads or the super jumps. If you don't like some events (haven't managed to win a single race, yet. I excel at kicking other cars off the street), you can simply ignore them and still complete enough of the game to have fun.

PixelJunk Eden

A definitive feel of Tarzan or Spiderman when you want to relax a bit. Simple, fitting graphics, no violence, no agression. All that and at the price it's a steal.

Ratchet & Clank - Tools of Destruction

My favorite jump'n'run. Lots of insane weapons, Ratchet's ears look great on the PS3, the story has more depth than usual; not sure I like the depressive realization at the end, though. Judge for yourself.

Flow

Like Eden, it's a brand new kind of game. One of a kind. I play that when I want to come down from all the stress in my life. Go get it!

Bad games

I've got a couple of other games. First, we have the Orange Box with Half Life 2, two of the extra episodes, Portal and Team Fortress. I liked the puzzles in Portal. That games was much too short. I didn't like Half Life 2 much and I hate the episodes. The story was great, the levels gigantic and intelligent. You could almost always find a way around without getting killed. But the handling ... Freeman feels like a block of wood when you move him through the levels: You'll get stuck all the time at hand rails and stuff like that. Sometimes, he'll be able to jump on something, sometimes not. Sometimes, he'll stay on top of a barrel, sometimes not. This sucks. And then those stupid zombie levels. Yeah, I'm stuck in the elevator scene in the first episode. Got killed five times in the dark and now, the games goes where it belongs: The trash.

Resistance - Fall of Man. I like the other games by Insomniac and I like this one, too. It's just too violent for my taste. I like shooting pixels or push empty cars off the street or zoom down a highway at break-neck speed. I don't like shooting at people. I finished the game but it left me asking: Is that all? Running around, shooting people, blow up stuff? Is that the result of many years of game evolution? Better graphics?

Uncharted: Drake's Fortune. Oh well. Okay, the levels look great. When you scale the wall of the castle, there is a sense of vertigo. It's breath taking. The jeep escape is a lot of fun. Smart story (mostly). The game character moves smart. You press a button and he takes cover. He's smart, not a dead puppet like Freeman. He moves as if he was real. Again the violence cooled me off quickly. Too much killing, not enough puzzles.

Enthought Traits

2008-10-09T15:21:00.003+02:00

I'm always looking for more simple ways to build applications. Let's face it, it's 2008 and after roughly 50 years, writing something that collects a few bits of data and presents them in a nice way is still several days of work. And that's without Undo/Redo, a way to persist the data, a way to evolve the storage format, etc.

Python was always promising and with the tkinter module, they set a rather high watermark on how you easily could build UIs ... alas Tk is not the most powerful UI framework out there and ... well ... let's just leave it at that.

With Traits, we have a new contender and I have to admit that I like it ... a lot. The traits framework solves a lot of the standard issues out of the box while leaving all the hooks and bolts available between a very thin polish so you can still get at them when you have to.

For example, you have a list of persons and you want to assign each person a gender. Here is the model:

class Gender(HasTraits):
    name = Str
    
    def __repr__(self):
        return 'Gender %s' % self.name

class Person(HasTraits):
    name = Str
    gender = Instance(Gender)
    
    def __repr__(self):
        return 'Person %s' % self.name

class Model(HasTraits):
    genderList = List(Gender)
    persons = List(Person)

Here is how you use this model:

female = Gender(name='Female')
male = Gender(name='Male')
undefined = Gender(name='Undefined')

aMale = Person(name='a male', gender=male)
aFemale = Person(name='a female', gender=female)

model = Model()
model.genderList.append(female)
model.genderList.append(male)
model.genderList.append(undefined)
model.persons.append(aFemale)
model.persons.append(aMale)

Nothing fancy so far. Unlike the rest of Python, with Traits, you can make sure that an attribute of an instance has the correct type. For example, "aMale.gender = aFemale" would throw an exception in the assignment.

The nice stuff is that the UI components honor the information you use to build your model. So if you want to show a tree with all persons and genders, you use code like this:

class Model(HasTraits):
    genderList = List(Gender)
    persons = List(Person)
    tree = Property
    
    def _get_tree(self):
        return self

class ModelView(View):
    def __init__(self):
        super(ModelView, self).__init__(
            Item('tree',
                editor=TreeEditor(
                    nodes = [
                       TreeNode(node_for = [ Model ],
                           children = 'persons',
                           label = '=Persons',
                           view = View(),
                       ),
                       TreeNode(node_for = [ Person ],
                           children = '',
                           label = 'name',
                           view = View(
                               Item('name'),
                               Item('gender',
                                  editor=EnumEditor(values=genderList,)
                               ),
                           ),
                       ),
                       TreeNode(node_for = [ Model ],
                           children = 'genderList',
                           label = '=Persons by Gender',
                           view = View(),
                       ),
                       TreeNode(node_for = [ Gender ],
                           children = '',
                           label = 'name',
                           view = View(),
                       ),
                    ],
                ),
            ),
            Item('genderList', style='custom'),
            title = 'Tree Test',
            resizable = True,
            width = .5,
            height = .5,
        )

model.configure_traits(view=ModelView())

First of all, I needed to add a property "tree" to my "Model" class. This is a calculated field which just returns "self" and I need this to be able to reference it in my tree editor. The tree editor defines nodes by defining their properties. So a "Model" node has "persons" and "genderList" as children. The tree view is smart enough to figure out that these are in fact lists of elements and it will try to turn each element into a node if it can find a definition for it.

That's it. Everything else has already been defined in your model and what would be the point in doing that again?

But there is more. With just a few more lines of code, we can get a list of all persons from a Gender instance and with just a single change in the tree view, we can see them in the view. If you select a person and change its name, all nodes in the tree will update. Without any additional wiring. Sounds too good to be true?

First, we must be able to find all persons with a certain sex in Gender. To do that, we add a property which gives us access to the model and then query the model for all persons, filter this list by gender and that's it. Sounds complex? Have a look:

class Gender(HasTraits):
    name = Str
    persons = Property
    
    def _get_persons(self):
        return [p for p in self.model.persons
                if p.gender == self]

But how do I define the attribute "model" in Gender? This is a hen-and-egg problem. Gender references Model and vice versa. Python to the rescue. Add this line after the definition of Model:

Gender.model = Instance(Model)

That's it. Now we need to assign this new field in Gender. We could do this manually but Traits offers a much better way: You can listen for changes on genderList!

    def _genderList_items_changed(self, new):
        for child in new.added:
            child.model = self

This code will be executed for every change to the list. I walk over the list of new children and assign "model".

Does that work? Let's check: Append this line at the end of the file:

assert male.persons == [aMale], male.persons

And the icing of the cake: The tree. Just change the argument "children=''" to "children = 'persons'" in the TreeNode for Gender. Run and enjoy!

One last polish: The editor for genders looks a bit ugly. To suppress the persons list, add this to the Gender class:

    traits_view = View(
        Item('name')
    )

There is one minor issue: You can't assign a type to the property "persons" in Gender. If you do, you'll get strange exceptions and bugs. Other than that, this is probably the most simple way to build a tree of objects in your model that I've seen so far.

To make things easier for you to try, here is the complete source again in one big block. You can download the Enthought Python Distribution which contains all and everything on the Enthought website.

from enthought.traits.api import \
        HasTraits, Str, Instance, List, Property, This

from enthought.traits.ui.api import \
        TreeEditor, TreeNode, View, Item, EnumEditor

class Gender(HasTraits):
    name = Str
    # Bug1: This works
    persons = Property
    # This corrupts the UI:
    # wx._core.PyDeadObjectError: The C++ part of the ScrolledPanel object has been 
    # deleted, attribute access no longer allowed.
    #persons = Property(List)
    
    traits_view = View(
        Item('name')
    )
    
    def _get_persons(self):
        return [p for p in self.model.persons if p.gender == self]
    
    def __repr__(self):
        return 'Gender %s' % self.name

class Person(HasTraits):
    name = Str
    gender = Instance(Gender)
    
    def __repr__(self):
        return 'Person %s' % self.name

# Bug1: This doesn't work; you'll get ForwardProperty instead of a list when
# you access the property "persons"!
#Gender.persons = Property(fget=Gender._get_persons, trait=List(Person),)
# Same
#Gender.persons = Property(trait=List(Person),)
# Same
#Gender.persons = Property()
# Same, except it's now a TraitFactory
#Gender.persons = Property

class Model(HasTraits):
    genderList = List(Gender)
    persons = List(Person)
    tree = Property
    
    def _get_tree(self):
        return self
    
    def _genderList_items_changed(self, new):
        for child in new.added:
            child.model = self

Person.model = Instance(Model)
Gender.model = Instance(Model)

female = Gender(name='Female')
male = Gender(name='Male')
undefined = Gender(name='Undefined')

aMale = Person(name='a male', gender=male)
aFemale = Person(name='a female', gender=female)

model = Model()
model.genderList.append(female)
model.genderList.append(male)
model.genderList.append(undefined)
model.persons.append(aFemale)
model.persons.append(aMale)

assert male.persons == [aMale], male.persons

# This must be extenal because it references "Model"
# Usually, you would define this in the class to edit
# as a class field called "traits_view".
class ModelView(View):
    def __init__(self):
        super(ModelView, self).__init__(
            Item('tree',
                editor=TreeEditor(
                    nodes = [
                       TreeNode(node_for = [ Model ],
                           children = 'persons',
                           label = '=Persons',
                           view = View(),
                       ),
                       TreeNode(node_for = [ Person ],
                           children = '',
                           label = 'name',
                           view = View(
                               Item('name'),
                               Item('gender',
                                  editor=EnumEditor(
                                      values=model.genderList,
                                  )
                               ),
                           ),
                       ),
                       TreeNode(node_for = [ Model ],
                           children = 'genderList',
                           label = '=Persons by Gender',
                           view = View(),
                       ),
                       TreeNode(node_for = [ Gender ],
                           children = 'persons',
                           label = 'name',
                           view = View(),
                       ),
                    ],
                ),
            ),
            Item('genderList', style='custom'),
            title = 'Tree Test',
            resizable = True,
            width = .5,
            height = .5,
        )

model.configure_traits(view=ModelView())

UPCScan 0.6: It's Qt, Man!

2008-10-08T21:20:00.006+02:00

Update: Version 0.7 released.

Getting drowned in your ever growing CD, DVD, book or comic collection? Then UPCScan might be for you.

UPCScan 0.6 is ready for download. There are many fixed and improvements. The biggest one is probably the live PyQt4 user interface (live means that the UI saves all your changes instantly, so no data loss if your computer crashes because of some other program ;-)).

The search field accepts barcodes (from a barcode laser scanner) and ISBN numbers. There is a nice cover image dialog where you can download and assign images if Amazon doesn't have one. Note: Amazon sometimes has an image but it's marked as "customer image". Use the "Visit" button on the UI to check if an image is missing and click on the "No Cover" button to open the "Cover Image" dialog where you can download and assign images. I haven't checked if the result of the search query contains anything useful in this case.

UPCScan 0.6 - 24,055 bytes, MD5 Checksum. Needs Python 2.5. PyQt4 4.4.3 is optional.

Security notice: You need an Amazon Web Service Account (get one here). When you run the program for the first time, it will tell you what to do. This means two things:

Your queries will be logged. So if you don't want Amazon to know what you own, this program is not very useful for you.
Your account ID will be stored in the article database at various places. I'm working on an export function which filters all private data out. Until then, don't give this file to your friends unless you know what that means (and frankly, I don't). You have been warned.

Name of the Longest Distance Between Two Points

2008-10-07T09:34:00.001+02:00

Q: What's the name of the shortest distance between two points?

A: The straight line.

Q: What's the name of the longest distance between two points?

A: The shortcut.

You Can't Stop OOXML!? Watch Me :-)

2008-10-06T10:15:00.003+02:00

MicroSoft is the de-facto standard on the desktop. Despite all the efforts to break the monopoly, the average user still doesn't want to switch. Alas, the average user is not an expert and when MicroSoft tries its way with the experts, that usually backfires. Unlike the Average Joe, geeks and nerds are no cattle and they find creative ways to get even when they are served what MicroSoft dishes out.

So in Norway, 21 of 23 experts voted against OOXML as a new ISO standard. That didn't stop the ca...administration of Standard Norge to embrace this great work from Seattle (which has eaten thousands, maybe millions, of dissertations over the years) and so they announced that Norway votes "Yes". Every geek out there knows what it means when your management has stopped listening to you: Get a new job. And they did.

Right on, commander!

Somebody is Playing Pong With Two Elevators and the Floorlights...

2008-10-06T10:10:00.002+02:00

Once, it was just a joke in the UserFriendly comic strip. Now, it's real: Welcome to Project Blinkenlights

Scanning Your DVD, Book, Comic, ... Collection

2008-10-04T22:59:00.004+02:00

Update: Version 0.6 released.

If you're like me, you have a lot of DVDs, books, comics, whatever ... and a few years ago, you kind of lost your grip on your collection. Whenever there is a DVD sale, you invariantly come home with a movie you already have.

After the German Linux Magazin published an article how to setup a laser scanner with Amazon, I decided to get me one and give it a try. Unfortunately, the Perl script has a few problems:

It's written in Perl.
It's written in Perl.
It's written in Perl.
There is no download link for the script without line numbers.
The DB setup script is missing.
The script uses POE.
It's hard to add new services.
Did I mention that it's written in Perl? Right.

So I wrote a new version in Python. You can find the docs how to use it in the header of each file. Additionally, I've included a file "Location codes.odt". You can edit it with OpenOffice and put the names of the places where you store your stuff in there. Before you start to scan in the EAN/UPC codes of the stuff in a new place, scan the location code and upcscan.py will make the link for you. It will also ask you for a nice name of the location when you scan a location code for the first time.

If you need more location codes, you can generate them yourself. The codes starting with "200" are for private use, so there is no risk of a collision. I'm using this Python script to generate the GIF images. Just put this at the end of the script:

if __name__=='__main__':
    import sys
    s = checksum(sys.argv[1])
    img = genbarcode(s, 1)
    img.save('EAN13-%s.gif' % s, 'GIF')
    print error

There is a primitive tool to generate a HTML page from your goods and a small tool to push your own cover images into the database if Amazon doesn't provide one.

Note: You'll need an AWS account for the script to work. The script will tell you where to get your account ID and where you need to put the ID when you start it for the first time.

Download upscan-0.1.tar.gz (54KB, MD5 Checksum)

Firefox 3.0.1 and GMail: Gray Background???

2008-09-29T10:22:00.002+02:00

Am I the only one who gets a gray background on many websites after the update to Firefox 3.0.1?

Update: Apparently so. If not, browse to about:config and search for "browser.display.background_color". In my version of FF, that was set to #C0C0C0 for some reason (when it should have been #FFFFFF).

Installing Eclipse 3.4.1 Despite p2

2008-09-29T09:58:00.003+02:00

If you're, like me, one of the unlucky ones that aren't on p2's friends list (translation: Eclipse p2 provisioning causes you an endless stream of pain and suffering), then you can't install the 3.4.1 patches because p2 won't let you.

There are several ways to deal with this. One of them is to delete your workspace's .metadata, your Eclipse install and start from scratch, installing all plug-ins again, etc., always hoping that p2 doesn't mess up until you've installed everything.

The other way is a workaround. It needs a bit of disk space and discipline. Do this:

Leave your original Eclipse install alone. Specifically, never ever use the menu "Software updates..." again! Never. I mean it. Disable the entry if you can.
Install eclipse again in a new place. This must be a standard install (not a shared one!!!)
Do not start this install! Specifically, do not attempt to add all your update sites to this base template! Just unpack it and rename the "eclipse" directory to "eclipse-template".
Copy "eclipse-template" to "eclipse-install".
Start eclipse-install. If you worry that you might accidentally start the template once in a while, rename "eclipse.exe" to "eclipse.exe Is this install".
Download the 3.4.1 updates.
Exit eclipse-install.
Use your favorite file copy tool to copy all new files and directories in eclipse-install\plugins and eclipse-install\features to your working copy of eclipse.
Start your working copy.

Installing and updating plug-ins works in a similar way:

Delete eclipse-install and recreate it from eclipse-template.
Start eclipse-install.
Open the install software dialog. Add the update site. You may be tempted to add the update sites to eclipse-template. Don't do this! As soon as p2 can see more than one update site, it will eventually mess up in the dependency calculation.
Install the plug-in.
Create a directory for your new plug-in in the driectory "dropins" of your working copy of Eclipse.
Copy the new files and directories from eclipse-install\features and eclipse-install\plugins to the new directory below "dopins" in your working copy of Eclipse.
If you need to install more than one plug-in, start with step 1. After you have installed anything in eclipse-install, the Eclipse instance is tainted and shouldn't be used again.

That's all folks. At least until the p2 guys fix the many bugs in their code. Which will probably in the Eclipse release in 2010.

That's not because I believe that the p2 guys are stupid or lazy but because this kind of product just takes three years to mature and they started in 2007, so the first working version can be expected in 20010.

Stack Overflow Launches

2008-09-18T20:44:00.002+02:00

Stack Overflow is a Q&A site for programmers. If you're looking for an answer to a question or if you know a lot and can't really fill your needs to help in your current position, have a go at it.

Amiga Forever 2008

2008-09-08T10:26:00.002+02:00

I have the set for a while now (thanks Michele!) and after reading the announcement elsewhere, I'd like to remind all you Amiga fans out there that the 2008 release of Amiga Forever is ready. Did you know that Andy Warhol (yeah, that Andy Warhol) gave a demo of the system when it was launched 1985? If not, grab the premium edition and watch him do his thing with Marilyn Monroe (or rather a picture of her).

How To Launch Software

2008-09-05T15:58:00.001+02:00

If you're still wondering if "big bang" software releases are a good thing, read this.

The Space Between Two Characters

2008-08-31T21:21:00.002+02:00

If you're claustrophobic, you're afraid of confined spaces. If you're a software developer, you can be afraid of non-existing space.

When it comes to editing text, we usually don't think about the space between the characters. There simply isn't any. When you write a text editor, things start to look different. Suddenly, you have a caret or cursor which goes between the characters and that space between two characters can suddenly become uncomfortably tight.

Fire up your favorite text editor, Word, Writer, whatever. It has to support character formatting, though. Now enter this:

Hello, world!

If in doubt, the bold text ends before the comma and the italic part starts with the "w" and ends with "!" including both.

Now move to the "e" and type "x". What do you get?

Hxello, world!

Piece of cake.

Now move to the "H" and type "x". What do you get? Is the new x bold or not? Do you get "xHxello" or "xHxello"? How about typing "x" after the "o" of our abused "Hello"? Is that new x bold or not? If it is, what is the most simple way to make it non-bold? Do you have to delete the comma, do you have to go through menus or toolbars or is there a simple, consistent way to add a character inside and outside of a formatted range of text?

Let's go one step further. Add a character after the "!". Is it italic? If not, you're lucky. If it is ... what's the most simple way to you get rid of the italic? If you press Return now, will the italic leak to the next line? If not, how can you make it leak? If that italic is the last thing in your text, can you add non-italic text beyond without fumbling with the formatting options?

There is no space between two characters and when you write a text editor, that non-existing space is biting you. Which is actually the problem: There is no consistent way to move in and out of a formatted range of characters.

The naive attempt would be to say "depending on the side you came from, you're inside or outside." So, if we have this (| is the cursor or caret): "Hello |world" and you type something, the question is: How did the caret end there? Did it come from "w|o" and moved one to the left? Or from "o| " and move one position right?

That works somewhat but it fails at the beginning and the end of the text plus you're in trouble during deleting text. What should happen after the last character of "Hello" has been deleted? Should that also delete the character range or should there be an empty, invisible bold range left and when you type something now, it should appear again? If you keep the empty invisible range, when do you drop it? Do you keep it as long as the user stays "in" it? Or until the document is saved? Loaded again from disk?

It's a mess and there is a reason why neither Word nor OpenOffice get it right: You can't. There is information in the head of the user (what she wants) but no way for her to tell the computer. Duh.

That is, unless you start to give the user a visual cue what is going on. The problems we have is that there is no simple, obvious way for the user to say "I want ..." because there is no space on the screen reserved for this. We barely manage to squeeze a caret between the characters. There is just not enough room.

Well, there could be. A simple solution might be to add a little hint to the cursor to show which way it is leaning right now. Right. How about "A|B"? Here, you have three options. Add bold, italic and normal.

In HTML, this is simple. I'm editing this text in Firefox using the standard text area. What looks fancy to you looks like this to me: "<b>A</b>|<i>B</i>"

And this is the solution: I need to add a visual cue for the start and end of the format ranges. Maybe a simple U-shape which underlines the text for which the character format applies. Or an image (> and < in this example): ">A<|>B<". And suddenly, it's completely obvious on which side of the range start and end you are and what you want. You can delete the text in the range without losing it or you can delete both and you can move in and out of the range at will.

The drawback is that you need to keep that information somewhere. It adds a pretty huge cost to the limits of a format range. I'll have to try and see how much that is and if I can get away with less by cleverly using the information I already have.

Also, it clearly violates WHYSIWYG. On the other hand, we get WYSIWYW which is probably better for the user.

DecentXML 1.2

2008-08-31T15:55:00.002+02:00

DecentXML 1.2, my own XML 1.1-compliant parser, is now available.

Text Editor Component and JADS

2008-08-27T21:35:00.003+02:00

While working on DecentXML (1.2 due this weekend), I've had those other two things that were bugging me. One is that there is no high-quality, open-source framework with algorithms and data structures. I'm not talking about java.lang.Collections, I'm talking about red-black trees, interval trees, gap buffers, things like that. Powerful data structures you need to build complex software.

Welcome the "Java Algorithm and Data Structure" project - jads. I haven't started opened a project page on SourceForge or Google Code, yet, but I'll probably do that this weekend.

Based on that, I'm working on a versatile text editor component for Java software. The final editor will work with user interfaces implemented in Swing, SWT and Qt. It's an extensible framework where you can easily replace parts with your own code to get the special features you need. I currently have a demo running which can display text, which allows scrolling and where you can do some basic editing. Nothing fancy but it's coming alone nicely.

If you want to hear more about these projects, post a comment or drop me a mail.

Death Star in EVE Online

2008-08-19T14:30:00.003+02:00

Apparently, a group of 4000 players of EVE Online have built a kind of a "Death Star" (a "titan ship" in the language of the game) to rule the game galaxy. Assembly took 8 months in total secrecy and the result was destroyed completely within 3 months.

Another Lesson on Performance

2008-08-19T09:20:00.002+02:00

Just another story you can tell someone who fears that "XYZ might be too slow":

There are various data structures which are suitable for a text editor and some of those depend on copying data around (see gap buffers). The question is: How effective is that? The first instinct of a developer is to avoid copying large amounts data and to optimize the data structure instead.

After years of training, I've yet to overcome this instinct and start to measure:

    public static void main (String[] args)
    {
        long start = System.currentTimeMillis ();
        
        int N = 10000;
        for (int i=0; i<N; i++)
        {
            int[] buffer = new int[1024*1024];
            System.arraycopy (buffer, 0, buffer, 1,
                buffer.length-1);
        }
        
        long duration = System.currentTimeMillis () - start;
        System.out.println (duration);
        System.out.println (duration / N);
    }

On my computer at work (which is pretty fast but not cutting edge), prints: "135223" and "13". That's thirteen milliseconds to copy 4MB of RAM. Okay. It's obviously not worth to spend a second to think about the cost of moving data around in a big block of bytes.

Lesson: If you're talking about performance and you didn't measure, you have no idea what you're talking about.

Still not convinced? Read this.

Quantity Always Trumps Quality

2008-08-04T17:38:00.001+02:00

While I wouldn't completely subscribe to that without a grain of salt, the story is nice.

Four harmful Java idioms, and how NOT to fix them

2008-08-04T14:24:00.003+02:00

In his article "Four harmful Java idioms, and how to fix them", John O'Hanley writes about how to make Java more maintainable. He picks four common patterns and gives tips how to fix them ... or not. Follow me.

Names

The first idea is to prefix names with a letter giving a hint what they mean: Is "start" a method? A field? A parameter? The goal is to make the code more readable to humans.

Unfortunately, this doesn't work. The human brain doesn't read letters, it reads words. So "fStart" (meaning a field with the name "start") is rejected by the brain because it's not a word. This triggers the conscious analysis which John tries to avoid! Which is why modern IDEs use color to tell you what something is: The brain can decode color and words in independent parts - unconsciously.

Packaging Convention

Next, he moves on how to split code into packages. Currently, we use a "package by layer" scheme, meaning all DB code goes into one package and the model code into another and UI layer in a third, etc. He proposes to use a "by-feature" packaging with the litmus test "you should be able to delete a feature by deleting a single directory, without leaving behind any cruft".

Uhm. When have you ever written any code where you could remove a feature just by deleting a class? This sounds nice and simple but it's fails Einstein's litmus test: "Make it as simple as possible but not more simple". Even if you have a plug-in based software like Eclipse, this doesn't work because there are still references outside (otherwise, your plug-in wouldn't be able to do anything).

Also, to keep a feature as isolated from everything else as possible (which is a good thing), you need to copy a lot of code into the feature which would otherwise reside elsewhere, neatly packed up in its own package. Really just a limitation of Java where you can't tell the compiler to generate boiler plate code for you. Still, you need to cut code in such a way that it reduces dependencies, not increases them. Therefore, a general rule won't cut (or maybe it will cut: you).

Immutables

John quotes: "'Classes should be immutable unless there is a very good reason for making them mutable,' says Bloch.". And later: "From a practical perspective, many widely-used frameworks require application programmers to use JavaBeans (or something similar) to model database records. This is deeply unfortunate, because it doesn't allow programmers to take advantage of the many positive qualities of immutable objects."

From a practical perspective, immutable objects are dead weight. Applications are all about changing something. I read data from the database, I modify it, I write it back. I rarely read, display and forget about something. Yes, immutables have advantages because they can be shared between threads but that's their only advantage.

Just think about this: You must modify data from the database. So you read the data into an immutable. How to modify it, now? Obviously, you need a method to change it. If you prefer setters, the "setter" must return a copy. So you need to copy the object for every single change. If you want to get a feeling for that, try to do math with BigDecimal. Okay, after the copy you can write the copy back to the database. Question: How do you notify everyone else who might have a (now stale) copy of the old immutable? There are no listeners; immutables can't have listeners. Duh. Driving this to the extreme, lists wouldn't offer methods to add or remove items; or rather they would return new copies of themselves after every add/remove operation.

Sorry, no sale. I can't add money to my cash register. It's immutable.

And a colleague just introduced me to another great concept: Constructors which require values for all fields. The class in question has 95 fields. This idea has the following flaws: a) No matter how big your screen, you can't fit the call onto it. b) After argument #10, you lost track and you can't see anymore which value goes into which argument. Now imagine you have to remove a field. How do you find the right one in this mega-call?

No, nothing beats the no-arg constructor plus a list of setters, all costs considered.

Private members

John proposes to move private members to the end of the class. Here, I agree. I'd even put them close to the getter and setter so that a lot of stuff that belongs together is together.

In todays IDEs with their superb code navigation (I can't really believe there was a time before the F3 key), this doesn't matter much, though.

Conclusion: Think about it, but don't bother.

DecentXML 1.1

2008-08-01T21:45:00.001+02:00

I've just released 1.1 of DecentXML.

Update to DecentXML

2008-07-30T21:50:00.004+02:00

I've updated my XML parser. The tests now cover 97.7% of the code (well, actually 100% of the code which can be executed; there are a couple of exceptions which will never be thrown but I still have to handle them) and there are classes to read XML from InputStream and Reader sources (including encoding detection).

The XMLInputStreamReader class can be used standalone, if you ever want to read an XML file with the correct encoding.

You can download the sources and report issues in the new Google Code project I've created.

Information Management With Zotero

2008-07-30T13:41:00.003+02:00

I've been long looking for a nice tool to manage my vast extra-brain information collection, i.e. the stuff that I don't want to save in my long term memory. Web snippets, notes, that kind of stuff. All the usual solutions didn't appeal to me. Either I was locked to Windows or to a single computer or the UI was bad or the feature list lacked some important points.

Zotero to the rescue. This beast is advertised as "Zotero [zoh-TAIR-oh] is a free, easy-to-use Firefox extension to help you collect, manage, and cite your research sources. It lives right where you do your work — in the web browser itself."

Which makes sense. I watch most of my information in my web browser, so why no collect it in there, too? The UI is nice, I'm just missing a few features. Also being able to sync with my own server would be nice. But I'm sure that will be fixed, soon. In the mean time, I can at least tag and order my snippets.

A Decent XML Parser

2008-07-29T22:21:00.005+02:00

Since there isn't one, I've started writing one myself. Main features:

Allows 100% round-tripping, even for weird whitespace between attributes in elements
Suitable for building editors and filters which want to preserve the original file layout
Error messages have line and column information
Easy to reuse
XML 1.0 compatible

You can download the latest sources here as a Maven 2 project.

DSLs: Introducing Slang

2008-07-28T17:29:00.003+02:00

Did you ever ask for a more compact way to express something in your favorite programming language? Say hello to DSL (Domain Specific Language). A DSL is a slang, a compact way to say what you want. When two astronauts talk, they use slang. They need to get information across and presto. "Over" instead of "I'll now clear the frequency so you can start talking." And when these guys do it, there's no reason for us not to.

Here is an artical on Java World which gives some nice examples how to create a slang in Java and in Groovy. Pizza-lovers of the world, eat your heart out.

FREE! Really.

2008-07-28T15:49:00.002+02:00

I just found a nice comment under my blog. It offered a free service. One sentence was: "REGISTRATION IS ABSOLUTELY FREE!" When you see that, you know you're being ripped off. I'm not mentioning the name of the guys who tried that stunt in order to give them no additional advertisement. 'Nuff said.

Tip: If you want me to join your planet or RSS mega feed or whatever, it's not smart to post a comment in my blog. This is my blog, my reputation, my honor. I decide who gets free advertisement here.

Testing With Databases

2008-07-26T13:43:00.002+02:00

When it comes to testing, databases are kind of a sore spot. People like to think that "you can't test when you need a database" or "it's too complicated" or "it's not worth it." I'd like to give you some ideas what you can do when you need to test code that depends on a database. This list is sorted in the order in which I try to tackle the problem:

Use POJOs to store the data from the database in the real code and for the tests, create some dummy objects with test data and use them.
Make the database layer a plug-in of your application and replace it with a mockup for testing that doesn't need the database and which returns test objects instead.
Instead of connecting to the real database, get HSQLDB or Derby and use an embedded or at least local database. I prefer HSQLDB because it's smaller and starts faster (and tests should always be fast) but Derby has more features.
Create a second instance of the production database system on a different machine, preferably your own computer.
Create another instance of the real database with test data on the same machine as the real database.
Use database schemas to create a logical database in the real database, for example if all tables are in the schema APP, create APP_TEST and in your code, add a way to replace the schema name in the SQL statements. If you wrote the DB layer yourself, use a system property which isn't set in production. If you're using Hibernate, walk the mapping objects which are created and replace the table names after loading the production configuration. Field.setAccessible(true) is your friend.

If you can't decide, here are a few hints:

Creating two databases using schemas in the same instance can get you into serious trouble without you noticing. For example, the tests should be able to rebuild the test database from scratch at the press of a button so you can be sure in which state the database really is. If you make a mistake with the schema name during that setup, you'll destroy the real database. You might not notice you did, because the flawed statement is usually hidden under a few hundred others.

Installing a second instance on a different machine might trigger license fees or your DB admin might not like it for some reason. Also, a test database should be very flexible because you'll need to be able to drop and recreate it a dozen times per hour if you need to. Your DB admins might not like to give you the necessary rights to do that. Lastly, this means only one developer can run all the tests at any given point in time because you're all going against the same database. This is bad, really bad. More often than not, you'll have spurious errors because of that.

If you can legally get a copy of the real database on your own machine, that's cool ... until you see the memory, CPU and hard disk requirements plus a DB admin will probably hog your machine for a day or two to install it. Having to run two applications which need 1GB of RAM (your IDE and the DB) with a machine that has only 1GB of RAM isn't going to fun.

For many cases, using HSQLDB or Derby is a good compromise between all forces that pull at you. While that will make your tests slow, they will often run much faster than against the real DB. You can install these as many times you like without any license issues, fees or DB admins bothering you. They don't take much memory or hard disk space and they are under your total control.

Only, they are not the real DB. There might be slight differences, hidden performance issues and other stuff that you won't notice. Don't bother about that, though. If you can test your application, you'll find that you'll be able to fix any problems that come up when you run against the real database in little time. If you can't test your application, thought, well, you're doomed.

I strongly recommend to be able to setup the database from scratch automatically. With Derby, you can create a template database and clone that on the first connection. With HSQLDB, loading data is so fast that you can afford to rebuild it with INSERT statements every time you run the tests.

Still, test as much code as possible without a database connection. For one, any test without a DB will run 100-1000 times faster. Secondly, you're adding a couple more points of failure to your test which are really outside the scope of your test. Remember, every test in your suite should test exactly one thing. But if you can't get rid of the connection, you're testing the feature you want plus the DB layer plus the DB connection plus the DB itself. Plus you'll have the overhead of setup, etc. It will be hard to run a single test from your suite.

At the end of the day, testing needs to be fun. If you feel that the tests are the biggest obstacle in being productive, you wouldn't be the good developer you are if you didn't get rid of them.

One last thing: Do not load as much data as possible! It is a common mistake to think that your tests will be "better" if you have "as much data as possible". Instead load as little data as possible to make the tests work. When you find a bug, add as little data as possible to create a test for this bug. Otherwise, you'll hog your database with useless junk that a) costs time, b) no one can tell apart from the useful stuff and c) it will give you a false feeling of safety that isn't there.

If you don't know which data is useful and which isn't, then you don't know. Loading of huge amounts of junk into your database won't change that. In order to learn, you must start with what you know and work from there. Simply copying the whole production system will only slow you down and it will overwrite the carefully designed test cases you inserted yesterday.

Nexus, a Maven Repository Manager

2008-07-26T12:10:00.002+02:00

If you're using Maven in a corporate environment, then you're struggling with the same problems all over again: How to make sure that the build builds?

While a simple task at first glance, there are a few hidden obstacles which boil down to two things: Downloads via the Internet and plugin or dependency version stability. Both can be solved by a using a proxy or a in-house repository.

The guys from Sonatype have been busy in the last months and have released Nexus 1.0.0-beta-4.2 which gives you another option to chose from besides Archiva or DSMP (my own Maven 2 proxy). I've tried Nexus yesterday and I have to say that I'm very pleased with the result. As usual for Open Source Software, the beta is more stable than some post-beta commercial products and it delivers with very little setup (follow the link to see the documentation).

Now, we have a second issue: version stability. Here is my recipe to achieve that. First of all, version anything in your POM. All dependencies, all plugins, everything. I'm using properties for that which I define in a common parent POM plus I'm using the dependency management. Maven 2.0.9 helps a lot here because it forces you to add version elements everywhere.

The next step is to make sure the maven builds can find their stuff. To do that, I suggest to set up two Nexus repositories. The first one is the "build" repository, the second one is the "cache" repository. While all developers should use the "build" repository, the "cache" repository can actually download dependencies from the Internet.

The "build" repository, on the other hand, is just a local repository with no Internet connection. To avoid mistakes, I suggest to install the build repo with the default settings but with all remote repositories deleted or turned into local ones. The "cache" repository should run on an unusual port and with the remote repositories enabled as described in the installation documentation.

Next, you need to create a profile in your settings.xml which switches mirrors between the two. When you want to check out a new version of some plugin, switch to the cache repository and have it download all the new stuff. This will pollute your local copy of the maven repository but only yours. After you have verified that the build completes (or fixed all the problems you've got), check the RSS feeds of Nexus for stuff it downloaded. Then, all you have to do, is to copy those to the "build" repository. After a refresh, all the other developers in your company can use the new, verified downloads.

Clean your local repository and build again to make sure that your colleagues won't have any problems after the change and you're set.

Management Is The Art of Choosing What Not To Do

2008-07-23T10:44:00.002+02:00

From Rands in Repose: "... management is the art of choosing what not to do ..."

If you want to know more about management told in a way an engineer can understand, consider Rands' book "Managing Humans".

The Code Reuse Myth

2008-07-22T16:49:00.004+02:00

The post "The Code Reuse Myth" by James Sugrue got me thinking.

The main problem with code reuse is that our programming languages don't support it. We sacrificed this to the gods of efficiency four decades ago and, while a few people dared to question the practice, all of them were struck down by unexpected lightning out of the blue, so far.

As James said, "context" is the keyword. What we'd need is a programming language where you can adapt concepts to a context that goes beyond "object instance" or "class" or "application". What we need is an efficient way to say "collect all objects and sort them by ID" where the context defines what "ID" is. What we need is an efficient way to describe a model (relations of basic data types) and then have some tool map that efficiently to reality so we can reuse parts or fragments of the model in different contexts.

OO can't do that because it's limited to factories and inheritance. Traditional preprocessors can't do it, because they can't see the AST. Having closures in a programming language is the first, tiny step in the direction to be able to push external context into existing code. This allows us to put the code to access the database into a library and influence what it does per row of the result with a closure.

But to be a real new paradigm, we need "closures" in data types as well. This means being able to reuse fragments of code and data structure definitions in a new context. These fragments need to pull along all the algorithms and structures they need without the developer having to pay close attention what is going on until the point where the result needs peep hole optimization because of performance issues.

Example:

fragment Named {
    String name

    String toString () {
        return "name=${name}"
    }
}

class File : Named {...}

class Directory : Named, File {...}

Looks simple but with OO, this will get you in a lot of trouble: Directory gets a name from the class File and from the fragment Named (this is an artificial example, bear with me). Which one should the compiler chose? In OO, I can't say "I don't care" or "Merge them".

With real fragments, you could say "Directory is a File in the sense that it supports a lot of the operations of a file (like rename, delete, get last modification time) but not all (you can't execute a directory or open it for reading)." So the example would look like this:

class File : Named { 
    void delete () {...}
    Reader openForReading () {...}
}

class Directory : Named, File.delete {...}

Now, a Directory has a name and it "copies" the method "delete" from File along with anything this method would need. Internally, the compiler could create an invisible File delegate or it could clone the source or byte code of the File class, or whatever. Or, even better, we could say "give me a copy of File.delete() but replace all references to the field File.path with Directory.path."

The main goal would be to allow to use the compiler as a cut'n'paste tool which checks the syntax and allows me to say "copy that method over there and replace xxx with yyy". Because that's why we think that code reuse could work: We see the same code over and over and over again and each time, the difference is just a tiny little bit of code which we can't "patch" because the language just doesn't allow it.

Lovin' Linux? Dig This!

2008-07-22T10:34:00.002+02:00

Want to make linux better? Ask the Linux Hater. If in doubt: He wouldn't write 15 articles per month telling where Linux sucks if he didn't care.

How to Cure a Fanatic

2008-07-21T21:54:00.003+02:00

Like many people, I've always been wondering how the Jews, barely escaped from being extinct, can behave like they do in Israel and Palestine today. It seems, some of them wonder as well. One of them is Amos Oz who has written a wonderful book about fanaticism: How to Cure a Fanatic.

If you don't understand that I'm arguing against violence here, get the book and read it.

According to the book, a fanatic is a person who cares so much about you that he'd rather kill you than let you be miserable.

Oddly, this makes sense. Fanatics want to make the world a better place -- at all cost. In the second chapter of the book, Oz tells a short story why this doesn't work. He does that in a way that even a fanatic might understand (translated into English by me; all mistakes are mine).

A friend of Amos Oz, the Israeli romancer Sami Michael once made a long trip in a car. During the ride, the driver gave him the usual lecture how important it was for the Jews to kill all Arabs.

Instead of harassing this guy with "What a horrible man you are! Are you Nazi? A Fascist?", Sami listened. He had decided to try a new approach and he asked the driver: "And who, in your opinion, should actually kill all the Arabs?"

The driver replied: "What are you talking about? We! The Israeli Jews! We have to! We have no choice, just look at what they do to us every day!"

"But who exactly should do the job in your opinion? The police or maybe the army or the fire brigade or a team of doctors? Who should do the work?"

The driver scratched his head: "I think it should be spread among us. Everyone should kill a few."

Sami went along with the game. "O.K., I assume you will pick an apartment building in the capital of Haifa, you ring the doorbell or you knock on every door and you say: 'Excuse me, dear Sir or Madam, are you an Arab by any chance?' And if he or she should reply with 'yes', you will shoot them. Then, you just finished your block and want to go home, just then, you hear a baby cry somewhere on the third floor. Would you go back and shoot the baby? Yes or no?"

There was a moment of silence, then the driver said to Sami: "You know, you are a very cruel person."

Now, if your feel anger or disgust, you didn't understand the point of the story, so get the book and read it. For everyone else, think about it. You'll be surprised how many levels of understanding this simple story has and how well it explains the reasons and the fundamental flaw of a fanatic.

Disclaimer: No humans and no animals were harmed, tortured or killed for this blog entry. Only my cat is now mad at me because I dared not to devote her my full attention while I wrote this.

Toying With Swing

2008-07-21T17:20:00.003+02:00

I've been toying with Java Swing (the UI which comes with Java in case you're wondering) a bit lately to determine which UI to use for my ePen project. I'll post a longer article about my findings in the next few days but for now, just a few links I've collected:

From LegHumped:

JTextField with background image (as in the location field in your browser)
Word-wrap for JLabels
Close a JFrame with Escape
Utility methods for tree nodes

Then, there is the JFC Swing FAQ and of course the Java Swing Trail by Sun.

I've been looking for a good source on editor components. Swing Hacks looked promising but it seems to only scratch the surface like the rest.

Docs? Ask The Sphinx

2008-07-16T09:41:00.003+02:00

If you need to generate docs for your Python projects, try Sphinx.

There Are Two Kinds of People ...

2008-07-15T11:04:00.004+02:00

"... Those who separate people in two kinds and those who don't." But I digress.

Ever wondered about the wars in IT? VI vs. Emacs? KDE vs. Gnome? IntelliJ IDEA vs. Eclipse? PC vs. Mac? Why can't people pull along the same string for once?

Well, because they can't. Duh. We all have differences and we find these to make our life more rich or more simple. Can't discuss with a guy who always agrees with you, can you? Or just image your better half to do as you do ... you couldn't even out your advantages and disadvantages! During work, we accept that people are different and we find that useful because it means that work can be spread and people can do what they're good at (instead of where they suck).

Sometimes, this difference goes deep. Way deep. It's so fundamental to our personality that we don't even question this. That's the fundamental schism which fuels the wars in IT. There are "vi" people and there are "emacs" people. Each member of both groups thinks the others are imbeciles who just won't see the light, no matter how often they got beaten some brains into them.

The "vi" people wants to get things done and they don't want the tools to get in the way. A tool should be like a hammer: Simple, to the point, easy to understand and use. If it comes with a manual, it's not a tool, it's a distraction.

The "emacs" people, on the other hand, like to have the most powerful tool they can find at their fingertips. They want to abstract, hide, build tool layer upon tool layer until the task at hand literally happens at the press of a button. If the tool can't be customized, it's not a tool, it's a nuisance.

No matter how much you wish for it, these two kinds of people will never use the other ones tool. If they have to, they will be constantly irritated. Take IntelliJ IDEA, for example. I'm a "vi" guy and this IDE just freaks me out. It's always doing something with my source that I never told it to do, always reformatting, always adding and removing whitespace, always getting in my way. I hate it.

Eclipse, on the other hand, comes with a rich tool set. I can have my source formatted any way I like, but it only does so when I tell it to. The default is to leave my artwork alone. Eclipse doesn't try to be smarter than me. Eclipse gets my jobs done when I want them to be done and it doesn't get in my way.

Don't get my wrong. I'm not telling you why Eclipse is better for you than IntelliJ, I'm saying it's better for me. I'm a "vi" guy.

Now, you may argue that I could probably hack IntelliJ into doing what I want. That's my point exactly: If I have to turn IntelliJ into an Eclipse clone to be able to use it, why not use the tool which fits my hand to begin with? And let's face it: No matter how customizable a tool is, after you've turned it into a clone, there will still be a lot of corner cases.

These come from the core of the design of these tools and that's what makes them as fundamentally different as two humans and no argument in the world will change that.

So, next time someone comes up and says "This or that would be better", answer: "It is better for you.. How about me?"

Fastest Way to Collecting Objects in a String

2008-07-11T20:08:00.002+02:00

The fastest way to collect a list of objects in a String in Java:

StringBuilder buffer = new StringBuilder ();
String delim = "";
for (Object o: list)
{
    buffer.append (delim);
    delim = ", "; // Avoid if(); assignment is very fast!
    buffer.append (o);
}
buffer.toString ();

Are Bad Tests Worse Than No Tests At All?

2008-07-09T09:25:00.002+02:00

In his article "Are Bad Tests Worse Than No Tests At All?", olivstor writes:

Are the drawbacks to bad tests worse than having no coverage at all? I think the answer is that in the short term, even bad tests are useful. Trying to squeeze a extra life out of them beyond that, however, pays diminishing returns. Just like other software, your tests should be built for maintenance, but in a crunch, you can punch something in that works. It's better to have bad tests than to have untested code.

Tests are like any other code: They can go bad.

In my career, I've found that it's surprisingly hard to write good tests if you have no experience in doing so. People starting to write tests make them too complex, too long, let them have too many dependencies and they take too long to run.

If you're in such a situation, you have to face the fact that you just programmed yourself in a corner and you must spent the effort to get out of there. Tests are no magic silver bullet. They are code and follow the usual rules of coding: When it hurts, something is broken and it won't stop hurting unless you fix it.

So in this sense, I agree that bad tests are better than no tests because they tell you early that you need to fix something. That's what their core purpose is.

Management might argue that you're spending too much time on testing. I've never had a problem to sell myself to them. I usually figure that I spend 50% of my time (or more!) writing tests and 50% actual coding - and I'm still much faster than those who write code 80% of the time or more. What's more: when my code goes into production, it's is rock solid or at least easy to fix when something comes up. In 99% of the cases, the things I need to fix were those which I didn't test.

This is a positive reinforcement loop which drives me to test more and more because it stops the hurting. If your tests cost more than they seem to return, you need to fix them until you get the same positive feeling when you think about them.

MQSeries 2045: MQRC_OPTION_NOT_VALID_FOR_TYPE

2008-07-08T13:14:00.003+02:00

While doing some work with MQSeries, I got an error "MQJE001: Completion Code 2, Reason 2045" in MQQueueManager.accessQueue() which translates to "MQRC_OPTION_NOT_VALID_FOR_TYPE". Hm. Hey, IBM, how about adding real error messages to your products instead of having people look up odd codes in tables?

Anyway, the error means that I'm trying to open a queue for output which doesn't support this. For example, remote queues can't be opened with the option MQC.MQOO_OUTPUT. Other queues don't allow to read from them, i.e. you have to get rid of MQC.MQOO_INPUT_AS_Q_DEF in the openOptions.

TurboGears 2.0 Is On Track

2008-07-08T11:04:00.002+02:00

There are three things which hooked me to TurboGears:

Every day stuff is simple, complex stuff is possible
Automatic reload after code change (no need to restart)
It's in Python

What I didn't like is that TG 2.0 has been so quiet for so long. I'm on the Planet Turbogears RSS feed and I wasn't sure whether 2.0 was alive or dead or whatever.

Well, it seems to be more alive than I expected and hopefully, we'll see a 2.0 soon. In "Doing the right thing should be easy" by Mark Ramm, you can find more details.

Starting Your Own OSS Project

2008-07-08T11:01:00.001+02:00

If you're planning to roll your own little OSS toy project, you should read the article "Party of one: Surviving the solo open source project" by Kirill Grouchnikov. Very good points on what to do and what to avoid and why.

Genes in Wikipedia

2008-07-08T10:54:00.003+02:00

So if you ever wanted to know how that stuff works that you're made of, scientists have started to put their knowledge about genes on Wikipedia. Beware, though, it's heavy stuff.

It's interesting to see how you can use an automated process to merge complex information from one system (the scientist's databases) into another. Now, I'm waiting for the "translate this goo into language xyz" bot :)

Hancock

2008-07-06T12:08:00.003+02:00

Strange movie. Here in Switzerland, it's sold as a "comedy" but it's not, and people will be disappointed. Also, I'm unhappy about the amount of futile violence and gore in the movie. There are a couple of scenes where you'll sit in you chair and think "What the f***!?". This is bad. While in the theater, you should never realize that you're watching a movie.

All in all, I think that the movie failed to deliver because it couldn't explain enough. Maybe it was too short or maybe the wrong scenes were chosen, I don't know. I left the cinema with a strange feeling of confusion, things just didn't add up. Unlike in other movies, I'm not able to say what they could have done different. After the big surprise in the second part of the movie, the behavior of the characters is suddenly consistent and you know why Hancock is such a bastard. Only, I don't know, it's as if something is lacking.

Hancock is shallow and that fits for a comic character but he's more a tragic character and this doesn't add up for me. So in the end, even when he finds his only love and gets killed for her, I don't really care anymore (as much as you care when Garfield gets flattened by a door).

See what others have to say.

Seeing Is Believing

2008-06-29T21:46:00.003+02:00

If you take a photo of something, does that photo show reality? What is there?

Most of us believe so but actually, the photo only shows what was there when you pressed the button. This misconception led to the Image Fulgurator. What's that? It's a device which projects a picture when it notices a flashlight, i.e. when someone makes a photo. Because of the flash, you can't see the image of the Fulgurator, but you will see it later on your photo. The photo shows what was there, not what is there.

Fan Programming Language

2008-06-26T22:46:00.003+02:00

Another language running on the VM which is worthwhile to check out: Fan.

Jazoon: The Closures Controversy

2008-06-26T22:24:00.003+02:00

If you ever wanted to know if it is possible to go through 60 information-packed slides in 50 minutes: Yes, it is. I've been there and Joshua pulled it off in from of about 100 people, so it's not a delusion of mine, either.

In his talk, Joshua presented a host of reasons why the BGGA proposal is a really bad idea. The key information here is "BGGA". We all want closures and Joshua is no exception, it's just that the BGGA proposal is like Generics on steroids and if you can't wrap your brain about Generics wildcards, then you won't understand BGGA closures as well.

The code in the proposal doesn't look too bad at first glance:

{int x, int y => x+y}

Unless you try to use an array. If you did "int[] x", you'd get an error because closures are based on generics and generics can't handle arrays or primitives.

But the next example is better to understand why BGGA will add a new level of hell to Java which will be worse than generics:

{int,String=>Number throws IOException} xyzzy;

is translated into

interface Function1 { // system-generated
R invoke(int x1, A2 x2) throws E;
}
Function1 xyzzy;

Doesn't bother you because you're never going to see this? Well, think again because if you try something that the generics type system doesn't understand, you'll get a generics error message like this one:

NewtonWithClosures.java:26: invoke(capture#418 of ? super {double => double}) in {capture#418 of ? super {double => double} => capture#928 of ? extends {double => double}} cannot be applied to (double)

The reason for this is that the closures are implemented using generics and without any high level support in the compiler so the compiler can't generate a more useful error message. And this was a simple example. We all know how quickly generics get message and for me, that means that any implementation of any new feature that is based on generics is a threat to the future of Java.

Again, I'm for closures and I use them as often as I can in Groovy but the current proposal is just a new way to make Java harder to use. Why don't they simply change the compiler to allow to access field and method objects (from java.lang.reflect) via the class? Like so:

collect(list, Math.class.min)

If we could use Method objects like first class citizens (instead of via the horrible reflection API with it's horde of checked exceptions), we could use Method objects as simple closures. Then, all we would need is a set of util classes for single mutable primitives and we're done. Sure, the syntax wouldn't be as compact but every normal Java developer would be able to understand the concept and how it's supposed to be applied in a few hours.

If you care, here is a link to the whole set of arguments by Joshua Bloch. Read it, keeping in mind that Joshua is pro closure, he just wants to avoid a second generics debacle.

Jazoon: One Year Of Groovy

2008-06-26T22:20:00.003+02:00

That was my talk. Attendance could have been better but I guess with Joshua Bloch presenting Java Puzzlers, Cédric Beust with "A quick guide to modern languages and interesting concepts for the busy Java programmer" and Adam Bien with Glassfish at the same time, it wasn't so bad.

Here is the link to the presentation as OpenOffice document. Feel free to post any comments or questions below.

Jazzon: Weblog in 15 minutes II

2008-06-26T22:09:00.002+02:00

I wish I could link that presentation because it was the best I've seen in the whole show, both visually and how it was presented. So here is the link to the abstract. There is something to be learned by this talk for anyone who wants to demo a software: One human being can't talk and use a computer at the same time. While David explained things, Bertrand wrote the code. Thumbs up!

And in this case, the visual candy didn't distract from the fact that these guys were really showing off something that ought to have an impact. They showed agile web development with Apache Sling.

A sling gives you range and power and Apache Sling does just that. With just a few pieces of JavaScript added to an existing static HTML web site, they built a blog in roughly 10 minutes. Impressive. I've got to try this out myself, probably this weekend.

Jazoon: Thursday Keynote

2008-06-26T22:01:00.002+02:00

Thursday's keynote was "Effective Java Reloaded" by Joshua Bloch where he presented a few key points from his new book.

The examples showed how to replace bit fields (or rather int constants which are used a bit fields) with EnumSet and EnumMap, and how to do lazy initialization for various cases effectively and correctly. If you're a Java programmer who does more than "Hello World", this book is a must have because it explains not only how to do things but how to do things elegantly and why.

Jazoon: Web Tests

2008-06-26T21:50:00.003+02:00

In his talk, Dierk König showed ways to test a web applications with canoo webtest. There are several way to come up with a test script: Recording the user actions with a browser plug-in or by writing the script directly, for example.

He also explained the best ways to test an old application (just test what you can, look for NPE's, for example, load all pages, make sure some properties appear). Of course, since the app wasn't designed to be testable, you'll be limited in what you will be able to test.

IFRAMEs, on the other hand, are not a problem, as are AJAX requests. The main issue with AJAX is that while they are still asynchronous, the test framework has some limitations as to what it can test (permutation of request order). Testing special code for IE and FF is not a problem, the framework supports some browser bugs, too. Also, you can have special code to login or to get the app into a certain state and use this code as a kind of "subroutine" in several tests to avoid code duplication.

Internally, webtest will use htmlunit to examine the HTML returned by the app and execute the JavaScript in it. So DOM manipulations can be tested, too.

It's not really suitable for load tests, though, since it lacks the features to run concurrently on several computers at once. Use JMeter for that.

If you need to prepare the DB before the tests, use dbunit.

Jazoon: Distributed Client/Server Persistence

2008-06-26T21:43:00.002+02:00

In his talk, Alexander Snaps presented a framework called Hölchoko which allows to cache objects from the server on the client. This is a bit like Gears but for Hibernate. No magic bullet, just a layer over the OR mapper to push objects over the wire, cache them in a local DB and make the merge with the server more simple once you're connected again.

See his blog for more details.

Jazoon: Spring and Maven 2

2008-06-26T21:37:00.004+02:00

After some technical difficulties (the only ones I saw during the whole show, well done Jazoon!), we got a short company placement. One or two sentences, next slide, that's how it's ought to be (if at all; I mean if I was interested in your company, I'd look at the web site and not at the presentation but I digress).

They presented the EL4J project which is the result of several years of developing web applications. From what I gathered, it makes it a bit more simple to wire web apps together using a bit of convention over configuration and Maven 2. It also has some Swing support in it.

Jazoon: Overview of Jazz and GWT

2008-06-26T21:31:00.003+02:00

After spending an hour talking to Ted and keeping him away from the work he had to do, I attended the presentation of Jazz. Another talk that would have benefited greatly from a live demo. Walked out after a few minutes to see the GWT presentation. The presenter of that one had the same problem. *sigh*

Guys, we're developers, not managers. Show us code and an IDE with a "Run" button.

Still stayed till the break so I could ask some questions.

Jazoon: Wednesday Keynotes

2008-06-26T21:21:00.002+02:00

I don't remember much from these keynotes except that Ted was great as usual and that the second keynote bored me to death. I did take some notes, though: I should have a look at Apache Sling (in incubation) and Erlang.

Ted gave some examples of simple questions which a third grader could solve but which most adults (even those with a degree) can't solve in any reasonable time. Most needed one hour or more. One was this:

Take two sticks, one is 4 meters and the other is 5 meters. Throw them in a well which is 3 meters wide. In which height do the sticks intersect?

It took me 15 minutes to come up with 2.7 meters (or (5*sqrt(34)) / (5+sqrt(34))). Anyone?

Jazoon Cut: Privacy Supporting Identity Systems

2008-06-26T20:29:00.002+02:00

Jazoon Cut is a nice idea: You got a project, they give you 20 minutes to present it (i.e. "cut" as in "cutting edge"). In this Cut, we had NetKernel, iGesture, Interactive Paper, and Privacy Supporting Identity Systems. A rather interesting mix.

When we buy something, we hand over money. This money is untraceable. If the vendor passes it on to his bank later, there is no way anyone could tell that it was you gave him this specific bank note or what you bought with it (the vendor might know that but again, he couldn't say which bank note you gave him). This is a good thing. If the bank could figure out what you bought, some people would become very interested in this data, for example marketing people. They are looking for a way to measure how easy we are to influence for ages.

But the area is getting more tight. If you had some RFID chips on you, say, one of those new passports or a contactless credit card, I could place some people in a shopping mall or city center with technology to track where you go. Under optimal conditions, an RFID chip can be tracked over the distance of 60 meters. That's not much and I probably need a couple of radio stations spread over the city to do that but with such a system, I could find out what you buy.

"Now," you say, "I can't be identified by that," and you'd be wrong. Some credit cards will offer all the details stored on them (enough to buy something with that data in the Internet) when asked. For the rest of you, you're lucky until someone figures a way out to do that. Nonetheless, even if I don't know your identity I can track you. So if your last stop was at an expensive jeweler, chances are that you have enough money that it's worthwhile to mug you.

And I can pick you from a safe distance of 20 to 60 meters, follow you around out of sight and wait for the perfect moment to strike. Oh, and better not step into some sex shop because I'd know. In fact, I can track your movements for a couple of days, find out where you live (and thus your identity). All without you ever knowing that I even exist.

I hope I have made you understand that the question is not that you have nothing to hide (which is not true; when was the last time you filed a correct tax declaration?), the question is what evil someone could come up with if he knew something about you. The problem with this question is: Nobody knows the answer until someone comes up with a new evil that nobody else thought about so far!

In his talk, Thomas Heydt-Benjamin showed how you might be able to have both: Comfort and security. The attack described above wouldn't work if the range of the RFID chip in your credit card would be reduced to a few centimeters. It would still be comfortable if you had to put it on the desk for a moment, making hard contact with the surface to be able to read it. Surely, you would notice if someone followed you holding a strange device to your pocket.

The next thing is the data on the card. The vendor needs the data to know who you are and if your credit is valid. But actually, he does never have to see the real data. The only thing he needs to know is "credit is valid" or not. He doesn't need your credit card number. Or the name. Or the expiration date. Or the security code on the back. What could be done is this:

You check into a hotel. You present your passport to the reader device on the desk. On your side of the screen, you can select which data the hotel guy can see. To allow the police to track you in case you didn't pay your bill or you're involved in some kind of crime, the hotel's computer gets an encrypted code that identifies you. For everyone who can't decrypt this key, it's just a long, random string of data. For all legal means and purposes, you're as anonymous as you want. While this might not make much sense in the hotel scenario (hotel staff hates it to call everyone "Mr. Smith"), it starts to make sense on the Internet.

You're stuck in some kind of game and want to check the walk through. Only, the game is for people over 16. So the site with the solution should check your age because the walk through contains all those images which were the reason for the age limitation. In this case, you only want to tell the site your age or rather that you're older than 16 and nothing else. You don't want to tell them your name, or where you live. Otherwise, the police might decide to shoot you for reasons of precaution (this is an example, okay?).

Or let's face the truth: 99% of the Internet is pr0n. And you surely don't want to leave your personal details with people who treat women like in the movies they sell.

With the Higgins framework, such things are possible. This framework allows to transfer data like "I'm older than 18" in such a way that the other side can be sure you are even though they will never find out what your real age is or your name. At the same time, it doesn't allow for illegal activity because you can be identified by your IP address and when the police compares the data sent to the web site in question and your passport, they can prove without doubt that it was your passport that was used in the transaction.

At that time, you want a passport that can't be skimmed in the subway.

While I talked to Mr. Heydt-Benjamin, he also explained to me that certain pacemakers contain a radio interface. Which makes sense. By this interface, you can query for the patient's data (name, for example) and you can also control the pacemaker. This interface is not protected by any encryption or password. So you can see the heart curves recorded by the sensors. But you can also change the sensor settings of the pacemaker. This is actually what happens when the doctors adjust them to your needs after the surgery.

The scary part is that the pacemaker has a function to stop a heart. This is necessary to "heal" certain kinds of heart rhythm irregularities like ventricular fibrillation. Now if I'm a hacker and I don't like you, I can turn off the sensors and stop your heart. The device will try to start your heart again after stopping it but that will fail because the settings of the sensors will prevent it to get any feedback.

So if you can see a large antenna at the horizon, the words "denial of service attack" might make you feel a little bit uneasy in the future if you do need a pacemaker.

Jazoon Cut: Interactive Paper

2008-06-26T20:26:00.001+02:00

This talk really intrigued me. As we all know, the "paperless office" really means "kill more trees". Every year, the work consumes about 5% more paper and that's despite declines in book and newspaper sales. The guys around Nadir Weibel came up with a system to link paper with the screen. When Nadir started to draw on the screen as he talked, nobody noticed at first. We were thinking he used a graphics tablet or something like that but he actually used a clipboard with a few sheets of paper on it. The paper was just normal paper with the exception of a fine grid of points printed on it which you can barely make out (The concept is explained here). You'll need a special pen that can recognize that pattern but after that, you're set.

The iPaper guys have created two demonstrations of their technology: PaperPoint and PaperProof. The former is a printout of a presentation with a menu and some buttons printed on the paper. So you can use the pen to control the presentation just by using your pen. You want to jump to a certain slide? Draw a dot on the "here" "button" on the printout and the computer will recognize where you are on the printout and sent the command to your presentation software. There are options to select a color and line width and then you can just draw on the paper and the same lines will appear on the screen. Hint: You will want to sit down when you do that; drawing something more complex than a little arrow while juggling the clipboard isn't for the easily irritated.

Not convinced? Well, PaperProof takes it step further. You don't actually have the computer running while you use iPaper. What you can do is print out some long text (like the book or article you're currently working on). Then you go out, relax in your deck-chair with a nice drink at your side and the sun above and edit that text, making corrections, marking errors, etc. With the traditional way, you eventually return to your computer and then you have to go through all that again copying everything you did from paper into your word processor.

PaperProof does that for you. It can recognize a few gestures and your handwriting. So when you return in this scenario, you start your computer, sync it with the pen and when you open your text document, it will already contain all the annotations, error corrections and insertions you made with your pen. Editors and authors around the world, isn't that a dream come true?

Jazoon Cut: iGesture

2008-06-26T20:25:00.000+02:00

The presentation of iGesture was a disaster from my point of view. The software is a workbench to create gesture recognition software or configuration packages for a gesture recognizer. That's it. And it took the people in the audience about the same time to figure this out as it took you to read this. After that, we waited for something to happen and nothing did. Shame. My suggestion: Next time, bring a laptop, show the first two slides of your presentation and then take a mouse and show the real thing: Train the thing and see the people rip it out of your hands.

I did learn a few tricks how you can recognize gestures, though. The most simple way is to filter the input until you have general directions like up (U), left (L), right (R) or down (D), then simply create a string with these characters and use a regular expression to describe the gesture (a rectangle would then be /U+R+D+L+/).

Jazoon Cut: NetKernel

2008-06-26T20:21:00.003+02:00

In the NetKernel talk, Brian Sletten again tried to sell his "RDF is the best and you should use it everywhere." Basically, NetKernel is a little core where you can register translation services (called ... I don't know what he called them and I can't find the link to the actual presentation, just the abstract :/). So when a service needs some data (or "resource"), he calls the kernel and the kernel figures out who might be able to serve that request which might go through several hoops before it comes back. Nothing fancy here, Unix pipes do that for ages with the exception that they don't build themselves.

His demo was to show how you could calculate Fibonacci numbers by using a "bsh" service (BeanShell, that's JavaScript) to add two intermediate numbers of the result. You would imagine that this is slow as hell with all that creating messages, sending them around, starting a JavaScript interpreter and run each add. As you might remember, the Fibonacci generators are usually implemented recursively and that should kill the NetKernel.

Only it doesn't. If you look at the runtime graphs, the Java version of the Fibonacci generator needs exponential time as the input grows. Around 30, the Java version takes seconds to run while the NetKernel version always needs to same amount of time. The nice thing about the design is that you can cache the results. So the call to fibonacci(30) will just add the cached results of fibonacci(29) and fibonacci(28) and be done. One level of recursion required.

While this is mighty impressive and surprising, the question remains how that will scale in reality. After all, caching a 500MB result from some service might not be feasible or even possible.

Jazoon: Data Driven Applications

2008-06-25T13:41:00.001+02:00

Not one of my favorite talks, to be frank. Brian Sletten tried to explain how we would write less code in the future because data would become more important. Seeing is believing and I haven't seen nothing, yet.

He has a few good point like that we should be able to recall data sets which we created with Web 2.0 sites (kind of like bookmarks on steroids that can save the state of the web 2.0 app as you bookmark it). Also, that the semantic web will make a lot of things possible and happen.

I buy that. But on the other hand, we don't even have a decent UI framework for Java yet and that's eight (8) years after Swing came out. This sure looks promising; I just wonder if I will be still around when it starts to deliver and companies get the CPU power and network throughput to really run all their data through RDF converters. And see the result before you get downsized.

Jazon: Building DSL's

2008-06-25T13:28:00.004+02:00

Neal Ford's talk about DSLs and how to create them in static and dynamic languages (Java and Groovy in this case) was very interesting because he clear up the fog around DSL's a but, why they are great and when they help. Basically a DSL is "jargon". It is compact code that allows you to express much more tightly what you want than any "general purpose" language ("one site fits all" actually makes everyone look ridiculous).

Remember that old OO promise to allow to mirror the world in a programming language, to allow to model the code after what the customer had in mind? Turns out, that OO was ... nah, not lying ... it was just overselling itself because customers don't think in objects or models. They think jargon. And DSL's allow you to make a compiler understand jargon. That's even possible with Java even though "Java is like taking to a retard": You always have to repeat yourself to make sure that the stupid compiler gets what you want (pun intended).

A DSL is not to be confused with an API. An API uses explicit context like "obj.setThis(...); obj.setThat(...);" etc. Here "obj" is the context and "setXyz()" is not meaningful without knowing the object we're talking about here. A DSL, on the other hand, is all about context and context is implicit. It goes "obj.should(...).allow(...).this(...)". An example is hamcrest: "assertThat(theBiscuit, is(myBiscuit));" Reads fluently, compact code, and it's obvious that "is(myBiscuit)" should be compared to "theBiscuit" in some way.

The handouts of his presentation are available from his homepage but without his witty remarks. A pity but still worth to check out if you want to see just how far you can push Java to get code that your customer might understand.

Jazoon: Rich GUI development with Java

2008-06-25T13:11:00.002+02:00

If you ever tried to develop a nice you with Java, you'll know the pain. Swing: Old, steep learning curve, unweildy widgets. SWT: Nice, good learning curve, "which JAR's do I need, Sir?" and who puts that DLL in my search path. There might be other contenders but I don't even remember their names.

So the guys at Canoo faces the dragon and chipped a few scales off it. As Bruno Schäffer said in his talk:

Developers should focus on the what and not how.

They used Swing, some components from SwingX, JGoodies Smart Client and Jemmy for testing for the UI and something called c3pO for JDBC connection pooling to build a smart client. openArchitectureWare was used to generate a model editor for the data model, so they could generate the DAO classes for that efficiently.

Their application is split into modules but they don't use OSGi or any other module framework out there because they got in their way, it's just good old, traditioal loading on demand from the classpath. I can relate to that. OSGi is so great and stuff ... but it somehow lost contact to the base. It's just eithet not flexible enough or not simple enough for me to understand and use.

Good talk but less product placement for your company next time, Bruno.

Jazoon: openID

2008-06-25T12:59:00.002+02:00

Robert Ott showed off openID. If you wondered about these strange links which start to pop up all over the web near the login buttons, openID is a way to register yourself once at a place you trust and then use that to login elsewhere.

The idea behind openID is that you click on the link, the web site passes you on to an identity Provider which can use any means to make sure you are you and then give that confidence back to the original site. See the openID.net website for details.

The cool thing about openID is that you just have to manage a single identity and use that to log in into various services on the net without giving all of them the same password (and thus make you very vulnerable agains attacks). The main problems which openID doesn't solve is stalking and spamming.

openID makes stalking easier because you have to enter your openID at every place you want to log in to. Same id means same person. There are talks in the openID community to allow a person to have more than one ID and still use the same identity provider but there is nothing definite, yet.

Also, openID doesn't protect against spammers. It might help, though, because of two things: a) You can black list identity providers which allow spammers and b) you can send the police to the other identity providers which will hand over the real identity of the person behind the openID to the authorities on court order. So this might help in the war against forum spam, we'll see.

All in all, a good talk.

Blogging From The Jazoon: Keynotes Tuesday

2008-06-24T16:04:00.004+02:00

So this is day one of the Jazzon with the keynotes and talks. I'll go through things as they happened.

Scalable Languages

Martin Odersky, father of Scala, gave an introduction to scalable languages or rather to Scala. Well, Scala looks really promising with its traits and other features. But my original objection still remainds: The syntax is just ugly. Proves my point that there are only five people on this planet who can design a language and of them, I only know Guido van Rossum.

Adoption-Led Market

Simon Phipps followed wth a talk about the "Adoption-Led Market". It's astonishing how many OSS projects Sun supports or started and how little is known about that. Marketing again. *sigh* His talk wasn't all about Sun but how people start to despise vendors or rather their promise that their product is going to deliver to the promise ... after you've paid for it. Of course in an OSS world, where you can just download something and try it out, it becomes increasingly hard for them to "justify their 1000% profit margin."

This is a very good point. With OSS software, you pay for what you need (and not what the vendor things is best for him\b\b\byou). If you need a feature, support or whatnot, well, you can always pay someone to give it to you. But then, you pay when you need it. That makes is a justifiable cost, not an arbitrary one. Also, support gets you what you need when you need it. Just imagine to ask MicroSoft for a specific feature in Windows which you need. How much would that cost? What are you're chances to actually get it? Forget it! But with OSS, for the first time in software history, you can get what you need for a reasonable price and you might even make someone on the other side of the planet very happy.

Of course, "the greatest threat to freedom is a happy slave" but the talk gave a lot of arguments how to sell OSS to your company, even if that company happens to be the government.

He also gave a good reason which OSS license to chose for your next OSS project:

A license is the constitution of a community
Eben Moglen

Nice guy, good talk. If you can, vote for his photo at his website.

Innovation

The third keynote was by Rod Johnson: "Where will tomorrows innovation in Java Enterprise come from?" After explaining where innovation comes from, he shows how standards kill innovation and how committees can't drive innocation due to political games by the companies who dispatch the people that form these committees plus general group stupidity.

After the more or less failure of the JCP in recent years, attributed mostly to ignoring feedback from the community and drowning ideas in ceremony, he hopes that JSR 316 (that's Java Enterprise version 6) will help to fix Java EE. If it does, then that might mean that a good sign that the JCP has become a helper for the Java cause instead of another reason to abandon it.

Jazoon 2008, Day 0

2008-06-23T22:31:00.003+02:00

Just back from the Jazoon 2008, Tutorial day (the day before the actual conference starts). I attended Ted Neward's talk "The Busy Java Developer's Guide to Groovy". As always, the talk was quite a lot of fun even for someone like me, who already knows a lot about Groovy. If you get a chance to attend a talk given by Ted: Go. It's worth it.

In his talk, Ted crammed a pretty complete introduction in the key features of Groovy into three hours. For me, the key points of the talk were these: Productivity and ceremony.

Research shows that all developers roughly write the same amount of code lines per day. The difference between guru developers and the greenhorns is that the gurus write a line once and it works first try and each line achieves much more in respect to reaching the goal than any line of a novice. So in order to boost productivity of a guru, the only way is to use a programming language which can achieve more in a single line of code (and we're talking one statement per line here, obviously). So it's not that the guru writes ten times as much code per day as the novice, the novice just writes every line ten times because every time, (s)he gets something wrong and has to fix it.

The other aspect is ceremony. A lot of code in Java is ceremony. In most cases, the Java compiler knows where a statement ends and it wouldn't need the semicolon. Still it insists that I press the key anyway (and it's two keys on the German or Swiss keyboard). The same goes for many other things in Java which are just there for tradition. It would be simple for the Java compiler to infer the type of an object after if (obj instanceof ...). There really is no need for the cast in the next line. I could find another 10 examples easily.

Groovy does away with all that. It comes at a price, though. If you like to rely on the compiler creating a cozy padded cell for you, checking types and syntax as much as possible, then Groovy is not for you. In Groovy, the idea is that the developer knows exactly what (s)he's doing and all (s)he needs is the fastest path to get there. You can do all the dirty things you wished for in Java but you don't have to. Groovy is an "also" language, not an "either-or" language. If you feel that Java is dirty enough, already, try Scala.

Thanks, Ted.

If you want to know more about Groovy and you're in Zurich, why not attend my talk on Thursday?

Collaborative Editing

2008-06-21T17:30:00.003+02:00

Mustafa K. Isik has a video on vimeo demonstrating collaborative editing of a Java class in Eclipse. I'm not sure where this leads and my experience winces at the idea but it would sure help if the superman of your team could attach to your editing session, see the code you're working on and help you fix it (instead of having to get up and coming over).

prefuse - Information Visualization Toolkit

2008-06-20T09:42:00.002+02:00

If you're looking for a cool way to visualize your data for the user in Java, try prefuse. The examples are impressive.

ePen

2008-06-19T11:27:00.004+02:00

After a long time, I'm starting a new OSS project called "ePen". From the description:

ePen - A cross platform tool for creative writers and book authors. ePen is able to manage all information around a book project: The actual text, descriptions of the characters and scenes, the storyline.

ePen should become for authors what a pen was for centuries: A simple tool to convert ideas into amazing, breathtaking stories.

Right now, ePen is in the planning stage even though I'll commit a prototype this weekend. The first thing on the agenda is to decide how to implement the tool. The prototype was done in Python and PyQt and while it was a nice experience, the text editor leaves a lot to be desired (read: it sucks). I've collected my thoughts in this wiki.

I've also create a Google Group for discussions.

How to Work Better

2008-06-17T10:21:00.002+02:00

From the wall of a corporate building in Zurich:

Do One Thing at a Time
Know the Problem
Learn to Listen
Learn to Ask Questions
Distinguish Sense from Nonsense
Accept Change as Inevitable
Admit Mistakes
Say It Simple
Be Calm
Smile

Communication Skill

2008-06-17T10:18:00.001+02:00

From a calendar:

The real art of conversation is not only to say the right thing at the right time, but also to leave unsaid the wrong thing at the tempting moment.
-- Author unknown

Eye Candy For Developers

2008-06-16T17:39:00.003+02:00

There are two new projects on the 'net which might influence the way we see software in the future (in the literal sense). code_swarm can visualize the CVS history of a project in a cunning way and Wordle creates tightly packed tag clouds.

What I find especially interesting about Wordle is that it creates the tag cloud in real time (as you watch) using a Java applet. If you happen to have Java 6 installed, the animations are very smooth, almost like Flash.

The Lucifer Effect

2008-06-13T10:10:00.000+02:00

I'm back from a 2 week holiday in Ischia (Italy). I like to take a book or two with me because I rarely find the leisure (and the concentration) to read something complex during my normal life. This time, it was The Lucifer Effect by Philip Zimbardo. If you haven't heard about this guy, you may have heard about the Stanford Prison Experiment (SPE). If you haven't heard about this as well, here is a short summary: Take 20 random, normal people (i.e. someone like you and me and your neighbor), put them into a situation where they can get away with anything and watch them turn into human nightmares within a few days.

The book explains in great detail how normal people turn evil. Evil in the sense that they stop acting kind and instead start to humiliate, torture, even kill other people. The hard part of the book is to understand that these people aren't evil to begin with. They are just ordinary people. For example, most suicide bombers are well educated, have an intact family background. Still, it's exactly these people who are most vulnerable to the mechanics which turn a nice guy into Darth Vader.

I'll point out a few places in the book in this post which impressed me most. On pages 167f, Jerry-5486 (one of the prisoners in the SPE) explained how he rationalized his own evil actions against a fellow prisoner by thinking "It's only a game, and I know it and I can endure it easy enough, and they can't bother my mind, so I'll go through the actions." Doing this, he could completely ignore how much his actions would hurt the people around him, even people he thought of as friends. Instead of supporting those he liked, he submitted to the cruel commands of the guards. Because of how our brains work in these circumstances, he only noticed what he had done afterwards.

On page 205, there is a great quote:

Life is the art of being well-deceived; and in order that the
deception may succeed it must be habitual and uninterrupted.
-- William Hazlitt, "On Pedantry", The Round Table, 1817

as is on page 208:

Wherever anyone is against his will, that is to him a prison
-- Epictetus, Discourses, second century A.D.

On page 210ff, Zimbardo explains the different factors which led to the humiliation/torture in the SPE: Situation, role expectations, responsibility (or lack thereof), anonymity and deindividuation, cognitive dissonance and social approval. On page 220, he explains how cognitive dissonance forces us to rationalize our actions when we hurt others:

Oddly enough, the dissonance effect becomes greater as the justification for such behavior decreases, for instance, when a repugnant action is carried out for little money, without threat, and with only minimal sufficient justification or inadequate rationale provided for the action. Dissonance mounts, and the attempts to reduce it are greatest, when the person has a sense of free will or when she or he does not notice or fully appreciate the situational pressures urging enactment of the discrepant action.

Instead of using our brain to stop what we do, we use it to explain the pain away. The less reason someone has to be cruel, the stronger the desire to give the cruel actions a reason that will make it "look good". Ever wondered why anyone will come up with such unreal excuses when they are caught? That's why.

In out daily life, we feel that our reason is in control when in fact, it's the situation that shapes how we see the world and thus forces our hand without our reason even noticing. This is what happens when the government calls the population to arms because of a "threat to national security". Ironically, it was always those who yelled that were the threat (and not those who were accused). Unfortunately, situational forces, our shared desire for social approval, drives us to believe the yellers even when five minutes of rational thought would show how silly their words and how false their motives must be (p. 226ff). This is what lead to slaughter of the Jews in World War II, the Challenger disaster, the genocide in Rwanda and the abuses in Abu Ghraib (which was one of the more harmless incidents in US prisons in the area). It's always the same mechanisms which lead to these results.

The basics of these mechanisms are listed on p. 230: Our need "for consistency and rationality", our need "to know and to understand our environment and our relationship to it" and "our need for stimulation". Also, how we sense time has a great influence on our behavior (p. 243): Do we believe in the past? The future? Do we have reason to plan or do we sit back in depression because we feel all our plans and hopes are in vain?

These same mechanisms are used by the military to train normal people (which still have an inhibition to kill) into people who can kill on command. A visitor to the SPE website explains how the drill instructors used the usual old, irrational lie to punish the group for a mistake: "if you guys would have moved faster, we wouldn't be doing this for hours". Well, the only reason why the group is going "this" is because of the drill instructor (p. 256).

Tests show that group pressure is a great tool to create obedience. Imagine you had to judge the length of a line on a card against three reference cards. Doing the test alone, you'd get it right more than 99% of the time. But what if you were in a group with several people who would chose the wrong answer on purpose? Could you resist the urge to comply to the group or would you submit? When the experiment was done, 70% of 123 test persons caved and agreed with the group that two obviously different lines had the same length (p. 263). Modern technologies allow to look into the brain and help to understand what happens. If you deviate from the group's consensus, the images from your brain will show that "resistance creates an emotional burden for those who maintain their independence -- autonomy comes at a psychic cost" (p. 265). The images also show that often, you don't even notice the conflict. The brain will adjust it's perception of the world to what the group wants to believe before reason is even asked for its opinion.

That all leads to Milgram's experiment where unsuspecting test persons would be lead by the experimenter to apply deadly electrical shocks to another person. Of course, the other person was a confederate and the shocks were never actually applied but the test person didn't know that. 66% of the test persons would go all the way even when the "victim" would scream in pain and beg for mercy. Most would even continue after the "victim" stopped responding. Because the results were so shocking, the experiment was repeated with different people in different countries. With different settings, Milgram could get the compliance rate (people who would go through the whole hell until the bitter end) from 10% to 90%. It didn't matter who was tested but how. From the results of his various experiments, Milgram compiled a list of ten methods to turn anyone into a torturer (p. 273f):

Create a sense of obligation (for example, make a contract or say "you have to protect your country/family/whatever").
Give the participant a positive role (teacher, protector, soldier)
Present a set of rules that seem to make sense at first glance. It doesn't matter if they are really consistent since you can later change them arbitrarily.
Make the job sound harmless. Don't say "hurt the prisoner", say "protect the country".
Make sure no one really feels responsible for what happens or that everyone can point at someone else to blame when something bad happens. In Milgram's Experiment, a lot of people stopped unless the experimenter told them "I'll accept full responsibility if the other person is harmed".
Start with an act that points in the general direction but doesn't seem too bad ("We'll just lock these people up to protect society").
Work with small, seemingly insignificant steps ("that's not much different from what I did yesterday"). Humans are great for detecting big changes but they suck with little ones.
Gradually change the nature of the authority figure (leader, boss) from "just and reasonable" to "unjust, demanding, even irrational". If you do this in little steps, the subject won't notice and still believe in the "first impression" even after the transformation is complete. At that stage, the leader can ask anything and (s)he will get it.
Make exiting hard. People are allowed to complain but insist that they still follow the orders to the letter ("Do you want to ruin everything we achieved?")
Offer an ideology, an explanation or a big lie to justify the use of any means to achieve the seemingly desirable ends.

If you get everything right, 90% of all people will follow you to hell and beyond. If you take these rules, you can easily find parallels to the many dictatorships around the planet. That's how and why they work and why it is so hard to break them down from within.

There are also references to anonymity experiments with schoolchildren during Halloween (p. 301) which shows what happens if people can be aggressive anonymously. In this scenario, they will act more aggressive unless they can be identified, even if their aggressive behavior is not in their best interest.

Being part of a group apparently causes us to reduce our self-awareness (a process called deindividuation, p. 305) and our ratio (our "good" sides) in favor of our "Dionysian nature", where instincts, lust and desire rule. This is what creates the strong group cohesion or "peer pressure" which allows us to create a civilization despite our egos. But if you don't know about these effects, then they can get out of hand, driving the group ever deeper towards chaos or evil.

These effects are enhanced when we see other people as inferior, non-human, animals or objects. In the end, it leads to a state of moral disengagement (p. 310). In this state, the people of the group are still able to understand that their actions are wrong but they don't care anymore. This state allows people to kill, torture, rape.

We don't even have to go that far. People are less likely to help when they are in a hurry (p. 316). The same effect is at work when a large group of bystanders watch someone suffer and don't intervene to help. Or when a sane person is taken to a mental hospital (p. 321). Staff members of a hospital played the role of patients for a short period of time and were very surprised how they were treated by their colleagues on duty. In the end, lawyers had to demand their release or they would still be in the ward.

Or Abu Ghraib. In chapter 14, Zimbardo dissects the situation and the people behind the media hype we all know from the images in the news shows. An apparently normal, mental healthy soldier is thrown into a situation where a life doesn't count much, there is no support from outside or superiors, the place is under constant attack by the locals and local forces that are supposed to help instead smuggle weapons into the cells of the prisoners for money and your superiors order you to torture to get intelligence from the prisoners. And the icing of the cake is that you get the night shift which means sleep deprivation. How long could you withstand such a stress? Staff Sergeant Chip Frederick (that's the guy who got all the blame for the incident) made it for more than two months until he caved (p. 343).

From the official news, you might get the idea that he was accused to have tortured the prisoners. I find that hard to believe. Others have killed prisoners and are still walking free (see "The Iceman"). From my point of view, he was convicted for making trophy photos. Other teams with similar records were instructed to destroy the evidence before returning home. The kind reader is welcome to draw his/her own conclusions.

This pattern of taking trophy photos comes up again and again when people are abused. It happened when black men and women were lynched in the US, German soldiers took them after committing atrocities against Polish Jews and Russians, etc. The list is endless. But there is a difference between these photos and real trophy photos (taken when a big-game hunter had his shot). On the original photos, the hunters don't laugh and cheer. They appear "rather grim, rarely are they smiling: they are victors in a battle against formidable adversaries." (p. 364).

The author goes on to explain how the Bush government actively helped to produce the environment which lead to abuses like in Abu Ghraib and many other places in Afghanistan and the Iraq, Guantanamo, and probably elsewhere, too. The last figure I know is of 600 similar incidents. An interesting if controversial read.

Chapter 16 contains the most important part of the book: How to fight evil. Not on a big scale but on a personal one. How to notice and withstand forces which try corrupt you. On page 451ff, you can find a list of ten steps to "resist unwanted influences". I especially like "Reject simple solutions as quick fixes for complex personal or social problems." Print that out and read it twice every morning.

Even if you don't read the rest of the book, look at these few pages. They alone are worth the buy.

In the last part of the book, Zimbardo takes a look at heroism. Unlike popular belief, heroes and heroines are ordinary people who just did the right thing at the right time at great personal risk because they believed they had to do it (p. 457). Or as the author puts it (p. 466):

Heroism can be defined as having four key features: (a) it must be engaged in voluntarily; (b) it must invoke a risk or potential sacrifice, such as the threat of death, an immediate threat to the physical integrity, a long-term treat to health, or the potential for serious degradation of one's quality of life; (c) it must be conducted in service to one or more other people or the community as a whole; and (d) it must be without secondary, extrinsic gain anticipated at the time of the act.

Recommendation: Must Have.

The Craft of Text Editing

2008-06-12T18:05:00.003+02:00

While looking for a decent code base for a text editor on Google Code (tip: search for "label:editor"), I found a wiki page in the epytor project which links to "The Craft of Text Editing", by Craig A. Finseth. That document not only explains all the basic algorithms behind the scenes of a text editor but also compares different implementations and contains a wealth of useful general comments.

Does It Have To Be a Scrollbar?

2008-06-12T14:17:00.003+02:00

Every once in a while, someone comes up with a nice idea and everyone adopts it. People needed a way to navigate in a document larger than the screen, so scrollbars were invented to give an idea where they are related to the whole document.

Scrollbars offer a consistent interface, they are well understood and they don't change size while you use them. That's good. But they also fail give you an idea where to look when you search something. Like: Where did I modify my document? What's above and below?

farmhand shows this information. Instead of a gray/blue blob that moves, you get a zoomed view of the document along with change marks. It takes a bit more space on the screen and I'm not totally convinced how it fares with really large documents but it's definitely a new idea with some potential.

I'd just put these "farmhand bars" as pop ups on the editor tabs, so they don't clutter the screen when hidden and are still easily accessible when you switch editors.

Rich Applications in 2008

2008-06-11T17:28:00.003+02:00

For about five years, I'm trying to write a small OSS application to help me write SciFi stories. Today again, I ran into this familiar pain: There is just no good OSS framework to write applications. Here is what I tried:

PyQt 4.4. This is nice but you need to do a lot of manual plumbing to connect your Python model to the Qt UI. Why are there no support classes to display a Python list in a QListView? Or to map fields from objects in a list to a QTableView? You can't even simply connect a field of an object to a basic text widget.

There are other Python frameworks (wxWidget, for example) but they are even worse (less features, more complex to use, no data binding as well).

So let's look at Java. What do we have? Spring Rich Client is more or less dead (they say they are working on Spring Desktop but there isn't even a web site, yet). Also, this project is locked to Swing. Not a killer feature in 2008.

So let's look at Eclipse. We have SWT, Data Binding and ... well ... EMF. Some love EMF, some like it, others ... you get the idea. Also, this solution again asks for a lot of plumbing (manually connecting objects with the UI). And while I gave Eclipse 3.4 RC4 a try (to see if EMF got usable in the last three years), it crashed in USER32.dll. *sigh*

Then, there is TurboGears. Automatic data binding, automatic generation of a nice UI, cool. Web only ... *sigh* Maybe I should try it with Firefox and XUL.

Also, none of these frameworks offer a simple WYSIWYG editor which can be extended (for example with a typing help, tooltips, proportional font support).

After ten years of OSS application frameworks, you'd expect that there was at least a single choice which allows to build a toy application with ten simple objects within a few minutes.