Sunday, June 29, 2008

Seeing Is Believing

If you take a photo of something, does that photo show reality? What is there?

Most of us believe so but actually, the photo only shows what was there when you pressed the button. This misconception led to the Image Fulgurator. What's that? It's a device which projects a picture when it notices a flashlight, i.e. when someone makes a photo. Because of the flash, you can't see the image of the Fulgurator, but you will see it later on your photo. The photo shows what was there, not what is there.

Thursday, June 26, 2008

Fan Programming Language

Another language running on the VM which is worthwhile to check out: Fan.

Jazoon: The Closures Controversy

If you ever wanted to know if it is possible to go through 60 information-packed slides in 50 minutes: Yes, it is. I've been there and Joshua pulled it off in from of about 100 people, so it's not a delusion of mine, either.

In his talk, Joshua presented a host of reasons why the BGGA proposal is a really bad idea. The key information here is "BGGA". We all want closures and Joshua is no exception, it's just that the BGGA proposal is like Generics on steroids and if you can't wrap your brain about Generics wildcards, then you won't understand BGGA closures as well.

The code in the proposal doesn't look too bad at first glance:

{int x, int y => x+y}

Unless you try to use an array. If you did "int[] x", you'd get an error because closures are based on generics and generics can't handle arrays or primitives.

But the next example is better to understand why BGGA will add a new level of hell to Java which will be worse than generics:

{int,String=>Number throws IOException} xyzzy;

is translated into

interface Function1 { // system-generated
    R invoke(int x1, A2 x2) throws E;
Function1 xyzzy;

Doesn't bother you because you're never going to see this? Well, think again because if you try something that the generics type system doesn't understand, you'll get a generics error message like this one: invoke(capture#418 of ? super {double => double}) in {capture#418 of ? super {double => double} => capture#928 of ? extends {double => double}} cannot be applied to (double)

The reason for this is that the closures are implemented using generics and without any high level support in the compiler so the compiler can't generate a more useful error message. And this was a simple example. We all know how quickly generics get message and for me, that means that any implementation of any new feature that is based on generics is a threat to the future of Java.

Again, I'm for closures and I use them as often as I can in Groovy but the current proposal is just a new way to make Java harder to use. Why don't they simply change the compiler to allow to access field and method objects (from java.lang.reflect) via the class? Like so:

collect(list, Math.class.min)

If we could use Method objects like first class citizens (instead of via the horrible reflection API with it's horde of checked exceptions), we could use Method objects as simple closures. Then, all we would need is a set of util classes for single mutable primitives and we're done. Sure, the syntax wouldn't be as compact but every normal Java developer would be able to understand the concept and how it's supposed to be applied in a few hours.

If you care, here is a link to the whole set of arguments by Joshua Bloch. Read it, keeping in mind that Joshua is pro closure, he just wants to avoid a second generics debacle.

Jazoon: One Year Of Groovy

That was my talk. Attendance could have been better but I guess with Joshua Bloch presenting Java Puzzlers, Cédric Beust with "A quick guide to modern languages and interesting concepts for the busy Java programmer" and Adam Bien with Glassfish at the same time, it wasn't so bad.

Here is the link to the presentation as OpenOffice document. Feel free to post any comments or questions below.

Jazzon: Weblog in 15 minutes II

I wish I could link that presentation because it was the best I've seen in the whole show, both visually and how it was presented. So here is the link to the abstract. There is something to be learned by this talk for anyone who wants to demo a software: One human being can't talk and use a computer at the same time. While David explained things, Bertrand wrote the code. Thumbs up!

And in this case, the visual candy didn't distract from the fact that these guys were really showing off something that ought to have an impact. They showed agile web development with Apache Sling.

A sling gives you range and power and Apache Sling does just that. With just a few pieces of JavaScript added to an existing static HTML web site, they built a blog in roughly 10 minutes. Impressive. I've got to try this out myself, probably this weekend.

Jazoon: Thursday Keynote

Thursday's keynote was "Effective Java Reloaded" by Joshua Bloch where he presented a few key points from his new book.

The examples showed how to replace bit fields (or rather int constants which are used a bit fields) with EnumSet and EnumMap, and how to do lazy initialization for various cases effectively and correctly. If you're a Java programmer who does more than "Hello World", this book is a must have because it explains not only how to do things but how to do things elegantly and why.

Jazoon: Web Tests

In his talk, Dierk König showed ways to test a web applications with canoo webtest. There are several way to come up with a test script: Recording the user actions with a browser plug-in or by writing the script directly, for example.

He also explained the best ways to test an old application (just test what you can, look for NPE's, for example, load all pages, make sure some properties appear). Of course, since the app wasn't designed to be testable, you'll be limited in what you will be able to test.

IFRAMEs, on the other hand, are not a problem, as are AJAX requests. The main issue with AJAX is that while they are still asynchronous, the test framework has some limitations as to what it can test (permutation of request order). Testing special code for IE and FF is not a problem, the framework supports some browser bugs, too. Also, you can have special code to login or to get the app into a certain state and use this code as a kind of "subroutine" in several tests to avoid code duplication.

Internally, webtest will use htmlunit to examine the HTML returned by the app and execute the JavaScript in it. So DOM manipulations can be tested, too.

It's not really suitable for load tests, though, since it lacks the features to run concurrently on several computers at once. Use JMeter for that.

If you need to prepare the DB before the tests, use dbunit.

Jazoon: Distributed Client/Server Persistence

In his talk, Alexander Snaps presented a framework called Hölchoko which allows to cache objects from the server on the client. This is a bit like Gears but for Hibernate. No magic bullet, just a layer over the OR mapper to push objects over the wire, cache them in a local DB and make the merge with the server more simple once you're connected again.

See his blog for more details.

Jazoon: Spring and Maven 2

After some technical difficulties (the only ones I saw during the whole show, well done Jazoon!), we got a short company placement. One or two sentences, next slide, that's how it's ought to be (if at all; I mean if I was interested in your company, I'd look at the web site and not at the presentation but I digress).

They presented the EL4J project which is the result of several years of developing web applications. From what I gathered, it makes it a bit more simple to wire web apps together using a bit of convention over configuration and Maven 2. It also has some Swing support in it.

Jazoon: Overview of Jazz and GWT

After spending an hour talking to Ted and keeping him away from the work he had to do, I attended the presentation of Jazz. Another talk that would have benefited greatly from a live demo. Walked out after a few minutes to see the GWT presentation. The presenter of that one had the same problem. *sigh*

Guys, we're developers, not managers. Show us code and an IDE with a "Run" button.

Still stayed till the break so I could ask some questions.

Jazoon: Wednesday Keynotes

I don't remember much from these keynotes except that Ted was great as usual and that the second keynote bored me to death. I did take some notes, though: I should have a look at Apache Sling (in incubation) and Erlang.

Ted gave some examples of simple questions which a third grader could solve but which most adults (even those with a degree) can't solve in any reasonable time. Most needed one hour or more. One was this:

Take two sticks, one is 4 meters and the other is 5 meters. Throw them in a well which is 3 meters wide. In which height do the sticks intersect?

It took me 15 minutes to come up with 2.7 meters (or (5*sqrt(34)) / (5+sqrt(34))). Anyone?

Jazoon Cut: Privacy Supporting Identity Systems

Jazoon Cut is a nice idea: You got a project, they give you 20 minutes to present it (i.e. "cut" as in "cutting edge"). In this Cut, we had NetKernel, iGesture, Interactive Paper, and Privacy Supporting Identity Systems. A rather interesting mix.

When we buy something, we hand over money. This money is untraceable. If the vendor passes it on to his bank later, there is no way anyone could tell that it was you gave him this specific bank note or what you bought with it (the vendor might know that but again, he couldn't say which bank note you gave him). This is a good thing. If the bank could figure out what you bought, some people would become very interested in this data, for example marketing people. They are looking for a way to measure how easy we are to influence for ages.

But the area is getting more tight. If you had some RFID chips on you, say, one of those new passports or a contactless credit card, I could place some people in a shopping mall or city center with technology to track where you go. Under optimal conditions, an RFID chip can be tracked over the distance of 60 meters. That's not much and I probably need a couple of radio stations spread over the city to do that but with such a system, I could find out what you buy.

"Now," you say, "I can't be identified by that," and you'd be wrong. Some credit cards will offer all the details stored on them (enough to buy something with that data in the Internet) when asked. For the rest of you, you're lucky until someone figures a way out to do that. Nonetheless, even if I don't know your identity I can track you. So if your last stop was at an expensive jeweler, chances are that you have enough money that it's worthwhile to mug you.

And I can pick you from a safe distance of 20 to 60 meters, follow you around out of sight and wait for the perfect moment to strike. Oh, and better not step into some sex shop because I'd know. In fact, I can track your movements for a couple of days, find out where you live (and thus your identity). All without you ever knowing that I even exist.

I hope I have made you understand that the question is not that you have nothing to hide (which is not true; when was the last time you filed a correct tax declaration?), the question is what evil someone could come up with if he knew something about you. The problem with this question is: Nobody knows the answer until someone comes up with a new evil that nobody else thought about so far!

In his talk, Thomas Heydt-Benjamin showed how you might be able to have both: Comfort and security. The attack described above wouldn't work if the range of the RFID chip in your credit card would be reduced to a few centimeters. It would still be comfortable if you had to put it on the desk for a moment, making hard contact with the surface to be able to read it. Surely, you would notice if someone followed you holding a strange device to your pocket.

The next thing is the data on the card. The vendor needs the data to know who you are and if your credit is valid. But actually, he does never have to see the real data. The only thing he needs to know is "credit is valid" or not. He doesn't need your credit card number. Or the name. Or the expiration date. Or the security code on the back. What could be done is this:

You check into a hotel. You present your passport to the reader device on the desk. On your side of the screen, you can select which data the hotel guy can see. To allow the police to track you in case you didn't pay your bill or you're involved in some kind of crime, the hotel's computer gets an encrypted code that identifies you. For everyone who can't decrypt this key, it's just a long, random string of data. For all legal means and purposes, you're as anonymous as you want. While this might not make much sense in the hotel scenario (hotel staff hates it to call everyone "Mr. Smith"), it starts to make sense on the Internet.

You're stuck in some kind of game and want to check the walk through. Only, the game is for people over 16. So the site with the solution should check your age because the walk through contains all those images which were the reason for the age limitation. In this case, you only want to tell the site your age or rather that you're older than 16 and nothing else. You don't want to tell them your name, or where you live. Otherwise, the police might decide to shoot you for reasons of precaution (this is an example, okay?).

Or let's face the truth: 99% of the Internet is pr0n. And you surely don't want to leave your personal details with people who treat women like in the movies they sell.

With the Higgins framework, such things are possible. This framework allows to transfer data like "I'm older than 18" in such a way that the other side can be sure you are even though they will never find out what your real age is or your name. At the same time, it doesn't allow for illegal activity because you can be identified by your IP address and when the police compares the data sent to the web site in question and your passport, they can prove without doubt that it was your passport that was used in the transaction.

At that time, you want a passport that can't be skimmed in the subway.

While I talked to Mr. Heydt-Benjamin, he also explained to me that certain pacemakers contain a radio interface. Which makes sense. By this interface, you can query for the patient's data (name, for example) and you can also control the pacemaker. This interface is not protected by any encryption or password. So you can see the heart curves recorded by the sensors. But you can also change the sensor settings of the pacemaker. This is actually what happens when the doctors adjust them to your needs after the surgery.

The scary part is that the pacemaker has a function to stop a heart. This is necessary to "heal" certain kinds of heart rhythm irregularities like ventricular fibrillation. Now if I'm a hacker and I don't like you, I can turn off the sensors and stop your heart. The device will try to start your heart again after stopping it but that will fail because the settings of the sensors will prevent it to get any feedback.

So if you can see a large antenna at the horizon, the words "denial of service attack" might make you feel a little bit uneasy in the future if you do need a pacemaker.

Jazoon Cut: Interactive Paper

Jazoon Cut is a nice idea: You got a project, they give you 20 minutes to present it (i.e. "cut" as in "cutting edge"). In this Cut, we had NetKernel, iGesture, Interactive Paper, and Privacy Supporting Identity Systems. A rather interesting mix.

This talk really intrigued me. As we all know, the "paperless office" really means "kill more trees". Every year, the work consumes about 5% more paper and that's despite declines in book and newspaper sales. The guys around Nadir Weibel came up with a system to link paper with the screen. When Nadir started to draw on the screen as he talked, nobody noticed at first. We were thinking he used a graphics tablet or something like that but he actually used a clipboard with a few sheets of paper on it. The paper was just normal paper with the exception of a fine grid of points printed on it which you can barely make out (The concept is explained here). You'll need a special pen that can recognize that pattern but after that, you're set.

The iPaper guys have created two demonstrations of their technology: PaperPoint and PaperProof. The former is a printout of a presentation with a menu and some buttons printed on the paper. So you can use the pen to control the presentation just by using your pen. You want to jump to a certain slide? Draw a dot on the "here" "button" on the printout and the computer will recognize where you are on the printout and sent the command to your presentation software. There are options to select a color and line width and then you can just draw on the paper and the same lines will appear on the screen. Hint: You will want to sit down when you do that; drawing something more complex than a little arrow while juggling the clipboard isn't for the easily irritated.

Not convinced? Well, PaperProof takes it step further. You don't actually have the computer running while you use iPaper. What you can do is print out some long text (like the book or article you're currently working on). Then you go out, relax in your deck-chair with a nice drink at your side and the sun above and edit that text, making corrections, marking errors, etc. With the traditional way, you eventually return to your computer and then you have to go through all that again copying everything you did from paper into your word processor.

PaperProof does that for you. It can recognize a few gestures and your handwriting. So when you return in this scenario, you start your computer, sync it with the pen and when you open your text document, it will already contain all the annotations, error corrections and insertions you made with your pen. Editors and authors around the world, isn't that a dream come true?

Jazoon Cut: iGesture

Jazoon Cut is a nice idea: You got a project, they give you 20 minutes to present it (i.e. "cut" as in "cutting edge"). In this Cut, we had NetKernel, iGesture, Interactive Paper, and Privacy Supporting Identity Systems. A rather interesting mix.

The presentation of iGesture was a disaster from my point of view. The software is a workbench to create gesture recognition software or configuration packages for a gesture recognizer. That's it. And it took the people in the audience about the same time to figure this out as it took you to read this. After that, we waited for something to happen and nothing did. Shame. My suggestion: Next time, bring a laptop, show the first two slides of your presentation and then take a mouse and show the real thing: Train the thing and see the people rip it out of your hands.

I did learn a few tricks how you can recognize gestures, though. The most simple way is to filter the input until you have general directions like up (U), left (L), right (R) or down (D), then simply create a string with these characters and use a regular expression to describe the gesture (a rectangle would then be /U+R+D+L+/).

Jazoon Cut: NetKernel

Jazoon Cut is a nice idea: You got a project, they give you 20 minutes to present it (i.e. "cut" as in "cutting edge"). In this Cut, we had NetKernel, iGesture, Interactive Paper, and Privacy Supporting Identity Systems. A rather interesting mix.

In the NetKernel talk, Brian Sletten again tried to sell his "RDF is the best and you should use it everywhere." Basically, NetKernel is a little core where you can register translation services (called ... I don't know what he called them and I can't find the link to the actual presentation, just the abstract :/). So when a service needs some data (or "resource"), he calls the kernel and the kernel figures out who might be able to serve that request which might go through several hoops before it comes back. Nothing fancy here, Unix pipes do that for ages with the exception that they don't build themselves.

His demo was to show how you could calculate Fibonacci numbers by using a "bsh" service (BeanShell, that's JavaScript) to add two intermediate numbers of the result. You would imagine that this is slow as hell with all that creating messages, sending them around, starting a JavaScript interpreter and run each add. As you might remember, the Fibonacci generators are usually implemented recursively and that should kill the NetKernel.

Only it doesn't. If you look at the runtime graphs, the Java version of the Fibonacci generator needs exponential time as the input grows. Around 30, the Java version takes seconds to run while the NetKernel version always needs to same amount of time. The nice thing about the design is that you can cache the results. So the call to fibonacci(30) will just add the cached results of fibonacci(29) and fibonacci(28) and be done. One level of recursion required.

While this is mighty impressive and surprising, the question remains how that will scale in reality. After all, caching a 500MB result from some service might not be feasible or even possible.

Wednesday, June 25, 2008

Jazoon: Data Driven Applications

Not one of my favorite talks, to be frank. Brian Sletten tried to explain how we would write less code in the future because data would become more important. Seeing is believing and I haven't seen nothing, yet.

He has a few good point like that we should be able to recall data sets which we created with Web 2.0 sites (kind of like bookmarks on steroids that can save the state of the web 2.0 app as you bookmark it). Also, that the semantic web will make a lot of things possible and happen.

I buy that. But on the other hand, we don't even have a decent UI framework for Java yet and that's eight (8) years after Swing came out. This sure looks promising; I just wonder if I will be still around when it starts to deliver and companies get the CPU power and network throughput to really run all their data through RDF converters. And see the result before you get downsized.

Jazon: Building DSL's

Neal Ford's talk about DSLs and how to create them in static and dynamic languages (Java and Groovy in this case) was very interesting because he clear up the fog around DSL's a but, why they are great and when they help. Basically a DSL is "jargon". It is compact code that allows you to express much more tightly what you want than any "general purpose" language ("one site fits all" actually makes everyone look ridiculous).

Remember that old OO promise to allow to mirror the world in a programming language, to allow to model the code after what the customer had in mind? Turns out, that OO was ... nah, not lying ... it was just overselling itself because customers don't think in objects or models. They think jargon. And DSL's allow you to make a compiler understand jargon. That's even possible with Java even though "Java is like taking to a retard": You always have to repeat yourself to make sure that the stupid compiler gets what you want (pun intended).

A DSL is not to be confused with an API. An API uses explicit context like "obj.setThis(...); obj.setThat(...);" etc. Here "obj" is the context and "setXyz()" is not meaningful without knowing the object we're talking about here. A DSL, on the other hand, is all about context and context is implicit. It goes "obj.should(...).allow(...).this(...)". An example is hamcrest: "assertThat(theBiscuit, is(myBiscuit));" Reads fluently, compact code, and it's obvious that "is(myBiscuit)" should be compared to "theBiscuit" in some way.

The handouts of his presentation are available from his homepage but without his witty remarks. A pity but still worth to check out if you want to see just how far you can push Java to get code that your customer might understand.

Jazoon: Rich GUI development with Java

If you ever tried to develop a nice you with Java, you'll know the pain. Swing: Old, steep learning curve, unweildy widgets. SWT: Nice, good learning curve, "which JAR's do I need, Sir?" and who puts that DLL in my search path. There might be other contenders but I don't even remember their names.

So the guys at Canoo faces the dragon and chipped a few scales off it. As Bruno Schäffer said in his talk:

Developers should focus on the what and not how.

They used Swing, some components from SwingX, JGoodies Smart Client and Jemmy for testing for the UI and something called c3pO for JDBC connection pooling to build a smart client. openArchitectureWare was used to generate a model editor for the data model, so they could generate the DAO classes for that efficiently.

Their application is split into modules but they don't use OSGi or any other module framework out there because they got in their way, it's just good old, traditioal loading on demand from the classpath. I can relate to that. OSGi is so great and stuff ... but it somehow lost contact to the base. It's just eithet not flexible enough or not simple enough for me to understand and use.

Good talk but less product placement for your company next time, Bruno.

Jazoon: openID

Robert Ott showed off openID. If you wondered about these strange links which start to pop up all over the web near the login buttons, openID is a way to register yourself once at a place you trust and then use that to login elsewhere.

The idea behind openID is that you click on the link, the web site passes you on to an identity Provider which can use any means to make sure you are you and then give that confidence back to the original site. See the website for details.

The cool thing about openID is that you just have to manage a single identity and use that to log in into various services on the net without giving all of them the same password (and thus make you very vulnerable agains attacks). The main problems which openID doesn't solve is stalking and spamming.

openID makes stalking easier because you have to enter your openID at every place you want to log in to. Same id means same person. There are talks in the openID community to allow a person to have more than one ID and still use the same identity provider but there is nothing definite, yet.

Also, openID doesn't protect against spammers. It might help, though, because of two things: a) You can black list identity providers which allow spammers and b) you can send the police to the other identity providers which will hand over the real identity of the person behind the openID to the authorities on court order. So this might help in the war against forum spam, we'll see.

All in all, a good talk.

Tuesday, June 24, 2008

Blogging From The Jazoon: Keynotes Tuesday

So this is day one of the Jazzon with the keynotes and talks. I'll go through things as they happened.

Scalable Languages

Martin Odersky, father of Scala, gave an introduction to scalable languages or rather to Scala. Well, Scala looks really promising with its traits and other features. But my original objection still remainds: The syntax is just ugly. Proves my point that there are only five people on this planet who can design a language and of them, I only know Guido van Rossum.

Adoption-Led Market

Simon Phipps followed wth a talk about the "Adoption-Led Market". It's astonishing how many OSS projects Sun supports or started and how little is known about that. Marketing again. *sigh* His talk wasn't all about Sun but how people start to despise vendors or rather their promise that their product is going to deliver to the promise ... after you've paid for it. Of course in an OSS world, where you can just download something and try it out, it becomes increasingly hard for them to "justify their 1000% profit margin."

This is a very good point. With OSS software, you pay for what you need (and not what the vendor things is best for him\b\b\byou). If you need a feature, support or whatnot, well, you can always pay someone to give it to you. But then, you pay when you need it. That makes is a justifiable cost, not an arbitrary one. Also, support gets you what you need when you need it. Just imagine to ask MicroSoft for a specific feature in Windows which you need. How much would that cost? What are you're chances to actually get it? Forget it! But with OSS, for the first time in software history, you can get what you need for a reasonable price and you might even make someone on the other side of the planet very happy.

Of course, "the greatest threat to freedom is a happy slave" but the talk gave a lot of arguments how to sell OSS to your company, even if that company happens to be the government.

He also gave a good reason which OSS license to chose for your next OSS project:

A license is the constitution of a community
Eben Moglen

Nice guy, good talk. If you can, vote for his photo at his website.


The third keynote was by Rod Johnson: "Where will tomorrows innovation in Java Enterprise come from?" After explaining where innovation comes from, he shows how standards kill innovation and how committees can't drive innocation due to political games by the companies who dispatch the people that form these committees plus general group stupidity.

After the more or less failure of the JCP in recent years, attributed mostly to ignoring feedback from the community and drowning ideas in ceremony, he hopes that JSR 316 (that's Java Enterprise version 6) will help to fix Java EE. If it does, then that might mean that a good sign that the JCP has become a helper for the Java cause instead of another reason to abandon it.

Monday, June 23, 2008

Jazoon 2008, Day 0

Just back from the Jazoon 2008, Tutorial day (the day before the actual conference starts). I attended Ted Neward's talk "The Busy Java Developer's Guide to Groovy". As always, the talk was quite a lot of fun even for someone like me, who already knows a lot about Groovy. If you get a chance to attend a talk given by Ted: Go. It's worth it.

In his talk, Ted crammed a pretty complete introduction in the key features of Groovy into three hours. For me, the key points of the talk were these: Productivity and ceremony.

Research shows that all developers roughly write the same amount of code lines per day. The difference between guru developers and the greenhorns is that the gurus write a line once and it works first try and each line achieves much more in respect to reaching the goal than any line of a novice. So in order to boost productivity of a guru, the only way is to use a programming language which can achieve more in a single line of code (and we're talking one statement per line here, obviously). So it's not that the guru writes ten times as much code per day as the novice, the novice just writes every line ten times because every time, (s)he gets something wrong and has to fix it.

The other aspect is ceremony. A lot of code in Java is ceremony. In most cases, the Java compiler knows where a statement ends and it wouldn't need the semicolon. Still it insists that I press the key anyway (and it's two keys on the German or Swiss keyboard). The same goes for many other things in Java which are just there for tradition. It would be simple for the Java compiler to infer the type of an object after if (obj instanceof ...). There really is no need for the cast in the next line. I could find another 10 examples easily.

Groovy does away with all that. It comes at a price, though. If you like to rely on the compiler creating a cozy padded cell for you, checking types and syntax as much as possible, then Groovy is not for you. In Groovy, the idea is that the developer knows exactly what (s)he's doing and all (s)he needs is the fastest path to get there. You can do all the dirty things you wished for in Java but you don't have to. Groovy is an "also" language, not an "either-or" language. If you feel that Java is dirty enough, already, try Scala.

Thanks, Ted.

If you want to know more about Groovy and you're in Zurich, why not attend my talk on Thursday?

Saturday, June 21, 2008

Collaborative Editing

Mustafa K. Isik has a video on vimeo demonstrating collaborative editing of a Java class in Eclipse. I'm not sure where this leads and my experience winces at the idea but it would sure help if the superman of your team could attach to your editing session, see the code you're working on and help you fix it (instead of having to get up and coming over).

Friday, June 20, 2008

prefuse - Information Visualization Toolkit

If you're looking for a cool way to visualize your data for the user in Java, try prefuse. The examples are impressive.

Thursday, June 19, 2008


After a long time, I'm starting a new OSS project called "ePen". From the description:

ePen - A cross platform tool for creative writers and book authors. ePen is able to manage all information around a book project: The actual text, descriptions of the characters and scenes, the storyline.

ePen should become for authors what a pen was for centuries: A simple tool to convert ideas into amazing, breathtaking stories.

Right now, ePen is in the planning stage even though I'll commit a prototype this weekend. The first thing on the agenda is to decide how to implement the tool. The prototype was done in Python and PyQt and while it was a nice experience, the text editor leaves a lot to be desired (read: it sucks). I've collected my thoughts in this wiki.

I've also create a Google Group for discussions.

Tuesday, June 17, 2008

How to Work Better

From the wall of a corporate building in Zurich:

  1. Do One Thing at a Time
  2. Know the Problem
  3. Learn to Listen
  4. Learn to Ask Questions
  5. Distinguish Sense from Nonsense
  6. Accept Change as Inevitable
  7. Admit Mistakes
  8. Say It Simple
  9. Be Calm
  10. Smile

Communication Skill

From a calendar:

The real art of conversation is not only to say the right thing at the right time, but also to leave unsaid the wrong thing at the tempting moment.
-- Author unknown

Monday, June 16, 2008

Eye Candy For Developers

There are two new projects on the 'net which might influence the way we see software in the future (in the literal sense). code_swarm can visualize the CVS history of a project in a cunning way and Wordle creates tightly packed tag clouds.

What I find especially interesting about Wordle is that it creates the tag cloud in real time (as you watch) using a Java applet. If you happen to have Java 6 installed, the animations are very smooth, almost like Flash.

Friday, June 13, 2008

The Lucifer Effect

I'm back from a 2 week holiday in Ischia (Italy). I like to take a book or two with me because I rarely find the leisure (and the concentration) to read something complex during my normal life. This time, it was The Lucifer Effect by Philip Zimbardo. If you haven't heard about this guy, you may have heard about the Stanford Prison Experiment (SPE). If you haven't heard about this as well, here is a short summary: Take 20 random, normal people (i.e. someone like you and me and your neighbor), put them into a situation where they can get away with anything and watch them turn into human nightmares within a few days.

The book explains in great detail how normal people turn evil. Evil in the sense that they stop acting kind and instead start to humiliate, torture, even kill other people. The hard part of the book is to understand that these people aren't evil to begin with. They are just ordinary people. For example, most suicide bombers are well educated, have an intact family background. Still, it's exactly these people who are most vulnerable to the mechanics which turn a nice guy into Darth Vader.

I'll point out a few places in the book in this post which impressed me most. On pages 167f, Jerry-5486 (one of the prisoners in the SPE) explained how he rationalized his own evil actions against a fellow prisoner by thinking "It's only a game, and I know it and I can endure it easy enough, and they can't bother my mind, so I'll go through the actions." Doing this, he could completely ignore how much his actions would hurt the people around him, even people he thought of as friends. Instead of supporting those he liked, he submitted to the cruel commands of the guards. Because of how our brains work in these circumstances, he only noticed what he had done afterwards.

On page 205, there is a great quote:

Life is the art of being well-deceived; and in order that the
deception may succeed it must be habitual and uninterrupted.
-- William Hazlitt, "On Pedantry", The Round Table, 1817

as is on page 208:

Wherever anyone is against his will, that is to him a prison
-- Epictetus, Discourses, second century A.D.

On page 210ff, Zimbardo explains the different factors which led to the humiliation/torture in the SPE: Situation, role expectations, responsibility (or lack thereof), anonymity and deindividuation, cognitive dissonance and social approval. On page 220, he explains how cognitive dissonance forces us to rationalize our actions when we hurt others:

Oddly enough, the dissonance effect becomes greater as the justification for such behavior decreases, for instance, when a repugnant action is carried out for little money, without threat, and with only minimal sufficient justification or inadequate rationale provided for the action. Dissonance mounts, and the attempts to reduce it are greatest, when the person has a sense of free will or when she or he does not notice or fully appreciate the situational pressures urging enactment of the discrepant action.

Instead of using our brain to stop what we do, we use it to explain the pain away. The less reason someone has to be cruel, the stronger the desire to give the cruel actions a reason that will make it "look good". Ever wondered why anyone will come up with such unreal excuses when they are caught? That's why.

In out daily life, we feel that our reason is in control when in fact, it's the situation that shapes how we see the world and thus forces our hand without our reason even noticing. This is what happens when the government calls the population to arms because of a "threat to national security". Ironically, it was always those who yelled that were the threat (and not those who were accused). Unfortunately, situational forces, our shared desire for social approval, drives us to believe the yellers even when five minutes of rational thought would show how silly their words and how false their motives must be (p. 226ff). This is what lead to slaughter of the Jews in World War II, the Challenger disaster, the genocide in Rwanda and the abuses in Abu Ghraib (which was one of the more harmless incidents in US prisons in the area). It's always the same mechanisms which lead to these results.

The basics of these mechanisms are listed on p. 230: Our need "for consistency and rationality", our need "to know and to understand our environment and our relationship to it" and "our need for stimulation". Also, how we sense time has a great influence on our behavior (p. 243): Do we believe in the past? The future? Do we have reason to plan or do we sit back in depression because we feel all our plans and hopes are in vain?

These same mechanisms are used by the military to train normal people (which still have an inhibition to kill) into people who can kill on command. A visitor to the SPE website explains how the drill instructors used the usual old, irrational lie to punish the group for a mistake: "if you guys would have moved faster, we wouldn't be doing this for hours". Well, the only reason why the group is going "this" is because of the drill instructor (p. 256).

Tests show that group pressure is a great tool to create obedience. Imagine you had to judge the length of a line on a card against three reference cards. Doing the test alone, you'd get it right more than 99% of the time. But what if you were in a group with several people who would chose the wrong answer on purpose? Could you resist the urge to comply to the group or would you submit? When the experiment was done, 70% of 123 test persons caved and agreed with the group that two obviously different lines had the same length (p. 263). Modern technologies allow to look into the brain and help to understand what happens. If you deviate from the group's consensus, the images from your brain will show that "resistance creates an emotional burden for those who maintain their independence -- autonomy comes at a psychic cost" (p. 265). The images also show that often, you don't even notice the conflict. The brain will adjust it's perception of the world to what the group wants to believe before reason is even asked for its opinion.

That all leads to Milgram's experiment where unsuspecting test persons would be lead by the experimenter to apply deadly electrical shocks to another person. Of course, the other person was a confederate and the shocks were never actually applied but the test person didn't know that. 66% of the test persons would go all the way even when the "victim" would scream in pain and beg for mercy. Most would even continue after the "victim" stopped responding. Because the results were so shocking, the experiment was repeated with different people in different countries. With different settings, Milgram could get the compliance rate (people who would go through the whole hell until the bitter end) from 10% to 90%. It didn't matter who was tested but how. From the results of his various experiments, Milgram compiled a list of ten methods to turn anyone into a torturer (p. 273f):

  1. Create a sense of obligation (for example, make a contract or say "you have to protect your country/family/whatever").
  2. Give the participant a positive role (teacher, protector, soldier)
  3. Present a set of rules that seem to make sense at first glance. It doesn't matter if they are really consistent since you can later change them arbitrarily.
  4. Make the job sound harmless. Don't say "hurt the prisoner", say "protect the country".
  5. Make sure no one really feels responsible for what happens or that everyone can point at someone else to blame when something bad happens. In Milgram's Experiment, a lot of people stopped unless the experimenter told them "I'll accept full responsibility if the other person is harmed".
  6. Start with an act that points in the general direction but doesn't seem too bad ("We'll just lock these people up to protect society").
  7. Work with small, seemingly insignificant steps ("that's not much different from what I did yesterday"). Humans are great for detecting big changes but they suck with little ones.
  8. Gradually change the nature of the authority figure (leader, boss) from "just and reasonable" to "unjust, demanding, even irrational". If you do this in little steps, the subject won't notice and still believe in the "first impression" even after the transformation is complete. At that stage, the leader can ask anything and (s)he will get it.
  9. Make exiting hard. People are allowed to complain but insist that they still follow the orders to the letter ("Do you want to ruin everything we achieved?")
  10. Offer an ideology, an explanation or a big lie to justify the use of any means to achieve the seemingly desirable ends.

If you get everything right, 90% of all people will follow you to hell and beyond. If you take these rules, you can easily find parallels to the many dictatorships around the planet. That's how and why they work and why it is so hard to break them down from within.

There are also references to anonymity experiments with schoolchildren during Halloween (p. 301) which shows what happens if people can be aggressive anonymously. In this scenario, they will act more aggressive unless they can be identified, even if their aggressive behavior is not in their best interest.

Being part of a group apparently causes us to reduce our self-awareness (a process called deindividuation, p. 305) and our ratio (our "good" sides) in favor of our "Dionysian nature", where instincts, lust and desire rule. This is what creates the strong group cohesion or "peer pressure" which allows us to create a civilization despite our egos. But if you don't know about these effects, then they can get out of hand, driving the group ever deeper towards chaos or evil.

These effects are enhanced when we see other people as inferior, non-human, animals or objects. In the end, it leads to a state of moral disengagement (p. 310). In this state, the people of the group are still able to understand that their actions are wrong but they don't care anymore. This state allows people to kill, torture, rape.

We don't even have to go that far. People are less likely to help when they are in a hurry (p. 316). The same effect is at work when a large group of bystanders watch someone suffer and don't intervene to help. Or when a sane person is taken to a mental hospital (p. 321). Staff members of a hospital played the role of patients for a short period of time and were very surprised how they were treated by their colleagues on duty. In the end, lawyers had to demand their release or they would still be in the ward.

Or Abu Ghraib. In chapter 14, Zimbardo dissects the situation and the people behind the media hype we all know from the images in the news shows. An apparently normal, mental healthy soldier is thrown into a situation where a life doesn't count much, there is no support from outside or superiors, the place is under constant attack by the locals and local forces that are supposed to help instead smuggle weapons into the cells of the prisoners for money and your superiors order you to torture to get intelligence from the prisoners. And the icing of the cake is that you get the night shift which means sleep deprivation. How long could you withstand such a stress? Staff Sergeant Chip Frederick (that's the guy who got all the blame for the incident) made it for more than two months until he caved (p. 343).

From the official news, you might get the idea that he was accused to have tortured the prisoners. I find that hard to believe. Others have killed prisoners and are still walking free (see "The Iceman"). From my point of view, he was convicted for making trophy photos. Other teams with similar records were instructed to destroy the evidence before returning home. The kind reader is welcome to draw his/her own conclusions.

This pattern of taking trophy photos comes up again and again when people are abused. It happened when black men and women were lynched in the US, German soldiers took them after committing atrocities against Polish Jews and Russians, etc. The list is endless. But there is a difference between these photos and real trophy photos (taken when a big-game hunter had his shot). On the original photos, the hunters don't laugh and cheer. They appear "rather grim, rarely are they smiling: they are victors in a battle against formidable adversaries." (p. 364).

The author goes on to explain how the Bush government actively helped to produce the environment which lead to abuses like in Abu Ghraib and many other places in Afghanistan and the Iraq, Guantanamo, and probably elsewhere, too. The last figure I know is of 600 similar incidents. An interesting if controversial read.

Chapter 16 contains the most important part of the book: How to fight evil. Not on a big scale but on a personal one. How to notice and withstand forces which try corrupt you. On page 451ff, you can find a list of ten steps to "resist unwanted influences". I especially like "Reject simple solutions as quick fixes for complex personal or social problems." Print that out and read it twice every morning.

Even if you don't read the rest of the book, look at these few pages. They alone are worth the buy.

In the last part of the book, Zimbardo takes a look at heroism. Unlike popular belief, heroes and heroines are ordinary people who just did the right thing at the right time at great personal risk because they believed they had to do it (p. 457). Or as the author puts it (p. 466):

Heroism can be defined as having four key features: (a) it must be engaged in voluntarily; (b) it must invoke a risk or potential sacrifice, such as the threat of death, an immediate threat to the physical integrity, a long-term treat to health, or the potential for serious degradation of one's quality of life; (c) it must be conducted in service to one or more other people or the community as a whole; and (d) it must be without secondary, extrinsic gain anticipated at the time of the act.

Recommendation: Must Have.

Thursday, June 12, 2008

The Craft of Text Editing

While looking for a decent code base for a text editor on Google Code (tip: search for "label:editor"), I found a wiki page in the epytor project which links to "The Craft of Text Editing", by Craig A. Finseth. That document not only explains all the basic algorithms behind the scenes of a text editor but also compares different implementations and contains a wealth of useful general comments.

Does It Have To Be a Scrollbar?

Every once in a while, someone comes up with a nice idea and everyone adopts it. People needed a way to navigate in a document larger than the screen, so scrollbars were invented to give an idea where they are related to the whole document.

Scrollbars offer a consistent interface, they are well understood and they don't change size while you use them. That's good. But they also fail give you an idea where to look when you search something. Like: Where did I modify my document? What's above and below?

farmhand shows this information. Instead of a gray/blue blob that moves, you get a zoomed view of the document along with change marks. It takes a bit more space on the screen and I'm not totally convinced how it fares with really large documents but it's definitely a new idea with some potential.

I'd just put these "farmhand bars" as pop ups on the editor tabs, so they don't clutter the screen when hidden and are still easily accessible when you switch editors.

Wednesday, June 11, 2008

Rich Applications in 2008

For about five years, I'm trying to write a small OSS application to help me write SciFi stories. Today again, I ran into this familiar pain: There is just no good OSS framework to write applications. Here is what I tried:

PyQt 4.4. This is nice but you need to do a lot of manual plumbing to connect your Python model to the Qt UI. Why are there no support classes to display a Python list in a QListView? Or to map fields from objects in a list to a QTableView? You can't even simply connect a field of an object to a basic text widget.

There are other Python frameworks (wxWidget, for example) but they are even worse (less features, more complex to use, no data binding as well).

So let's look at Java. What do we have? Spring Rich Client is more or less dead (they say they are working on Spring Desktop but there isn't even a web site, yet). Also, this project is locked to Swing. Not a killer feature in 2008.

So let's look at Eclipse. We have SWT, Data Binding and ... well ... EMF. Some love EMF, some like it, others ... you get the idea. Also, this solution again asks for a lot of plumbing (manually connecting objects with the UI). And while I gave Eclipse 3.4 RC4 a try (to see if EMF got usable in the last three years), it crashed in USER32.dll. *sigh*

Then, there is TurboGears. Automatic data binding, automatic generation of a nice UI, cool. Web only ... *sigh* Maybe I should try it with Firefox and XUL.

Also, none of these frameworks offer a simple WYSIWYG editor which can be extended (for example with a typing help, tooltips, proportional font support).

After ten years of OSS application frameworks, you'd expect that there was at least a single choice which allows to build a toy application with ten simple objects within a few minutes.