Performance Review

By Darryl K. Taft  |  Posted 2007-04-30 Print this article Print

What about performance? Whats the performance like? Hugunin: We look at performance relative to the current implementations of these languages. The idea is people are able to write code in these languages very productively today. I would consider performance successful if we did nothing but matched the level of the existing languages, plus provided all of this infrastructure and integration that you have. Now I wouldnt be thrilled and jumping up and down if all that we did was matched it, but at a basic level what we need to do succeed with performance is show that you can run these languages all on a shared engine without giving up any performance; because you get huge benefits from this shared engine—starting with the security model, then a lot less work in the implementation and the integration with all these other libraries. Now for IronPython, weve always been shooting for at least two times faster than the native C-based implementation. And we still seem to be on track for that. Our Pystone numbers have gone up since our last benchmark. They went up from one and a half times faster to two times faster.
Our JavaScript performance in our first release is not particularly impressive. And thats because this is a new language on a new platform, and so its a little bit slower than the native implementation on a lot of things. Its faster on some things, slower on others – because we havent had the time to do any performance tuning on it yet. Weve gotten it up and running and getting it to run large swaths of existing JavaScript code was our first point.
And then we all know we can solve the performance issues because it works great in IronPython, so well get that level of performance. Have you worked with any outsiders on this? Lam: The first time that were going to be working with outsiders on this will be an event we have coming up in May, which is the Compiler Dev Lab [May 22]. That will be the first time that well be inviting folks from the outside community to come in and look at it. Inviting the folks from the outside is going to be the first thing to really try and validate a lot of the ideas. Were going to be shipping source code for this stuff at MIX as well, so that before people get to the Dev Lab theyre going to have a chance to look at this code up on CodePlex for the DLR. Then they can come in and start asking some questions. Hugunin: Were shipping both IronPython and the DLR layer, which is a layer on top of the CLR. And were shipping both of those on CodePlex under the Microsoft Permissive License, which is the BSD-style [Berkeley Software Distribution] Microsoft license. And were doing that partially to invite people to play with it, give us a lot of feedback and go do interesting things with it. So its very much in the source-available, do-with-it-what-you-want-to spirit. Have you kept up with what Suns doing with JRuby? How does that compare to what you guys are doing? Lam: I like reading Charlies [Charles Nutter, a lead developer of the JRuby project] blog. Those guys have two things. Their compliance efforts have all been centered around an interpreter theyve been building. Charlies also got a compiler up his sleeve as well that isnt as conformant, but certainly runs things faster than their interpreter does. And Charlies been really good at trying to spearhead a lot of effort in the community to try and get folks together to write a standard and a spec and that kind of stuff. As well as creating a foundation. Hugunin: I have to keep pushing back on that because the DLRs sort of my baby. The DLR piece is the piece I havent seen anyone else doing—the idea of really building a shared dynamic type system so that all the languages can talk to each other. People have done shared hosting APIs before. Thats actually very common. Java, I believe, shipped a shared hosting API in their last release, and thats great. In fact, when I was building Jython, I wish theyd built that 10 years ago. It would have been a nice thing to have. But all that that does is it lets static code host individual dynamic languages one at a time. Without the shared type system those dynamic languages cant talk to each other. And the hosts all have to be written 100 percent in Java because thats the only thing each language understands. I cant write part of my host in Jython and then have JRuby be able to use it. So that shared type system piece, which has been a heck of a lot of work just to get half right, and the other half will come as we get more third-party folks beating on it. I think thats a really unique part about what were doing. Can you talk a bit about the effort that went into building the DLR? What were some of the issues you had to overcome? Hugunin: One of the challenges was just how to do Get-Member. If you think about it this is really easy. Every one of these languages has member assets. You write object.x and it means get the thing called x on the object. Thats really straightforward so that should be easy. We thought this would be really trivial. And IronPython supports it. And IronPython integrates beautifully with the static languages on top of it.  Thats an interesting notion of a type system. Most people build static type systems. In most of the type systems in .Net that have been built for the static languages, the attribute lookup is driven by the programming language that you wrote the lookup in, not the object that youre doing the lookup on. So that was one of the ways we did kind of a major change. So we said thats it, were done. Then we started working with the VB team. And Visual Basic, of the four languages that were working on is the one that has case-insensitive lookup. So that kind of threw a wrench for the works, because wed been working with these case sensitive languages and get-member is really obvious. So for VB we had to add a tweak to allow get-member to know if it needed case sensitive or case insensitive member lookup. And we were good, because when we looked at Ruby, Ruby didnt say well you also need this other thing. So one of the challenges on the DLR is we need to keep adding features as languages need them. So we had to add this case insensitivity thing for VB. However, if every time we add a language we have to add a new feature we fail. Because thats just building a lot of separate implementations. If we find that we need to add a set of features to get the core set working and then the others just fall on top, thats the mark of success. Rubys been one of the most pleasurable ones in one way because we beat on a lot of things in Python and JavaScript to get them to work together. VB was the next one we tackled. It had some issues but nearly as much JavaScript did because it was the third one. Ruby is having its own fair share of issues…but were finding this lovely trailing tail of the remaining issues of each language. When did this concept come together? Were you working on this back at Lang.Net in August? Hugunin: Yes, we were just starting to explore how to do this at Lang.Net. And we were even telling people that we were. I deliberately showed Python, VB and PowerShell and we talked about them talking to one another. And at that point we were working on putting these pieces together, but at that point there wasnt a DLR. It was just a concept of we need to build something like this. But since then its been really coming together. A lot of it was right after we shipped IronPython 1.0. That was when all the developers that were working on IronPython came and started working on the DLR. So when it really became a serious development effort was after we shipped IronPython. That was always our plan all along – to do it for real with one real language and then turn it into a framework. Im very skeptical of frameworks that are built without a real implementation first. Check out eWEEK.coms for the latest news, reviews and analysis in programming environments and developer tools.

Darryl K. Taft covers the development tools and developer-related issues beat from his office in Baltimore. He has more than 10 years of experience in the business and is always looking for the next scoop. Taft is a member of the Association for Computing Machinery (ACM) and was named 'one of the most active middleware reporters in the world' by The Middleware Co. He also has his own card in the 'Who's Who in Enterprise Java' deck.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel