eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Microsoft has released a new Dynamic Language Runtime to support various dynamic languages running on the .Net platform. eWEEK senior editor Darryl K. Taft spoke with Microsoft dynamic language gurus Jim Hugunin and John Lam about the effort.
What can you tell me about the Dynamic Language Runtime and where it came from?
Hugunin: Where this all came from is I joined Microsoft with IronPython, but I joined to make the CLR [Common Language Runtime] a better platform for dynamic languages. In some ways this is what weve been working on for the past two and a half years. The way that we did that is we started by building one real high-quality production implementation of a language, which was IronPython, and getting a lot of users to beat on it and make sure that we could do it well. Now that we have that one quality implementation we started saying, Well, how can we make more languages run well on the platform and make them run even better?
So the idea of the DLR is to support a broad spectrum of languages building on what weve learned already. taking all the things in the CLR… IronPython runs well in the CLR because its got great things inside it, like really excellent GC [garbage collection], JITs [just-in-time compilation], excellent type system that you can work within, debugging, profiling, sandbox security model – great features that make it easy to build a language on top of it.
When we built IronPython we found that there were specific things to a dynamic language that we had to re-implement. So we had to re-implement, for the dynamic parts of Python, we had to build a Python type system on top of it. For the parts of hosting a scripting language, we had to build a hosting API that was a Python hosting API. And for making the dynamic operations run really fast, we had to build custom optimizations to make Python dynamic operations run really fast.
In many ways, all the DLR does is it takes these things that we did custom for IronPython and it makes them work for a broad spectrum of languages. So instead of having a Python type system we now have a dynamic type system. So any language that has dynamic operations can share them… The point that we reached… its been fun watching this come together. It was a couple months ago in our internal work with the JavaScript team – theyre the ones whore releasing with us at the same time we are – theyre the furthest along. And we reached a point where we were doing a demo and somebody was complaining that event wiring hadnt been wired up yet for JavaScript. But their functions were working great.
So we were able to write a little program where we took a JavaScript function, passed it off to the Python code where event handling was working great and we wired it up as an event handler using the Python code. That to me was kind of the essence what we were trying to do.
By having this shared type system and these dynamic languages using the same sets of types, you can go back and forth between them. In the long run, things like wiring up event handlers, every language is going to have. But every language is always going to have the things that it does better. And the really cool code that somebody went out and wrote in just one language…if they could all talk to each other, you just get a lot of power as a language developer.
In what ways?
Hugunin: The hosting API is in some ways obvious. Microsoft did this a long time ago with its IActiveScript APIs. And that worked really well. Theres an interesting thing about the IActiveScript APIs, the biggest technical issue, I find with the IActiveScript API was actually the security model. That in order to use those APIs…I can run Python in the browser today with the IActiveScript APIs. The only way I can do it, though, is if I have somebody ship me a version of Python that they guarantee is completely secure. With the DLR plus Silverlight I can now run Python in the browser again on top of the Silverlight engine. And the Python implementation that were shipping is completely security transparent. What that means is all the security is in the Silverlight engine; its in the .Net engine that were running on top of. And the language implementer just has to do all their code generation. And if we generate, and weve done it, but if we generate unverifiable code—essentially code that would lead to things like buffer overruns and horrible security exploits – and when our engine does that we get a nice error message that pops up in the browser saying you generated unverifiable code and tried to run it. And that makes it possible for a lot more languages to start running in this kind of environment, because you can build on the .Net security model.
The fast dispatch is sort of obvious. Everybody said these languages couldnt run fast on top of .Net, and IronPython was showing that they can run fast. But it was a lot of work. It took us a lot of work to get it to run fast. And what we want it to do is both to get it to run even faster and to make it easy for other languages to run in this environment.
John, what has your role been?
Lam: My role since getting here has been bootstrapping our Ruby effort. When I got here nothing had been done yet. Then we started off work in earnest against our Ruby implementation. And its really only been in the last month or so that things have really come together to the state where weve gotten this thing in the form that we could actually demo it outside.
And the cool thing is building Ruby on top of the DLR was so much quicker and cleaner and easier than having to go off and reinventing everything on our own – inventing our own object model, our own dispatch mechanism and the rest of those things. So the experience from Ruby really is to kind of validate the idea of the DLR.
Now, thats not to say that Ruby went into the DLR with no change to the DLR at all. But whats useful about the exercise of implementing Ruby and VB and JavaScript Python on top of the DLR is to validate the idea of the DLR, with fully understanding that expectation that the DLR will have to change at some level to accommodate these languages. But that goes to prove that its not just a pipe dream, but that there are going to be four solid implementations, and four fast implementations of these languages that all run on top of the same infrastructure.
Hugunin: Johns been really pushing on making sure that thats in fact true. Hes been building a wonderful demo for Mix. And its not those four languages; I want to make sure we dont forget that fifth language. The fifth language is C#.
Lam: I think whats really interesting about Silverlight is the fact that all this stuff, if youre building something in a dynamic language is just all text. And considering the way the Web was built using this kind of collaborative community knowledge of how to build stuff in HTML, JavaScript and CSS, much the same experience is going to happen in Silverlight, except were just going to replace some of the acronyms. Theres going to be some HTML still, but theres going to be XAML thrown into the mix, as well as the source code for a variety of different programming languages – be it JS, VB, Ruby or Python. But also the flexibility of, in certain circumstances, being able to throw some compiled C# code down there for performance reasons.
We have a demo of a chess application. And chess is great because its a very compute-intensive kind of thing that really makes static type systems shine in terms of showing the benefits of making code run really fast. So thats a great example of being able to use this chess engine, written in C#, and using it from the client inside of the UI that you can absolutely build out of one of the dynamic languages.
Performance Review
What about performance? Whats the performance like?
Hugunin: We look at performance relative to the current implementations of these languages. The idea is people are able to write code in these languages very productively today. I would consider performance successful if we did nothing but matched the level of the existing languages, plus provided all of this infrastructure and integration that you have. Now I wouldnt be thrilled and jumping up and down if all that we did was matched it, but at a basic level what we need to do succeed with performance is show that you can run these languages all on a shared engine without giving up any performance; because you get huge benefits from this shared engine—starting with the security model, then a lot less work in the implementation and the integration with all these other libraries.
Now for IronPython, weve always been shooting for at least two times faster than the native C-based implementation. And we still seem to be on track for that. Our Pystone numbers have gone up since our last benchmark. They went up from one and a half times faster to two times faster.
Our JavaScript performance in our first release is not particularly impressive. And thats because this is a new language on a new platform, and so its a little bit slower than the native implementation on a lot of things. Its faster on some things, slower on others – because we havent had the time to do any performance tuning on it yet. Weve gotten it up and running and getting it to run large swaths of existing JavaScript code was our first point.
And then we all know we can solve the performance issues because it works great in IronPython, so well get that level of performance.
Have you worked with any outsiders on this?
Lam: The first time that were going to be working with outsiders on this will be an event we have coming up in May, which is the Compiler Dev Lab [May 22]. That will be the first time that well be inviting folks from the outside community to come in and look at it. Inviting the folks from the outside is going to be the first thing to really try and validate a lot of the ideas. Were going to be shipping source code for this stuff at MIX as well, so that before people get to the Dev Lab theyre going to have a chance to look at this code up on CodePlex for the DLR. Then they can come in and start asking some questions.
Hugunin: Were shipping both IronPython and the DLR layer, which is a layer on top of the CLR. And were shipping both of those on CodePlex under the Microsoft Permissive License, which is the BSD-style [Berkeley Software Distribution] Microsoft license. And were doing that partially to invite people to play with it, give us a lot of feedback and go do interesting things with it. So its very much in the source-available, do-with-it-what-you-want-to spirit.
Have you kept up with what Suns doing with JRuby? How does that compare to what you guys are doing?
Lam: I like reading Charlies [Charles Nutter, a lead developer of the JRuby project] blog. Those guys have two things. Their compliance efforts have all been centered around an interpreter theyve been building. Charlies also got a compiler up his sleeve as well that isnt as conformant, but certainly runs things faster than their interpreter does. And Charlies been really good at trying to spearhead a lot of effort in the community to try and get folks together to write a standard and a spec and that kind of stuff. As well as creating a foundation.
Hugunin: I have to keep pushing back on that because the DLRs sort of my baby. The DLR piece is the piece I havent seen anyone else doing—the idea of really building a shared dynamic type system so that all the languages can talk to each other. People have done shared hosting APIs before. Thats actually very common. Java, I believe, shipped a shared hosting API in their last release, and thats great. In fact, when I was building Jython, I wish theyd built that 10 years ago. It would have been a nice thing to have.
But all that that does is it lets static code host individual dynamic languages one at a time. Without the shared type system those dynamic languages cant talk to each other. And the hosts all have to be written 100 percent in Java because thats the only thing each language understands. I cant write part of my host in Jython and then have JRuby be able to use it.
So that shared type system piece, which has been a heck of a lot of work just to get half right, and the other half will come as we get more third-party folks beating on it. I think thats a really unique part about what were doing.
Can you talk a bit about the effort that went into building the DLR? What were some of the issues you had to overcome?
Hugunin: One of the challenges was just how to do Get-Member. If you think about it this is really easy. Every one of these languages has member assets. You write object.x and it means get the thing called x on the object. Thats really straightforward so that should be easy. We thought this would be really trivial. And IronPython supports it. And IronPython integrates beautifully with the static languages on top of it.
Thats an interesting notion of a type system. Most people build static type systems. In most of the type systems in .Net that have been built for the static languages, the attribute lookup is driven by the programming language that you wrote the lookup in, not the object that youre doing the lookup on. So that was one of the ways we did kind of a major change.
So we said thats it, were done. Then we started working with the VB team. And Visual Basic, of the four languages that were working on is the one that has case-insensitive lookup. So that kind of threw a wrench for the works, because wed been working with these case sensitive languages and get-member is really obvious. So for VB we had to add a tweak to allow get-member to know if it needed case sensitive or case insensitive member lookup.
And we were good, because when we looked at Ruby, Ruby didnt say well you also need this other thing.
So one of the challenges on the DLR is we need to keep adding features as languages need them. So we had to add this case insensitivity thing for VB.
However, if every time we add a language we have to add a new feature we fail. Because thats just building a lot of separate implementations. If we find that we need to add a set of features to get the core set working and then the others just fall on top, thats the mark of success.
Rubys been one of the most pleasurable ones in one way because we beat on a lot of things in Python and JavaScript to get them to work together. VB was the next one we tackled. It had some issues but nearly as much JavaScript did because it was the third one. Ruby is having its own fair share of issues…but were finding this lovely trailing tail of the remaining issues of each language.
When did this concept come together? Were you working on this back at Lang.Net in August?
Hugunin: Yes, we were just starting to explore how to do this at Lang.Net. And we were even telling people that we were. I deliberately showed Python, VB and PowerShell and we talked about them talking to one another. And at that point we were working on putting these pieces together, but at that point there wasnt a DLR.
It was just a concept of we need to build something like this. But since then its been really coming together. A lot of it was right after we shipped IronPython 1.0. That was when all the developers that were working on IronPython came and started working on the DLR. So when it really became a serious development effort was after we shipped IronPython. That was always our plan all along – to do it for real with one real language and then turn it into a framework. Im very skeptical of frameworks that are built without a real implementation first.
Check out eWEEK.coms for the latest news, reviews and analysis in programming environments and developer tools.