Google announced the release of Go 1.5, the sixth major stable release of the company’s programming language, which brings much improved garbage collection along with several other enhancements.
Russ Cox, a principal engineer at Google and one of the two tech leads on the Google Go project, said there are four major new features or enhancements in Go 1.5. First, the compilers and the language runtime have been converted from C to Go. Second, there is a new low-latency, concurrent garbage collector. Third, it’s now possible for C or C++ programs to call Go libraries. And fourth is that there are new ports of Go to Linux on 64-bit PowerPC, and Linux and Darwin on 64-bit ARM.
These last two things enable a project that is separate from Go 1.5 that Google calls the Go Mobile Experiment. That experiment is intended to explore the use of Go in mobile application development, Cox said.
Go is Google’s attempt to make programmers more productive, he said. When you think about a programming language system, there are stacks, Cox explained. At the bottom, you have the language definition and above that you have the standard library APIs. Those two pieces are the foundation for everything else. Above those, you have the compiler and the library implementations and then there is the tooling to write, run and manage programs. That whole set serves as the foundation for additional libraries and projects that are written by end users—people outside Google.
When Google launched Go in November 2009, it launched some sketches of the bottom of the stack, such as the language definition and standard library and a prototype of the compilers and libraries. Their focus since then has been on refining the libraries and completing the stack. Go 1, which launched in March 2012, was the first step in finishing the stack. After Go 1.2, Cox said the team felt good about the core language and they started moving up the stack into the implementation of the runtime and the compilers. Go 1.3 and 1.4 built on that. Now, Go 1.5 is a major “painting in” of the key parts of the runtime and the compilers.
“The biggest change for us for Go 1.5 was that the compilers and the language runtime were converted from C to Go,” Cox told eWEEK. “Originally these were both implemented in C because Go didn’t exist and we needed something. But it made sense for a number of reasons to convert them to Go. For the runtime, the main reason was that writing a low-level language runtime depends on the tight integration with the compiler. So if your runtime is written in C that means you have to teach the C compiler about Go concepts like garbage collection pointer maps and write barriers. But the Go compiler already knows all of this anyway so it makes sense to convert the runtime to Go and not need the C compiler at all. So eliminating C code was very important to the concurrent garbage collection work because it reduced the amount of work we had to do in the compilers by half.”
Google Go 1.5 Revs Garbage Collection, More
That conversion from C to Go was the largest change for the Go team, but it enabled what is probably the most important change in Go 1.5, which is the introduction of the concurrent garbage collector. The simplest garbage collector you can imagine is called stop-the-world and it completely stops program execution, which is the “world,” so nothing in memory is moving anymore and it looks for what in memory is not needed anymore and it frees that garbage for reallocation and then it lets the world start again.
The period of time that the “world” is stopped is called a GC pause. The problem here is that the more memory you have, the more time it takes to look thru it and the longer the GC pause is. A GC pause of a second or two is a long pause, and the standard solution is to make the garbage collector concurrent, which means it can do what it does while the rest of the program is running.
Over a year ago, Google hired a garbage collection expert named Rick Hudson who had invented a number of significant garbage collection techniques. He started working on a concurrent collector designed for Go specifically and that collector is what the company released in Go 1.5. That new collector solves the latency problem—where at the worst case, Google was seeing one to two second pauses on really complex programs, now they are seeing 10 to 20 millisecond pauses on those same programs. And more normal-sized programs see pauses in the one millisecond range, Cox said. “So that’s a 100x improvement, which is not something we see every day,” he said.
“Related improvements to the scheduler allowed us to change the default GOMAXPROCS value (the number of concurrently executing goroutines) from 1 to the number of available CPUs,” said Andrew Gerrand, a Google engineer working on Go, in a blog post on Go 1.5. “Changes to the linker enable distributing Go packages as shared libraries to link into Go programs, and building Go packages into archives or shared libraries that may be linked into or loaded by C programs (design doc).”
Meanwhile, the third big new feature, being able to call Go libraries from C or C++ programs, is important when you have existing C or C++ programs and you want to add a piece written in Go or rewrite a piece written in Go. Before Go 1.5, it was possible for Go to call C libraries, but it wasn’t possible for C to call Go libraries. Now it is. This same work made it possible to compile a Go program as a collection of shared libraries, which is important to some Linux distributions, Cox said.
“There are also several new operating system and architecture ports,” Gerrand said. “The more mature new ports are darwin/arm, darwin/arm64 (Apple’s iPhone and iPad devices), and linux/arm64. There is also experimental support for ppc64 and ppc64le (IBM 64-bit PowerPC, big and little endian). The new darwin/arm64 port and external linking features fuel the Go mobile project, an experiment to see how Go might be used for building apps on Android and iOS devices. (The Go mobile work itself is not part of this release.)”
Thus, the fourth new feature is important for some server environments, and also for support for the iPhone and the iPad, Cox said. The goal of the mobile experiment is to explore the use of Go as a Native Development Kit (NDK) language on mobile devices like Android and iOS. Today, the primary NDK language on those devices is C++. “We designed and built Go initially as an alternative to C++ for backends on servers, but modern mobile apps tend to be structured the same way, so we think Go could be useful there, too,” Cox said.
Google Go 1.5 Revs Garbage Collection, More
“We were writing C++ programs before Go,” Cox explained in an interview. “And so we really wanted an alternative to that for building new systems languages. So being able to call Go from C programs gives us a way to replace some pieces of C++ instead of whole programs. But we’ve replaced whole programs as well. C++ is our background and so it’s sort of naturally what we want to do—we want to use Go instead of that. But we’ve found that many users want to use Go instead of what they were using. We’ve seen a number of users come from Python and Ruby, which was a bit of a surprise for us. But we’ve really seen people coming from all over.”
At Google, Go has primarily been used for a lot server software—a lot of the programs that do a lot of networking and coordinating network machines. One example is the download server that serves Chrome downloads, Android downloads and Google Play Store downloads. That’s a program that was rewritten from C++ to Go. Another Google system written in Go is the Chrome mobile proxy. Outside Google, there have been other systems. Docker, the container system for managing clusters of Linux machines in the cloud, is the one of the most popular. There’s another similar system built at CoreOS called rkt or “Rocket.” Google has released a variant of the software they use to run containers in their own cloud called Kubernetes—those are all written in Go.
“We have seen some serious traction in adoption of Go outside of Google,” said Al Hilwa, an analyst with market research firm IDC. “Many major cloud system software projects like Docker and Cloud Foundry are using Go today. It is good to see that all of Go is now done in Go. With built-in garbage collection, which is improved in the latest release, it offers a distinct advantage in productivity over C/C++.”
Cox said for the Go 1.5 project there were more than 100 external open-source contributors. “We’re starting to see more source code contributions from people at companies like IBM, Canonical, Intel and Oracle,” he said. “That shows a lot of progress on the open-source front.”
In addition, he said overall adoption of Go has gone faster than expected, although the language is relatively young. It’s also pretty much complete language-wise. “We think the language itself is pretty close to done,” Cox said. “We made real changes to the language in 1.1 and 1.2, but since then 1.3 had no language changes and 1.4 and 1.5 had very tiny ones. The standard library is on its way to done, but not done yet. And the standard tooling and the pieces further up the stack have plenty of work to do. The implementations will continue to improve in important ways like the garbage collector has in 1.5.”
In his blog post, Gerrand supports Cox’s stance on the language: “The only language change is very minor, the lifting of a restriction in the map literal syntax to make them more succinct and consistent with slice literals,” he said.