In terms of language design, there isn't really anything that should make Go slower than Java in general. In fact, it gives you more control of the memory layout of your data structures, so for a lot of common tasks it should be somewhat faster. However, the current primary Go compiler, scheduler, garbage collector, regexp library, and a lot of other things aren't particularly optimized. This is steadily improving, but the focus seems to be on being useful, simple, and fast enough over winning in microbenchmarks.
In the linked benchmark, Go loses big to Java on the binary tree and the regexp test. Those are tests of the memory management system and regexp library respectively. Go's memory management could be faster and will certainly improve over time, and the current standard regexp library is a placeholder for a much better implementation that is soon to come. So, losing on those two isn't surprising, and in the near future the margin should be more narrow.
For the k-nucleotide benchmark, it's somewhat hard to compare because the Java code looks to be using a different algorithm. The Go code will certainly benefit from compiler, scheduler, and allocator improvements to come, even as written, but someone would have to rewrite the Go code to do something more clever if we wanted to compare more accurately.
Java wins in the mandelbrot benchmark because it's all floating point arithmetic and loops, and this is a great place for the JVM to generate really good machine code and hoist things at runtime. Go, in comparison, has a pretty simple compiler that doesn't hoist, unroll, or generate really tight machine code currently, so it's not surprising it loses. However, one should keep in mind that the Java timing doesn't count the JVM start-up time or the many times it needs to be run for the JVM to JIT it nicely. For long-running programs, this isn't relevant, but it matters in some cases.
As for the rest of the benchmarks, Java and Go are basically neck-in-neck, with Go taking significantly less memory and in most cases less code.
So, while Go is slower than Java in a number of those tests, Java is pretty fast, Go does pretty well in comparison, and Go will probably get notably faster in the near future.
I'm looking forward to when gccgo (a Go compiler that uses the gcc codegen) is mature; that should make Go pretty much in line with C for many types of code, which will be interesting.