As @FrustratedWithFormsDesigner noted, backwards compatibility was the single biggest constraint. And the resolution they chose was to keep backward compatibility by using type erasure, i.e. removing all generic type information during compilation. This means a
List<String> and a
List<Integer> will become the same type at runtime (and the same as a pre-Java5
List). And it has a lot of negative consequences, including most of what you listed above.
The constraints set by this are much tighter than one may naively think:
What we require is that the same client code works with both the legacy and generic versions of a library. This means that the supplier and clients of a library can make completely independent choices about when to move from legacy to generic code. This is a much stronger requirement than backward compatibility; it is called migration compatibility or platform compatibility.
Java implements generics via erasure, which ensures that legacy and generic versions usually generate identical class files, save for some auxiliary information about types. It is possible to replace a legacy class file by a generic class file without changing, or even recompiling, any client code; this is called binary compatibility.
(Java Generics and Collections, Chapter 5)
Binary compatibility is not always automatically guaranteed; section 8.4 of the same book details some cases where binary compatibility may break during genericizing legacy interfaces even if all general rules have been followed.
Another aspect of backward compatibility is adherence to a type system where not everything is an object. So in Java primitive types are out of scope for generics just as for collections, and you have boxing/unboxing to complicate life (whereas in Scala everything is an object).
A third legacy in Java is native arrays which just don't fit with generics:
- arrays are always reifiable types whereas generic types are not;
- arrays can store primitive types whereas (generic) collections can not;
- array subtyping is covariant, whereas generic subtyping is invariant;
- as a consequence, arrays are less type safe than generics, and one can argue that they should be declared obsolete in favour of generic collections and not used in new code.
All this means a lot more rules to adhere to, thus corner cases and compromises to make, limiting the power and simplicity of Java generics.
Several such decisions can be seen in retrospect as wrong, but there is no way back anymore, thus no possibility to fix them without breaking backward compatibility, which - rightly or not - has become a sort of holy cow in Java over the years. I conjecture that the story of Java generics most probably brought a lot of experience and food for thought for its own designers as well, and these experiences were surely reused during the design of Scala.