Wednesday, July 15, 2009

Groovy Does Not Have Optional Static Typing

Every now and then somebody writes that Groovy supports optional static typing. But it doesn't. They've confused the use of type annotations on variables with a static type system.

This article is emphatically not an entry in the long, sad, and mostly insipid static vs. dynamic debate. It is a clarification of a misunderstanding.

Definitions

The word "static" in "static type system" is related to phrases like "static analysis." Basically, without executing the code in question, a static type system analyzes code to prove that certain kinds of misbehaviors won't ever happen.

The word "dynamic" in "dynamic type system" is related to the "dynamic execution" of a program. In contrast to a static type system, a "dynamic type system" automatically tests properties when expressions are being (dynamically) evaluated.[1]

The Proof

With those definitions its easy to show that adding type annotations to a Groovy program does not make it statically typed. Create a file named test.groovy. In it define a couple of unrelated classes

class Foo {}
class Bar {}

Now write a function that promises to take a Bar and return a Foo. Add a println for fun.

Foo test(Bar x) { return x }
println("Everything's groovy")

Compile it and run it (explicitly compiling isn't necessary, it just reinforces my point)

~/test$ groovyc test.groovy
~/test$ groovy test
Everything's groovy

There were no complaints at all. A static type system would have rejected the program.

Open up test.groovy and make a slight adjustment so that the function is actually called

Foo test(Bar x) {return x}
println("Everything's groovy")
test(new Bar())

Compile and run this

~/test$ groovyc test.groovy
~/test$ groovy test
Everything's groovy
Caught: org.codehaus.groovy.runtime.typehandling.GroovyCastException: 
Cannot cast object 'Bar@861f24' with class 'Bar' to class 'Foo'
 at test.test(test.groovy:4)
 at test.run(test.groovy:6)
 at test.main(test.groovy)

A runtime exception is finally thrown at the (dynamic) moment when the test function is executed. That's dynamic checking rather than any form of static proof.

Actual Static Typing

Now open a file named test.scala.

class Foo
class Bar
def test(x : Bar) : Foo = x

Compile as a script

~/test$ scala -Xscript test test.scala
(virtual file):1: error: type mismatch;
 found   : this.Bar
 required: this.Foo
object test {
^
one error found

Ta da, static typing. With no attempt to execute my test function the type checker says "I have reason to believe this might go sideways."

The Misunderstanding

There's an oft repeated phrase that "in a statically typed language variables are typed." There's also a common misunderstanding that static typing and type annotations are the same thing. I can show a simple counter example to both misconceptions with one language: Haskell.

Create a file named Test.hs. Here's a function called flatten. It flattens a list of lists one "level" to just be a list.[2]

flatten = foldl (++) []

No variables were harmed during the creation of that function. It's written in what's called "point free" style. Yet with neither variables nor type annotations I can show it's statically typed by defining a bogus main function

main = print $ flatten [1,2,3,4,5,6]

And trying to compile

~/test$ ghc Test.hs -o test
Test.hs:3:24:
    No instance for (Num [a])
      arising from the literal `1' at Test.hs:3:24
    Possible fix: add an instance declaration for (Num [a])
    In the expression: 1
    In the first argument of `flatten', namely `[1, 2, 3, 4, ....]'
    In the second argument of `($)', namely
        `flatten [1, 2, 3, 4, ....]'

Thus you need neither variables nor annotations to have static typing. A small change fixes things right up

main = print $ flatten [[1,2,3],[4,5,6]]

~/test$ ghc Test.hs -o test
~/test$ ./test
[1,2,3,4,5,6]

It's easy enough to see what type GHC has assigned the function. Just add a module declaration at the very top of the file

module Main (flatten, main) where

And fire up ghci

~/test$ ghci
Loading package base ... linking ... done.
Prelude> :load Test
Ok, modules loaded: Main.
Prelude Main> :type flatten
flatten :: [[a]] -> [a]

Exactly as described, it takes a list of lists and returns a list.

Groovy Does Have Static Typing

Having shown that Groovy does not have static typing let me show that it does have static typing. :-) At least, it has static typing for some kinds of things. Open up test.groovy again and add this

class MyRunnable implements Runnable {}

Compile that and you get an error

~/test$ groovyc test.groovy
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed, test.groovy: 8: 
Can't have an abstract method in a non-abstract class. 
The class 'MyRunnable' must be declared abstract or the method 'void run()' must be implemented.
 @ line 8, column 1.
   class MyRunnable implements Runnable {}
   ^

1 error

The error happens without executing anything so it's a form of static analysis, and it's caused by failure to adhere to the Runnable type's requirements hence it's a static type error. But it's not an optional static type check - you can't turn it off.

Interesting, no?

Conclusion

Whatever you feel positively or negatively about Groovy please don't perpetuate the myth that it has optional static types. It doesn't. Groovy's optional type annotations are used for dynamic testing not static proofs. It's just that with type annotations Groovy dynamically tests are for "dynamic type" tags attached to objects rather than its more normal mechanism of dynamically testing for the presence of certain methods. What little static typing Groovy does have is not optional.

Update: Some time after this article was written, a team announced Groovy++, an extension to Groovy that really does have optional static typing.

Footnotes

[1] In a certain formal sense the phrase "dynamic type system" is an oxymoron and "static type system" is redundant. However, the phrases are commonly used and there doesn't seem to be a commonly recognized concise phrase to describe what a "dynamic type system" does. Pedants will argue for "untyped" but that doesn't give me a concept on which to hang the different ways that say Groovy, Ruby, and Scheme automatically deal with runtime properties.

[2] A more general function called join already exist for all monads in Control.Monad module. It can be defined as "join m = m >>= id"

10 comments:

Guillaume Laforge said...

That's why I usually call that "optional typing". The 'static' word is again misleading. Grovy does support "optional typing" and in some cases also some "static typing" for certain aspects you summarized.

Tony Morris said...

Guillaume, The word 'static' is only misleading in the sense that James alludes to in that it misleads users of Groovy. The word has a very clear and distinct meaning and Groovy is very clearly and distinctly not "optionally statically typed".

Groovy supports optional type annotations -- this is a very different proposition as James points out.

paulk_asert said...

Hi James, thanks for the post. It is always good to strive for clarity on these topics.

Of course, in Groovy I can write an AST macro (compile-time meta-programming) which turns on additional static checking, i.e. makes your first example fail at compile time or which turns off, i.e. makes optional, the interface type checking, i.e. makes your second example compile happily but fail at runtime if the interface method is missing.

Of course the power with having such features in Groovy is of most benefit once conventions and IDE support is in place so that end users can make use of it. At the moment, probably only gurus could write such AST macros and there would be little IDE support to indicate if they were in place for a particular piece of code.

Currently IDE support (at least in IntelliJ which I mostly use) is sufficiently good that it infers most types when they aren't specified. So most users are getting feedback at 'type time' rather than 'compile time' or 'runtime'. It would certainly be good down the track if more optional static typing in Groovy was available when coders desired it and with appropriate IDE support.

paulk_asert said...

Also "Groovy's optional type annotations are used for dynamic testing not static proofs" isn't how I would phrase things either for maximum clarity.

I can certainly reason about Groovy code statically. I just have to know about what metaprogramming facilities are in place rather than just reason based on static type definitions alone. If you say in the general case that might be very hard to get automated tool support for this, then I would agree. The aforementioned IntelliJ support is definitely fundamentally flawed. It probably only gives the right answer 99.99% of the time. ;-)

And of course, for the hard cases, e.g. does my program exhibit certain liveness criteria, then I might reason about it using infinite set-based logic or whatever favorite formal method you prefer but I would hardly expect either a static or dynamic language to give me any feedback at compile time.

James Iry said...

paulk,

When you reason about a code base that's not a static type system in the language, it's a static type system in your head. The plus side is you can use whatever logic you want where a language type system is (usually) limited to one. On the downside an automated type checker is far less likely to make silly mistakes.

As for IDEA, I don't have it installed so I can't comment. At best you're arguing that there is a language which might be called IDEAGroovy which is syntactically identical to the Groovy language but which has a richer (optional?) static type system. The 99.999% comment relates to the differing semantics between the two languages.

paulk_asert said...

I mostly agree with your sentiment but again would use different wording. If I am reasoning about some Java or Scala code and don't have a compiler handy, then I am also reasoning in my head. If I have my compiler or my IDE then I get whatever support it gives me. I would expect a statically typed language to give me better feedback from the compiler.

I don't think talking about some hypothetical IDEAGroovy language helps clarify the situation. It is certainly fine to talk about the different support provided by different tools though. If you have code like the following in Scala (excuse the Groovy flavored syntax) it will have (ignoring implicit conversions which I don't know enough about to know whether they could apply here) a compile error on the last line. In the IntelliJ Groovy plugin the line doesn't get marked as an error but instead is marked as a warning in yellow which if I hover over it says "Cannot assign 'java.lang.Integer' to 'java.util.Date'". Depending on
whether I have metaprogramming in place will determine whether I get a runtime error or not for Groovy.

def x = [[0]] as Integer[][]
def y = x[0]
Date z = y[0]

So on the one hand, I am only get a warning rather than an error which is all that the is possible in a non-statically typed language but certainly many of the "silly mistakes" are sorted out straight away.

It is certainly true though that there is a fuzzy area here. In the above example, some very simple type inference is being done by IntelliJ. Depending on my IDE or code style checking tool it may have less capable "inference for warnings" features. It might have smarter inference capabilities which looks at what kind of metaprogramming might be in place and takes the inferencing further. It may have capabilities to let me teach my IDE when it gets the inferencing wrong or when it can't infer the types.

So I agree that Groovy compilers, IDEs and tools can't give as much
feedback as similar Java and Scala ones at compile time but you might be surprised at just how much feedback you get.

I should add that I find the Tool support for all the new languages
simultaneously impressive in how far they have come but also significantly behind those for the more mature Java and C# languages.

Jochen "blackdrag" Theodorou said...

James, may I ask who used the wording that Groovy has a optional static types? Guillaume and me never use this and we are more or less the creators of the language. I tend to say that Groovy is a dynamic typed language with static elements. I also always say optional typing (but I am already careful with that). A new term appearing might be gradual typing, but I am not yet familiar enough with that to know if it is really right.

As of an language having only one type system.. That is true and wrong at the same time. Normally every static typed language has also dynamic type checks. In Java the check done for a cast would be an example, possible missing methods I see as another dynamic check. A purely static language would not need runtime types, but Java does type checks each time a method is called for example. So if you come and say that Java is a static typed language I tend to say that this is not fully right, it is a mostly static typed language. In that respect you could say that Groovy is not a fully dynamic language, since it has static elements. But the focus is important. And the focus is being static typed for Java and being dynamic typed for Groovy. dynamic and static typing don't exclude each other, instead they fill gapes in each others system.

James Iry said...

Groovy In Action, Manning, 2007, Glover, King, Laforge, and Skeet

p61, 3.2.1 "Table 3.3 gives examples of optional static type declarations and the dynamic type used at runtime."

p62, 3.2.2 "The choice between static and dynamic typing is one of the key benefits of Groovy."

And see http://www.google.com/search?q=groovy+"optional+static+typing"

Regardless of where the phrase comes from, it's certainly out there and causes confusion.

The phrase "optional typing" is pretty awful too. It suggests that with annotations you get some typing and without annotations you get none, but that's not true. Either way you get a dynamic check of runtime properties. It's just a question of what properties get checked and when.

Still, I can totally understand that it's hard to name the feature in a way that's both concise and meaningful. This article isn't a bash on Groovy nor a bash on dynamic typing (see the follow up http://james-iry.blogspot.com/2009/07/fixing-groovy.html). It's just an attempt to clear up confusion.

Chris W said...

I suggest reading up on Groovy++, an extension to Groovy. That variant is indeed statically typed with the compilation safety and runtime performance that implies. The syntax is so familiar that I'd think of it as the true Java++. See an intro here:

http://groovy.dzone.com/articles/sneak-peak-groovy-what-it-why

James Iry said...

Yeah, Groovy++ was announced after I wrote this article. Maybe it deserves an update.