Effect Typing and Monads are yet another colour for the lipstick you put on the pig. You still have non-composability (i.e. the async monad spreads through your code), you still have the complexity, you still don't have proper tracebacks without special runtime support, and you still have the inefficiency that Bob didn't actually mention (unwinding all those stacks all the time).
Languages with threads (Java, C#) don't quite solve the problem as their threads have other disadvantages, e.g. creating too many threads causing contention, though it's much less of a problem than people make it to be.
Go actually completely removes the problem - you don't get a nice tool to pack the problem into boxes more easily, you don't get a different fancy colour for your lipstick, you just don't have the problem at all - you wouldn't know it existed was it not for those other languages.
There might be a deeper lesson here about good, pragmatic engineering vs awesome computer language geekery.
Go's "good, pragmatic engineering" solution to the problem - lightweight threads, channels, and the communicating sequential processes model - is built on the back of decades of academic research, including programming languages research.
And now for some computer language geekery:
Go solves one "code color" problem (concurrency/asynchrony), by hard-coding the solution into the language. Go programs are written in a particular monad (the "Go" monad, if you will), and so have one color. Indeed, monads originally arose in the study of the semantics of imperative languages; to be imperative is to be inside a particular monad. This is exactly the "async monad spreading through your code" problem taken to its logical conclusion: just put everything in the same monad!
This hard-coding solution has very concrete advantages. For one, you don't need to think about separate colors and how to transition between them, combine them, define them, etc. (IOW, monads are complex; by picking one, you reduce cognitive load.) For another, it's easier to optimize your implementation and tooling for that monad (lightweight threads, useful tracebacks).
But PL academics aren't satisfied with this solution, because it only solves one color problem, and it solves it non-composably: you can't put the solution in a library, you have to invent a whole new language for it. That's the holy grail (some) PL research is chasing. It's a question of concrete benefits in the here-and-now versus abstract benefits in the hopeful-future. But it pays to remember that today's concrete benefits are yesterday's green-field research.
Indeed, Go only solves one problem by baking it into the language/runtime, it doesn't have a generic mechanism to solve similar problems.
But my point is that it solves the problem much better than monads do - in particular if you don't just look at the syntax/user interface level, but at the non-functional aspects, like debuggability, performance, etc.
I remember a time when Aspect Oriented Programming was all the rage, with point cuts etc to capture cross-cutting aspects of your program. AOP is certainly less generic than monads, but even with rather specialized tools, it turns out the two "killer apps" of AOP were much better served by runtime support in the VM (logging/debugging) and explicit code (transactions).
I'm skeptical about the holy grail that you describe. I see how it's attractive, but in my work experience, the pragmatics of solving a particular issue often turn out to be harder (sometimes much) than the theoretical aspects.
There's certainly influence from more theoretical research in all we do in CS, but for example with Go, it's only based on Hoare's CSP model in the most abstract sense, and its implementation of lightweight threads, its stack model etc are definitely in the domain of clever engineering, not breakthrough research.
"AOP is certainly less generic than monads, but even with rather specialized tools, it turns out the two "killer apps" of AOP were much better served by runtime support in the VM (logging/debugging) and explicit code (transactions)."
Indeed... beware the tech promoted with the same example over and over again. That's a huge red flag.
> This hard-coding solution has very concrete advantages.
True, but you missed the major disadvantage of doing everything in a monad: there may be horrible side-effect demons lurking in every line of code!
In my experience, the most egregious offenders are shared mutable state (AKA multithreading) and exceptions. Introducing multithreading or exceptions into an application which previously shunned them feels like switching to a whole different language; one where all our intuitions and guarantees can be undermined at any time.
This assumes that the primary problem is merely asynchronous code. If that's the main problem you're facing then there's no reason in the world to believe that having more distinction at no gain is the right tradeoff.
I'm not claiming that you must program in a language which distinguishes lipstick colors. I'm merely claiming that you can and that it's a valuable tool.
To respond to your concrete criticisms:
The non-composability you refer to is actually what I call composability itself. You want the isolation of effectful worlds so that you have different things to choose to compose in the first place. You can do this "horizontally" with binding or "vertically" with free products or lifting (or any number of other techniques). Without this distinction things "compose automatically", but only because you've decided to mix all of your colors and go with brown.
Proper tracebacks and stack unwinding can be a problem, but fundamentally these are problems of how you compile/interpret languages like this instead of semantically whether these languages allow you to express what you would like to at high fidelity. Both of those problems can be addressed, though. The degree to which they're painful in practice is also a question.
The comment about threads is somewhat off topic. There's nothing in what I was saying which precludes the introduction of nice green threads and competitive scheduling. You can implement Go in Haskell quite neatly, for instance.
But yes, I agree. If your problem is pure multithreading then languages like Go and Erlang make it go away completely. They do so by collapsing layers of semantics down to a single one (completely antithetical to the red/blue distinction here) but choosing one such that multithreading is nice.
This can be an excellent spot in solution space for many problems. I would never suggest otherwise.
But as long as we're ostensibly on the topic of "blue/red"-ness of your language then it's high time to talk about effect typing.
> Effect Typing and Monads are yet another colour for the lipstick you put on the pig.
When I started to read the essay, I thought that it would be an anti-typing screed (it's nothing to do with the author, whom I don't know!), with blue functions being untyped (or, pace Harper, unityped) and red functions being typed. An untyped function can call a typed function by ignoring the types; a typed function cannot call an untyped one, because that might violate its contract. If you buy this analogy, then your first sentence might equally well have said "typing is yet another colour for the lipstick you put on the pig"—but lightweight HM inference shows that is not so. Maybe we're just awaiting the revolutions in effect typing that languages like ML and derivatives, and Haskell, brought to typing itself?
Actually, you can call both ways - from statically typed to dynamically typed and back. This is the premise of the work done in Racket on contracts, and more generally of the "gradual typing" movement. Harper's "unityped" term suggests exactly how this can work. An untyped function takes arguments that are all of one type, and returns one of the same type (that type being the "dynamic" type that includes every possible value, and thus is useless for static checking).
To call an untyped function from typed code, simply cast all your arguments to this type (trivial, since it contains anything), and check its return value has the type you expect (throwing an error if it's of the wrong type). To call a statically typed function from untyped code, simply check the types of all the arguments you wish to pass to it (throwing an error if they don't match its signature), and cast the returned value to the dynamic type (again, trivial).
I'm glossing over a very important difficulty, which is that you can't dynamically check something of a function type -- if I'm handed a function, how do I know at runtime whether it's of type `int -> int`? I can't! So the solution is to "wrap" this function with a contract that delays checking until the function is actually called, at which point you can check the argument is an `int` and the result is an `int`.
> To call an untyped function from typed code, simply cast all your arguments to this type (trivial, since it contains anything), and check its return value has the type you expect (throwing an error if it's of the wrong type).
To me, as soon as a function includes dynamic checks, it ceases to be statically typed; so I would say that a function that operates this way becomes only dynamically typed.
Do you consider any function which may raise an error to be "dynamically typed", then? (For example, any function which performs an array index; or any function which performs I/O; or, to be absurd, any function which allocates memory (since memory is a finite resource that may run out).) If so, I understand the distinction you're making, but it's a very strong position; most people do not mean that when they say "statically typed". If not, I don't understand the distinction you're making.
Admittedly, gradual typing does not mean that calling into untyped code from typed code is "safe" from runtime type errors (although it is safe in the technical preservation-and-progress sense). But it gives you a partial guarantee: if a type error occurs, you know that the fault does not lie in the typed code. This aids debugging, and helps you port over existing codebases gradually. So it's still quite useful.
> Do you consider any function which may raise an error to be "dynamically typed", then? (For example, any function which performs an array index; or any function which performs I/O; or, to be absurd, any function which allocates memory (since memory is a finite resource that may run out).)
That's a good question! Yes, in an idealised sense, I do; but, in practice, I think that I would make an exception for functions where the error comes from the host system, not from the program. (For example, if I were too strict about it, then I would have to disallow even the safest of Haskell functions because it might be executed on a computer whose hard drive is failing.) Thus, I would call Haskell's `head` function dynamically typed.
> Admittedly, gradual typing does not mean that calling into untyped code from typed code is "safe" from runtime type errors (although it is safe in the technical preservation-and-progress sense).
I think that I don't understand the definition. Isn't encountering a runtime error precisely getting 'stuck' (in the "opposite-of-progress" sense)?
> I think that I don't understand the definition. Isn't encountering a runtime error precisely getting 'stuck' (in the "opposite-of-progress" sense)?
Sorry, perhaps I should have said "exception" rather than "error". Raising an exception is absolutely "safe"/progress-obeying, as long as your language specifies the behavior of exceptions. That the exception happens to be called a "TypeError" doesn't change anything.
Getting "stuck" means being in a state such that there's no defined next state - a state which the language spec doesn't cover. So most "dynamically-typed" languages are in fact safe in progress-and-preservation terms, because they have well-defined behavior in all cases. C, otoh, is "statically-typed" but unsafe, because you can write C code with undefined behavior.
(That dynamically typed languages can be (statically) typesafe should be unsurprising, since they're just statically-unityped languages. Nothing says a unityped language can't be typesafe, it's just a very uninteresting property!)
Total agreement generally, but I wanted to mention that the Any -> X downcasting is trouble sometimes. It is difficult to do this in a type-safe way (e.g. without accidentally introducing holes) and if you have typecasing universally then you lose parametricity, one of the most powerful tools in any polymorphic type theory.
Go seems to have solved the IO problem, but what about the general asynchronous issue? How do you handle calling a function several thousand times on dozens of threads and combine the results?
How does Go solve "the IO problem"? (Probably what I want to ask is: which IO problem does Go solve? It can't be the one about purity and side-effects, and if it doesn't solve that...)
How do you do IO without stalling your whole program while it accesses the data, specifically the fact that the asynchronous call is terminated in your main loop, not where you are accessing the IO itself.
Languages with threads (Java, C#) don't quite solve the problem as their threads have other disadvantages, e.g. creating too many threads causing contention, though it's much less of a problem than people make it to be.
Go actually completely removes the problem - you don't get a nice tool to pack the problem into boxes more easily, you don't get a different fancy colour for your lipstick, you just don't have the problem at all - you wouldn't know it existed was it not for those other languages.
There might be a deeper lesson here about good, pragmatic engineering vs awesome computer language geekery.