To me, a big advantage of Scala's "enrichment" over monkey-patching in Ruby or JS is that it isn't global. That is, you have to import the enrichment. Another code module in the program won't be unexpectedly affected by it.

In practice, I almost never use monkey-patching in dynamic languages because it's too dangerous. While in Scala there are cases where enrichment won't work, you can always just write a regular function in those cases, and there are lots of cases where enrichment _does_ work...

Adding and/or altering functionality at runtime isn't dangerous. Monkeypatching may be, so avoid that.

Also, an entirely too-little used idiom (blame Rails programmers):

    module OverrideSomeMethod
      def some_method
        …
      end
    end

    s = SomeClass.new
    s.extend OverrideSomeMethod
    s.some_method

This is an idiom I use regularly (in Perl, Ruby, Io & Javascript) and come across it often in the Perl world where Moose roles are used.

The only downside of this idiom is the extra runtime cost which maybe an issue for Rails?

Some languages make monkeypatching a lot less dangerous.

For eg. in Perl you can use dynamic scoping to localise its effect:

  {
    no warnings 'redefine';
    local *SomeModule::some_func = sub { say "MONKEYPATCHED!" };
    
    # now everything in this scope that uses or calls SomeModule->some_func 
    # will now use the monkeypatched version
  }

  # where has everything else outside this scope remains unaffected

In Ruby Refinements earmarked for ruby 2.0 will have something similar: http://www.rubyinside.com/ruby-refinements-an-overview-of-a-...

C#:

    public static class EnumerableExtensions
    {
        public static IEnumerable<R> FilterMap<T, R>(this IEnumerable<T> list, Func<T, Option<R>> callback)
        {
            ...
        }
    }

There are some methods that need to be part of the class and carried around with the instance so that things like polymorphism work. But many operations work perfectly fine without. By making those lexically scoped, you avoid the problems of monkey-patching and method collisions. Extension methods are fantastic for this.

Also, for kicks, Magpie:

    def (items) filterMap(callback)
        var result = []
        for item in items do
            match callback(item)
                case true, mapped then items add(mapped)
                else nothing
            end
        end
        result
    end

    var result = [1, 2, 3, 4, 5, 6] filterMap with
        if it % 2 == 0 then (true, it * 2)
    end

    print(result) // 4, 8, 12

Magpie is dynamically-typed, but methods are lexically-scoped (and are multimethods).

you have the same problem in C#. you can't take a Foo class and extend it from outside to have all the IEnumerable methods then let it have all the IEnumerable extension methods.

I'm sorry, but won't these two examples add the method just to arrays? The author is intentionally trying to add a single method that will work for all collections, both scala's and non-scala's, and will always return a collection of the same type as the one the method was called on. Can you show how to easily do that in a dynamic language?

In Gosu, it's pretty much the same (though enhancements are statically dispatched and thus subject to a different set of limitations):

enhancement MyEnhancement<T> : T[] { function filterMap() { . . . } }

Roughly the same in C# with extension methods, though with C# you explicitly import the extension method while in Gosu they're automatically always there (sort of good, sort of bad). Again, not exactly the same as the dynamic language examples, but enhancements/extension methods do allow you to extend existing classes in a reasonable fashion while still being amenable to all the other advantages static typing gives you.

Both you and jashkenas are missing the point. The examples you are giving are also pretty easy to do in Scala.

The more complex method he is proposing (filterMap) cannot be done at all in the languages you mention, though he only uses it precisely to get the most complex kind of method that Scala collections offer. But it is also possible, and I just blogged about it here: http://dcsobral.blogspot.com/2012/01/adding-methods-to-scala....

But, no, that is not what he wants. He wants to add this method not to the collections, but to something that isn't a collection. Well, Scala can do that too -- it added all the collections methods to String and Array, didn't it?

And here comes the twist: he wants to add filterMap not by adding it directly to them, like Scala does. He wants, instead, to go _through_ that code to get at them.

With extension methods, the login would be like this:

* X adds extension methods to Y * Z adds extension methods to X * Therefore, Z extension methods should be available on Y

And, in fact, it is even possible to do that in Scala for many methods, but not for the particular combination he chose, and while still inferring all types.

Instead of getting bogged down in his specific example (the tree), I think it's more helpful to focus on his larger points in "On Acknowledging Problems" (the forest).

I disagree. The problem here is that those claiming “it is easy in language A” haven't even understood the problem.

People should first actually understand the problem, only after that a discussion about solutions makes sense.

That's not the problem, it's a problem. That is, it's a problem in this discussion, but I feel it has been dealt with well. The author's main point, though, was not about the specific example. That was to illustrate his larger point, which was about the complexity that arises when rich features interact.

The issue the author mentioned has already been solved in a much more easy and efficient way, without trying to use every feature of the type system.

Now the question remains: Should a language make almost-impossible and dangerous tasks easy or hard? I certainly prefer a language like Scala, which makes easy things easy, hard things possible and dangerous things hard, instead of the other way around.

That is a nice feature of dynamic languages, however you lose strong typing. I think the thing to take away is Scala hasn't gotten the perfect blend of these two yet. You can't make that competly generic map filtering extension yet. However you can make a less portable alternative. So you make you decision on what is more important.

> however you lose strong typing

You lose static typing; strong typing is different: http://en.wikipedia.org/wiki/Strong_typing

Just to be pedantic, you lose static typing. Ruby (and Python, etc.) are strongly, dynamically typed languages.

In F# this is approximated with:

  type 'a IEnumerable with
    member this.filterMap f = 
       [for x in this do if f x <> None then yield (f x).Value]

  let j =  Map([("a",1) ; ("b",2) ]).filterMap(fun kv -> if kv.Value = 1 then Some(kv.Key,kv.Value) else None) |> Map.ofList

  let n = [1.;2.;4.].filterMap(fun x -> if x % 2. = 0. then Some(x**2.) else None) 

The extension works with any Sequence (arrays, sequences, maps,etc.) with the caveat that it maps every enumerable to a list. You could also define an extension for a specific type where a broad definition does not make sense.

Personally, I lean more functional, I have never run across a need for extending things in this manner.

Arrays are not sequences in Scala. They are Java arrays, not Scala sequences.

Yes, I picked this up, and was not making any statement against that. In fact I do not subscribe to the validity of his approach. But I gave a shot at seeing if I could give code that was succinct and matched.

The requirement was to add a method that works for all collections, whether platform or language specific, while preserving type. The code I gave is an approximation of a solution - to use a rough analogy: topologically speaking the code matches but loses the geometry. The code I gave works on basically all .NET collections, whether C# or F#, string or tree - as long as they implement the interface - they are matched. That it leverages the existing organization should not count against it. The failing is that although types are preserved it is under a new geometry or structure.

Common Lisp:

(defmethod filter-map (f (a array)) ...)

That was not the question. It is trivial to do that in Scala for the use case mentioned.

The problem is adding it as an "instance" method to a collection and expecting that it is usable by something not being a collection at all.

Well that is the beauty of it. In Scala methods/functions that come after the dot are privileged. In a language with multiple dispatch they are not.

So CL solves the problem without adding more complexity, whereas in Scala you have to extort yourself to shoehorn some functionality after the dot.

CL isn't even statically typed. Everything is easy if you don't expect that the language gives you any useful guarantees.

I have no idea what you mean by "adding it (which "it"?) as an "instance" method to a collection". You can do this:

(defmethod filter-map (f (c (eql some-particular-collection))) ...)

but I suspect that's not what you meant.

I also can't make heads or tails out of "expecting that it (which "it"?) is usable by something not being a collection at all." I'm not even sure that's proper English, let alone semantically meaningful.

Maybe you should try to understand the actual issue first, _before_ claiming that "but it is easy in my pet language" stuff ...

Additionally, last time I looked Cl wasn't really statically typed. Has that changed recently?

Maybe if you stated what you think the actual issue is in clear, unambiguous terms instead of being pissy and snarky about it this discussion will not degenerate into chaos.

No, CL is not statically typed. And your point would be...?

The issue is with static typing. One can define a filterMap function in Scala much as you defined it in CL. The author's goal is to do that while also always statically knowing the most precise type of the returned collection. So, it's an issue that only comes up in a statically typed language.

Of course, I don't program in Scala, so my explanation may not be accurate.

> The issue is with static typing. One can define a filterMap function in Scala much as you defined it in CL. The author's goal is to do that while also always statically knowing the most precise type of the returned collection.

That's right.

> So, it's an issue that only comes up in a statically typed language.

No, that's wrong. You can do type inference in non-statically typed languages. Lisp compilers do this all the time.

If a compiler infers types at compile-time for a dynamically typed language, I still consider that "static typing" because it's statically inferring the types. If the term "static typing" is the problem, then I can rephrase: it only comes up when you try to determine all types before executing the program.

That's right.

At this point I would like to remind both you and soc88 of a parable:

Patient: Doctor, it hurts when I do this.

Doctor: Well, don't do that.

(Soc88's response, in the context of this parable, is something along the lines of, "But anyone who doesn't do this is a moron.")

Inferring types at compile time is necessarily hard. It is a corollary of the halting problem that no static type inference can be perfect. Therefore you have the following choices:

1. A simple compiler that sometimes fails to identify type errors at compile time

2. A simple compiler that sometimes produces false positives (i.e. signals a type error in a program that is in fact correct)

3. A complicated compiler. (Note that even a complicated compiler will also do 1 or 2 or both, but potentially less often than a simple compiler.)

Those are your only options. Reasonable people can disagree over which is preferable.

I don't disagree with your points, but now that we've established that, that is why your original example does not solve the problem as presented in the post. The problem inherently has to do with statically inferring types.

That depends on what you consider to be "solving the problem." Do you want to "do this" or do you want to be free from pain? You can have one or the other, but not both.

OF COURSE static type inference is hard. That's a straightforward consequence of the halting problem. Pointing to defmethod is just an oblique way of making the point that perfect type inference is NOT NECESSARY for getting things done. You can choose to lament the complexity of Scala (and static type inferencing in general) or you can use Lisp or Python and trade certain compile-time guarantees for simplicity. Like I said, reasonable people can disagree over which is preferable.

What reasonable people cannot do is insist that there is a single perfect solution that is both simple and error-free. Anyone who believes that has not understood the implications of the halting problem.

Another thing reasonable people cannot do is frame the tradeoff as a binary choice: either you use static type inferencing, or you give up all compile-time guarantees. That is simply not true, as is amply demonstrated by e.g. the SBCL compiler. It's a complex, multi-dimensional space of tradeoffs in language design, compiler complexity, and different kinds of compile-time guarantees. It's INHERENTLY complicated. The best you can hope to do is find a reasonable point in the design space for your particular quality metric. For the OP, Scala isn't it.

Since the OP uses Scala to solve his problems, I have a feeling that Scala is, to him, a reasonable place in the design space. Scala allows a function much like your example. He used that example not say "This is a failing of Scala, and why I will not use it," but to say "This example demonstrates a complexity that is a natural consequence of the design of Scala." In other words, he said something quite similar to what you said.

> Scala is, to him, a reasonable place in the design space

Reasonable perhaps, but manifestly not ideal or he would not be complaining about how complex it is.

> he said something quite similar to what you said

Well, I didn't actually say much, I just posted a snippet of code and left people to draw their own conclusions. Why soc88 chose to start a fight I can only guess, but it seems to be not uncommon behavior among people trying to defend untenable positions.

What you said two posts up about inherent complexities, not at the beginning.

> A simple compiler that sometimes requires a type annotation.

It is easy to show that that will not solve the problem. If your language is Turing-complete, then you can embed (say) a Lisp interpreter and arbitrary Lisp code within it. The only way your compiler can be complete and correct for your language is for it to be complete and correct for this embedded Lisp. This is a fundamental result. There is no way around it.

Hey, you're both [edit: mistake, see below] new here and obviously knowledgable about the topic at hand. At HN, we try to maintain civility - it's an explicit goal of the community. What this implies is that if you're in a discussion with someone, and you realize they don't understand an important point of the discussion, instead of using sarcasm, it's much better to say "Oh, I see, you're missing point x."

> you're both new here

My account was created 1458 days ago. Just how long does someone have to be here before you no longer consider them "new"?

Sorry, I said "both" by mistake. You are an active and well-known contributor to HN, and I know you're reasonable, which is part of why I felt soc88 was being unreasonable.

> Author complains about the inability of the compiler to prove some property of his code.

No, the author is complaining about the complexity of the language. Maybe you should go back and re-read the article. Start with the title.

> Untyped languages

Lisp is not untyped. "Not statically typed" is not synonymous with "untyped."

> Therefore, you are completely missing the point.

Which of us is missing the point remains to be seen.

I think it's a good post as well, but filterMap is a contrived example. It comes out of the box: flatMap can be used as a filterMap, because Option[T] is an acceptable substitute (through implicit convernsions) for Iterable[T]. (An option is a collection type; just one with 0 or 1 elements.)

The existing collections functions in Scala take you very, very far. If you need to write your own primitives for performance reasons, it can get tricky, but that's really uncommon. Odersky's book (chapter 25) explains how to do that.

Collections libraries are hard in Scala because of what you get. Once you define a few simple functions and possibly implicit conversions, you get all 50+ sequence methods "for free", and if you do it right, your map type functions will return collections of the same type (runtime and static) as the original-- without explicit typing. You get a lot of leverage, but you have to work a little bit for it.