e.g.

    map(expensive_call, filter(cond, seq))

    [expensive_call(each) for each in seq if cond(each)]

    filter(cond, map(expensive_call, seq))

    [each for each in [expensive_call(x) for x in seq] if cond(each)]

    [expensive_call(each) for each in seq if cond(expensive_call(each))]

That said, it's not really "compelling" though -- there is no real "compelling" reason to switch from one Turing-complete language to another given that you can do the same thing eventually. But hey, it's the itches that drive us nuts, isn't it? :)

Just my 2 cents

    (filter (map expensive_call X) cond)

    [y for y in [expensive_call(x) for x in X] if cond(y)]

Incidentally, this isn't even an issue in a pure functional language with sufficiently smart compiler.

Plus, you get 5 mentions (3 y's and 2 x's) of some intermediate variables instead of 0 in your code, so both token-wise and char-wise, map/filter alternative is shorter, and less a mental burden (think about the "succinct" idea by PG).

If you can put together a nontrivial usage of map+filter with at least three source collections that's more concise than the equivalent list comprehension I'll (figuratively) eat my hat.

    map(lambda (w,s,l): {'widget': w, 'sprocket': s, 'location': l}
        filter(lambda (w,s,l): l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s), 
            product(widgets, sprockets, locations)))

The argument in favor of set-notation being higher level is it's less specific (it doesn't explicitly provide a sequence of operations, just an outcome).

List comprehensions look like set notation but have an implicit procedural translation you have to keep in mind to use them well, so it's a toss-up.

I prefer map/filter/reduce when sequencing has large performance implications but for simple filtering or raw-data-shaping comprehensions read more smoothly.

http://books.google.com/books?id=RvY5BM0Xt1wC&lpg=PT367&...

Now you know why many people like me are gradually pissed off by Python and start the exile ... currently trying Scala and it seems a nice language (with list comprehension too! :)

> List comprehensions look like set notation but have an implicit procedural translation you have to keep in mind to use them well, so it's a toss-up.

Actually I think that's the problem I have with list comprehensions: I use them a lot in my code, usually 1~3 levels nested and then have a hard time tracking down the order of implicit loops (which is inner vs outer) and make sure the intermediate variables (x for x in y for y in z ...) do not clash ... OK maybe I'm using it too much and in the wrong way :(

> I prefer map/filter/reduce when sequencing has large performance implications but for simple filtering or raw-data-shaping comprehensions read more smoothly.

I didn't know map/filter is faster than list comprehensions? I thought both are optimized by Python interpreter. But I like the idea of knowing that at least map can be parallelized easily. But since Python does not utilize multicore in a decent way, all bets are off :(

  do { w <- widgets;
       s <- sprockets;
       l <- locations;
       guard (l `hasInStock` w);
       guard (l `hasInStock` s);
       guard (w `isUsableWith` s);
       return (w, s, l); }

If eg I edited it to be:

    do { l <- locations;
         w <- widgets;
         guard (l `hasInStock` w);
         s <- sprockets;
         guard (l `hasInStock` s);
         guard (w `isUsableWith` s);
         return (w,s,l); }

List comprehensions: the set contains this.

Map/filter: apply these functions to the set.

Procedural: follow this recipe for turning one list into another.

    [fn(x) for x in xs for xs in xss if cond(xs)]

  filtered = [x for x in seq if x>10]

For simple cases, yes. But I wouldn't say

    [each for each in [expensive_call(x) for x in seq] if cond(each)]

    filter(cond, map(expensive_call, seq))

Now the problem is, some people see Python as a very functional language (with first-class functions etc) and want to use it that way (like Lisp), but BDFL and some core Python devs believe it is better to keep it Pythonic, thus those functional people are kinda pissed off by this and switch away from Python.

Personally I don't think it will make Python a lot cleaner to remove two auxiliary functions and force people to use list comprehension when it is completely trivial to add these missing pair back (two lines of code).

(disclosure: I prefer FP, but I also think keeping things Pythonic is fine most of the time. It's just that in this case, I think map/filter is pretty "Pythonic" according to me. :)

Later, I ran into some cases where I wanted a multi-line lambda. And my thought was "aaragh134!#?!"

Then, a weird thing happened. I started making a conscious effort to follow PEP 8. 79 column limit? Seriously? That sucks. But after weeks of struggling with it, something finally hit me. I realized I was writing better code by forcing myself to reduce code density. Sure, it was a little bit longer, but I spend way more time reading it than writing it.

Stop trying to fight it; assign a name to that lambda. Readability counts.

Stop trying to be clever; assign a name to that inner expression. Flat is better than nested.

De-emphasizing functional operations makes it more likely for libraries to work in a non-functional style, for tutorials to do so, etc.

It's tiring fighting against a community and a (benevolent) dictator that disagree with you.

That summarizes the problem I guess :D

Actually only one line of code is enough for each of map/filter:

    def map(fn, seq): return [fn(each) for each in seq]
    def filter(cond: seq): return [each for each in seq if cond(each)] 

    (filter #(> x 10) seq)

I would like to point out, however, that CL allows you to write:

    (loop for x in seq
          when (> x 10)
          collect x)

    (loop for i in *random*
          counting (evenp i) into evens
          counting (oddp i) into odds
          summing i into total
          maximizing i into max
          minimizing i into min
          finally (return (list min max total evens odds)))

I switch between map/filter and loop depending on whether I'm working with predefined functions (e.g., (filter 'less-than-ten seq)), handling multiple sequences, doing side-effects, etc.

CL could really use something like Python's generators though. The loop macro is an ugly hack in comparison.

The question is whether "Pythonic," as the community defines it today, is optimal in all cases. As an analogue, there have been enough talks about things the Java community considers to be stylistically optimal that look really horrible compared to implementations in other languages -- like Python. :-) Pythonic style should be a guide, not an edict, and should be deviated from or redefined when it makes sense. The examples riobard provided already show the syntax weighing things down, a situation where syntax should give way to a more functional style approach.

As for it creating a cleaner language, I try to approach this, as with all things, from the perspective of being an language-agnostic programmer. From that perspective, I do like the python syntax for simple things like what you defined, but under heavier weight mapping and filtering operations, the map/filter function call syntax seems a lot cleaner. There's nothing wrong with syntactic sugar, but I would assert that it will tend to suck when it is all you have.

You can arguably trace Python's list comprehension syntax all the way back to setl, a "set-theoretic programming language", and the resemblance to mathematical set-builder notation is intentional; compare

- let B = { g(a) s.t. | a \in A and f(a) holds}

- B = [g(a) for a in A if f(a)]

If you're used to thinking in sets having to decompose into "maps" and "filters" is a speedbump; easy to do but nice to avoid.

Where list comprehensions really start to shine is making it comparatively trivial to pull from multiple source collections without a lot of ugly machinery:

    [{'widget':w,'sprocket':s,'location':l} for w in widgets for s in sprockets for l in locations if l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s)]

    map(lambda i: {'widget':i[0], 'sprocket':i[1], 'location':i[2]}, filter(lambda i: i[2].hasInStock(i[0]) and i[2].hasInStock(i[1]) and i[0].isUsableWith(i[1]), itertools.product(widgets,sprockets,locations)))

It gets even worse if you try to be clever with the sequence of operations.

You might look at that definition and say lo! I can pre-filter out stuff not in stock at each location and make things more efficient. Naively you'd wind up with:

  map(lambda d: {'location':d[0], 'widget':d[1], 'sprocket':d[2]}, flatten(map(lambda l: list(filter(lambda p: p[1].isUsableWith(p[2]), itertools.product([l],filter(lambda w: l.hasInStock(w), widgets), filter(lambda s: l.hasInStock(s), sprockets)))), locations))) #nb must-supply-you-own-flatten-method

    [{'widget':w,'sprocket':s,'location':l} for l in locations for w in [widget for widget in widgets if l.hasInStock(widget)] for s in [sprocket for sprocket in sprockets if l.hasInStock(sprocket)] if w.isUsableWith(s)]

Level 0: for-loops with an explicit accumulator

Level 1: map + filter (!)

Level 2: ??? arguably an atemporal set-theoretic approach

In practice in python list comprehensions are a superior syntax for computing with multiple source collections.

(!) Really all you need is reduce

    map = lambda f,l: reduce(lambda h,t: h + f(t), l, [])
    filter = lambda f,l: reduce(lambda h,t: h + t if f(t) else h, l, [])

A saner map/filter/product version:

    map(lambda (w,s,l): {'widget': w, 'sprocket': s, 'location': l}
        filter(lambda (w,s,l): l.hasInStock(w) and l.hasInStock(s) and w.isUsableWith(s), 
            product(widgets, sprockets, locations)))

But I often think 'functionally.' Since I have options, I'd rather go with a language that allows me to program how I think - instead of being forced to translate my thought into its semantics.

It depends on what you define as power, and how wide of a continuum you are willing to presume that it runs.

Named functions add two levels of clutter.

The first is to the actual code, because you have to add a name to something that never wanted one, and it has to be defined apart from where it is used. You could give it a fluff name, but that is worse than no name at all. The reader is left wondering what the function is for, which requires carrying unnecessary mental baggage. It would seem more powerful to me that you just define a function where it is used, and not worry about having to carry around that extra mental information for later on; the less state you need to keep the better. But I may only say that because I, like you, tend to think functionally.

The second is to the namespace, because now there is a symbol in the environment that doesn't need to be there. It's not a huge deal, but it is just another bit of extraneous fluff.

Anyway, not meaning to turn this into a language war. Just a couple of thoughts that popped into my head when I read your first paragraph.

  result = []  
  for item in iterable:
      # here goes multiline lambda as an explicit suite
      # that computes value
      result.append(value)

I think it is a matter of language design. Higher order functions are not as prominent in Python as say, Haskell, where map & friends are generally preferred for composability. Lispers prefer map & friends just because list comprehensions add all that messy syntax.

If I were a Python programmer, I would probably use list comprehensions, since they seem to be the preferred idiom.

Also, I like map and filter because they aren't just functions in the procedural sense but combinators: you can pass them around, using them as arguments and returning them. It's much harder to pass around a syntactic entity like a list comprehension (although I'm glad list comprehensions exist; as shorthand they are great in source code.)

All this said, I haven't used Python in 3 years, in favor of purely functional languages such as Clojure and ML, so I might be way out of date.

I still prefer to user list comprehensions, and don't think it would be so bad to have map, filter, reduce live in the itertools module. This would stylistically match what is done with the comparison operator functions living in their own module, which you can import if you need to pass the functions around.

Common Lisp has something similar, called LOOP. Implemented as a macro, it's a within-Lisp DSL for expressing looping constructs, e.g.

(loop for i from 1 to 10 sum i) => 55

(loop for c across "bar" collect c) => (#\b #\a #\r)

It's controversial within the CL community, because the loop language looks much more like traditional languages than Lisp.

It's an influential language, for sure, but I prefer Clojure for a number of reasons.

That CL contains TAGBODY is a brilliant thing. It amazes me that a language can be at once so high-level and so low-level. I haven't had occasion to use it yet, but the fact that it's there, and that the higher-level abstractions are built in terms of it, is a thing of beauty to me. (I don't know about PROGV.)

I'd prefer to code in CL over Java/Blub, but I prefer Clojure over CL, and probably Haskell or ML over Clojure. (Lisp has better syntax, and macros are very cool, but static typing wins for large projects, in my opinion.)

That's what inner functions are for. This isn't the best example, but you get the idea:

  def foo(mylist):
    def n_to_the_n(n):
      if n > 1:
        return n**n
      return 1
    return [n_to_the_n(i) for i in mylist]

I'm used to Lisps, where def/defun/define/defn imply internment. For example, in Clojure:

  (defn foo [list]
    (defn n-to-nth [n] (Math/pow n n))
    (map n-to-nth list))

  >>> def foo(mylist):
  ...   def n_to_the_n(n):
  ...     if n > 1:
  ...       return n**n
  ...     return 1
  ...   return [n_to_the_n(i) for i in mylist]
  ... 
  >>> print foo([1,2,3,4,5])
  [1, 4, 27, 256, 3125]
  >>> foo
  <function foo at 0x6e5f0>
  >>> n_to_the_n
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  NameError: name 'n_to_the_n' is not defined
  >>>

With regard to 'define', it doesn't work like that in Scheme. A define in a function body behaves just like a let.