It seems that having many small methods trades one complexity for another. For methods that are only used once and sequentially, it seems to me that it's easier to follow if they are all inlined (by the programmer).
If the methods can be reused elsewhere, it's a different matter.
On the other hand, methods are modules, giving syntactic and compiler-supported semantic separation - and you can do things like return early.
I can't tell which is better.
Some people claim that short is better, but it always comes across as a bit rabidly dogmatic, because... well... it's without evidence. There is so much of that in comp sci: design patterns, functional programming, editor and language choice. Most people giving opinions don't even mention what type of task that advice is relevant for, nor give their experience that supports their choice. (It's easy to argue for a choice - smart people unfortunately can argue convincingly for anything.)
I tend to use separate methods only if they are reusable (otherwise it's a waste of time making them reusable). I often think of better ways of doing things, so I don't like to invest too much in what I have now. I'm mostly writing prototype code for new kinds of products, not "production code", not for clients, and no one sees it but me. Much code I've seen that it made of many methods and classes looks horribly over-engineered to me, especially when the problem itself is actually very simple if you approach it in the right way.
I wrote my thoughts up above, but will respond to this here. I agree, both about the short methods school and about programming dogmas in general.
I went through a couple years of working in the OO short method style. Recently an old client called me back to help modify some code I'd done for them in 2004, so I went down for an afternoon to help them out. I was really embarrassed. It was obvious to me that I had strung things together in endless chains of delegation (tiny classes and short methods) not because that was simple but because it, at the time, was my belief about programming. I got that belief from other programmers I admired.
The truth is that this is how most of us work all the time. I don't mean short methods; I mean picking a style based on our beliefs -- mostly for emotional reasons -- and then seeing the entire programming world through that filter. To be able to just see the problem is difficult when you're operating from one of these positions. Much (most?) of what we do in software development is add extrinsic complexity, which is bad when the problems themselves are hard to begin with.
My experience is that your moment of "the problem itself is actually very simple if you approach it in the right way" does eventually come, if you make getting there a high priority. But it's challenging. Most of the time we don't even know what our assumptions and beliefs are, let alone have the flexibility to adapt them to the problem. It's usually the other way around: we adapt the problem to our beliefs because they determine how we even see the problem in the first place.
I think one key thing to keep in mind is that it is better to chose a style that is more likely to obviously have no errors, rather than one that has no obvious errors. (Wirth?)
With longer methods, it becomes more strenuous to say that it obviously has no errors.
And in It seems that having many small methods trades one complexity for another. is not a fair representation, as the implied relationship is not linear.
With longer methods, it becomes more strenuous to say that it obviously has no errors.
I disagree. The proper comparison is not between one long function and one short one (that's a no-brainer), it's between a long function and a corresponding set of short functions plus all their interactions. Posing the comparison correctly makes the complexity tradeoff look very different. I'm not saying it's obvious, but the prima facie bias goes the other way.
There's a shortcut for answering this kind of question that may not be infallible but is very useful: program length. Things that make a program longer tend to increase its complexity. One should hesitate to argue that something which inflates code size is making a program simpler. But that is what the short-methods-OO school does routinely.
I don't see why one can't take overall program size as the basic measure of total complexity.
Edit: from another comment in this thread I gather that you tend to see interactions between functions as less complicated than code inside functions. Boy, do we look at this differently! If a function can do a single meaningful thing in isolation, of course I'd factor it out (that's almost another no-brainer). Those are what PG calls "utilities" in his Lisp books. They're meant for random access. But when functions start to interact with too many other functions in ways that affect application logic, my complexity Geiger counter goes crazy. I'd much rather have those interactions isolated in one place, where nobody else can get random access to them and introduce even more dependencies. As befits a truly different world-view, I'm puzzled as to how you can even hold yours.* It seems like a simple matter of combinatorics.
* Doesn't stop the discussion from being delightful though. Just to be clear.
I like how you qualify program length as a fallible measure (they're more guidelines). Similar to your story of overdoing tiny classes and short methods, due to a belief from programmers you admired, I once dogmatically followed this as a rule, always extracting code than was used more than once, and only doing so then. I think that's a standard learning stage, where you learn something, over-apply it, then learn distinctions for when it is appropriate and when not.
Optimizing for length is only one criteria - optimizing for clarity is more important (strange observation: in writing, redundancy enhances communication); optimizing for flexibility/change is another. I like the idea of just expressing your current understanding, very simply - not weighed with suspect prophecies. Change it as your understanding improves; as you reuse it. Brooks observed that having a spec and different implementations leads to a more robust spec; and there's an idea of not generalizing code until after you're implemented it three times, for different purposes. This is the opposite of architecture astronautics - being grounded in actual instances of concrete experience.
So, I give up a simple, single theory of how to code, and I'm lost - which is perhaps an accurate appraisal of our current understanding of programming. Only the actual details of a problem guide you.
From what you said elsewhere, I think the simple key is to keep focusing on the problem, not the program. My old supervisor said I was over-concerned with theory. "Look at the data!" he admonished me.
C.A.R. Hoare (according to http://www.gdargaud.net/Humor/QuotesProgramming.html anyway), but I prefer Douglas Adams' formulation, though less clever: their
fundamental design flaws are completely hidden by their superficial design flaws
Yes, it's true that the compiled-supported semantic modularity of methods helps here: e.g. it can't access other methods' local variables, you can see what goes in and what goes out. But, in a long method, you can manually enforce the same modularity on sequential parts (you can even use local variables scoped by {} to borrow some compiler support). But, yes, point taken.
Can you elaborate on the relationship not being linear? I think you mean that the many parts of a long method can interact (if the coder doesn't enforce this manually).
Elaborating: Less complexity in each method leads to less overall complexity. And yes, the worst of long methods is that the internal interaction usually leads to more complexity.
Perhaps poor choice of phrase. It is my feeling that moving complexity out of methods and possibly having it in the interrelationships between methods results in a net gain in understandability. Perhaps asymmetric would have been a better word?
It codifies the relationships between the parts - but also adds ceremony (the call, the method definition, the arguments). Whether this is a net gain depends. Probably the key is whether it feels like a conceptually distinct component - a theory or abstraction that you can build on. This is probably true even if it isn't reused (but typically coincides with it).
There was some "non-linearity": the interaction between parts increases (roughly) with the square of the number of parts: 4 parts have 16 directed interactions. Combinatorial explosion is a more accurate measure. If you can separate the parts into modules, that interact with only the caller, then the complexity increases linearly (but if many methods need to interact, then it's as if you wrapped all the cables up in one tie, and forced them all to go through the "main" method - not actually an improvement. However, it's rare for everything to interact with everything, and even then, it may be clearer to codify it somehow).
I also wanted to say that while proving correctness is important, the organization that is best for proving is not always the same organization that is best for human clarity (sometimes they are). It depends on what your goal is. Proving absolute correctness is not important for most programs (all software has bugs; bugfixes are common) - to sacrifice human clarity for correctness is usually not the optimal trade-off.
You're talking about avoiding temporary variables (see the refactoring book), which is an independent feature. Just talking about method length will only illuminate so much.
On the other hand, methods are modules, giving syntactic and compiler-supported semantic separation - and you can do things like return early.
I can't tell which is better.
Some people claim that short is better, but it always comes across as a bit rabidly dogmatic, because... well... it's without evidence. There is so much of that in comp sci: design patterns, functional programming, editor and language choice. Most people giving opinions don't even mention what type of task that advice is relevant for, nor give their experience that supports their choice. (It's easy to argue for a choice - smart people unfortunately can argue convincingly for anything.)
I tend to use separate methods only if they are reusable (otherwise it's a waste of time making them reusable). I often think of better ways of doing things, so I don't like to invest too much in what I have now. I'm mostly writing prototype code for new kinds of products, not "production code", not for clients, and no one sees it but me. Much code I've seen that it made of many methods and classes looks horribly over-engineered to me, especially when the problem itself is actually very simple if you approach it in the right way.
I'd love to hear gruseom's opinion tho.