Alan Kay is a brilliant guy, but I really wish he could open his mouth without shooting it off. In particular -- to speak to my own domain -- he bleats about "feature creep" in an operating system with absolutely no demonstrated understanding for a modern OS. Is KVM "feature creep"? Is ZFS? Is DTrace? He shows not so much as an ounce of understanding for why these things exist or empathy with those for whom they were created. I have great reverence for history (and Kay and I share an intense love for the B5000[1]), but I also think it's a mistake to romanticize the past. Viz., when he wistfully recalls the Unix kernel having "1,000 lines of code", he can only be talking about Sixth Edition: even by Seventh Edition (circa 1979!), the kernel had (conservatively) over ten times that amount.[2] And this is still a system that lacked a VM system and TCP/IP stack -- and tossed if you created the 151st process![3] If he wants to criticize the path that history has taken, fine -- but when he is so ignorant of the specifics of that path, it's very hard to treat him as anything but a crank, albeit an eminent one.
Cranks don't really put their money where there mouth is. If you haven't seen it yet, I'm sure you'll enjoy looking at the STEPS project: http://www.vpri.org/html/writings.php
In summary, the goal is a user-facing system in 25k SLOC, from kernel to GUI apps, including all the code for compilers for the custom languages they made. They call the languages "runnable maths", or "active maths", where they think of a minimal notation to specify how a system should behave, and then implement a compiler to run that notation as code. They manage this through a compiler-writing language, ometa.
For example, to implement TCP/IP, they parse the ASCII tables from the RFCs as code. Seriously, they just take it as is, and have a compiler to run it as code. The implementation is supposed to be around 200 LOC (includes the code written in Ometa for their compiler).
There's also a graphic subsystem, apparently with all the functionality of Cairo (which is 20k LOC) in about 800 LOC.
Crucially, all this code is supposed to be very readable. Not line noise like APL. The expressiveness comes from having custom languages.
I believe most of the code is visible in the linked article. The packet diagrams were redone but are very nearly identical to the RFCs, just some minor formatting tweaks. They are very readable.
The compiler (in OMeta) is a very BNF like grammar.
The readability is striking. The first time I heard of it, I was expecting it to be extra dense and hard to parse. It turns out they have entire lines of just separators counting against their limit, which seems useless until you realize one of the main goals is readability.
Incredible work, I'm looking forward to the end result.
This style seems amazing, but I haven't seen any tutorials for people interested in learning how to do this themselves.. do you know of any such resources?
They are doing the real deal thing - for each problem, designing a language in which to express the solution, and then writing the solution. Part of the point is that the building such solutions isn't really teachable in the context of a tutorial. If a tutorial can teach it, then it's probably a pretty shallow skill, and shallow skills should get designed away into a solution-oriented programming language, and any tutorials for THAT language, etc etc...
The language they use to do that (O-Meta) is then a language-defining language, and designing THAT means balancing on a level of abstractions that is at least three levels deep, and each of those abstraction levels is hard and necessary. They achieve great simplicity by thinking damn hard.
On the other hand, the mailing list (as noted by the other respondent) is a pretty interesting place with few flamewars.
That would be nice, but I think STEPS is a research project that looks into how things can be done differently. It would be nice if this particular implementation would result in a shippable product. But I doubt if that is their primary goal.
Kay and Ingalls et al. had a very specific vision for personal computing as liberating, and the keys to that vision were full access, full comprehensibility and making no distinction between users and programmers. The market has clearly shown that people would rather not have full access, don't care about comprehending the system and desperately want a firmly regimented distinction between users and programmers.
For me personally, the thought of giving up entirely on this vision is repugnant and it's important, even if such systems aren't going to make a grand reappearance anytime soon, to at least make sure that people who care about computing know that they once were. I see that reminding as Kay's primary role. I'm not switching to Pharo for my day-to-day work (in my defense I did try) but I find the philosophy appealing and don't see a compelling philosophical disproof. So it seems to me it could re-emerge someday, and a world built on these ideas will have different tradeoffs, and may be preferable.
Don't mistake the fact that normal users aren't interested in current tool chains and coding means that they aren't interested in being able to solve problems creatively. Kay mentions Hypercard in this interview, which had a long history of drawing normal people into designing solutions to their problems from scratch.
Back in the mid/late 80s I used to do work for a public library and helped set up several public Macintosh computers that had, in addition to word processing and spreadsheet software, Hypercard installed. I was amazed how people "got it" so quickly. From kids, natch, to street people to soccer moms. And by "got it" I don't mean running the built in stacks, but in creating their own.
Before long I was helping people make some really interesting projects. Some of them were beautiful monsters, but others were very well thought out. And all of them solved specific problems, many of which would not have had enough general appeal to be marketable or even transferable to anyone else.
And there's real value in having that ability. But these days most people simply can't get past the gatekeeper of beginning complexity. To be sure, in Hypercard you quickly ran up against a wall of limited extensibility in terms of its objects. I can easily imagine better.
But Kay is right, The web and most modern OPP aren't it. And modern tools for software development are a nightmare.
The market has no incentive to create a system that hands users a microscope that shows what is happening at the cellular level, that can, as a natural science not programming, demonstrate what is happening. To say the market has demonstrated that people don't care about comprehending is false: the market has not found a marketable way to sell explodable systems, and CS has not created runtimes that can illustrate their functionality, can be introspected into, in a high level or compelling systems view.
_The big problem with our culture is that it's being dominated, because the electronic media we have is so much better suited for transmitting pop-culture content than it is for high-culture content. I consider jazz to be a developed part of high culture. Anything that's been worked on and developed and you [can] go to the next couple levels._
I'm not sure that our media is necessarily better suited towards transmission: I suspect we just lack mediums that make looking at what is happening evident and interesting (even with X years of programming experience, watching another persons webapp run is rarely fun or an obvious exercise). This is a failure of our runtimes and the weak structures we use to build code.
I think inplicitly Kay is talking about personal computing, not servers, and that in isolation is a very narrow perspective. But following his thinking for a moment: does a 'PC' need VMs, ZFS and dtrace? Apple has shown that, by removing many things that seemed essential, like a user-visible file system, you can build a personal computer that appeals to a lot of people. I firmly believe this is not the right tradeoff to make, even for personal computing in general, but people are throwing loads of money at them...
If you are talking about iOS... it still has the concept of a filesystem [1], albeit in a slightly different form than normal.
Even if Alan Kay is talking about PCs and not servers, you still need important things like the ability to connect to the Internet (TCP/IP), IPC, VMM, threading and multi-tasking, etc. I suspect I've not understood your point though, so feel free to clarify what I've missed :-)
When I started using Unix in the mid-90s, RAM was still expensive, and the rule thumb was that to get best use of a workstation you should have 2x main memory as swap, 4x for a server. This was reckoned to give the best trade-off between the number of processes running and the working set you actually needed.
Shortly afterwards this stopped making sense because RAM got cheap. Should a 4G PC have a 16G swapfile? Should a 256G server have a 1T swapfile?
In short virtual memory was a hack to get around economical constraints. If you have the main memory, use it! Were it not baked into the OS at a deep level, I've have stopped using swap 10 years ago. It could be taken out of the next version of all the main OSs altogether, and no-one would even notice.
I didn't actually mean to add that second link in there (I realised you were talking about virtual machines and edited my comment accordingly... But forgot to remove the second URL!)
However, disk-backed page faults are but one component of a virtual memory subsystem. The real genius is the OS can provide access to memory uniformly in a way that is transparent to processes.
Not really. In Windows its just a change to a system setting to disable the swap file, and in a unix it's a config file change or a command like swapoff to turn it off.
The fact that the two are entwined now is because of system defaults and decisions by OS developers on how to implement the page replacement algorithms in MM subsystems.
Apple doesn't "remove a user visible file system", they just make direct file system manipulation not as much of a necessity. And yet, this is not why their OS is good: it's because, if you want to, you can manipulate the file system and view it in as much detail as a BSD (OpenBSD) server, with just as much ease, combined with great hardware-software compatibility.
The system still has the complete filesystem implementation, and despite it not being visible to end users I imagine life would be much harder for developers without it, at least without replacing it with something likely to actually be even more complex (in code, not in UI). I mean, it isn't like VM or DTrace is "user-visible"; if anything, it is less user-visible/intrusive than its predecessors.
[1] http://news.ycombinator.com/item?id=4010407
[2] http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7
[3] newproc() in http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/sys/sys/...