How do you get around heap fragmentation? I know that the JVM (Oracle I believe) is really limited to about 32 GB of RAM before it has real issues. But the nice thing is that the GC will compact the heap for better future performance.
As a possible work around to the JVM limit, a distributed architecture with N JVMs running a portion of the task could solve the small memory space with minimal system overhead. What I mean by this let's say you need to have 64 GB of memory for your app. Given the comment above, Java would not do well with this. But you could have 4 16 GB VMs each handling 1/4 of the work. The GC would prevent fragmentation that you'd see in long running C++ apps and still provide you with operational capacity.
Heap fragmentation hasn't been a big problem for me. Using multiple JVMs means to reimplement all data structures in shared memory and create my own memory allocator or garbage collector for that memory. It's a huge effort.
Many applications can distribute work among multiple processes because they don't need access to shared data or can use a database for that purpose. But for what I'm doing (in-memory analytics) that's not an option.
You've probably since moved on from this converstation, but I wonder if Tuple Space might help [1]. It provides a distributed memory feel to applications. Apache River provides one such implementation.
Another question about in-memory analytics is do you have to be in-memory? I'm currently working on an analytics project using Hadoop. With the help of Cascading [3] we're able to abstract the MR paradigm a lot. As a result we're doing analytics across 50 TB of data everyday once you count workspace data duplication.
Thanks for the links. The reason why we decided to go with an in-memory architecture for this project is that we have (soft) realtime requirements and complex custom data structures. Users are interactively manipulating a medium size (hundereds of gigs) dataset that needs to be up-to-date at all times.
The obvious alternative would be to go with a traditional relational database, but my thinking is that the dataset is small enough to do everything in memory and avoid all serialization/copying to/from a database, cache or message queue. Tuple Spaces, as I understand it, is basically a hybrid of all those things.
As a possible work around to the JVM limit, a distributed architecture with N JVMs running a portion of the task could solve the small memory space with minimal system overhead. What I mean by this let's say you need to have 64 GB of memory for your app. Given the comment above, Java would not do well with this. But you could have 4 16 GB VMs each handling 1/4 of the work. The GC would prevent fragmentation that you'd see in long running C++ apps and still provide you with operational capacity.