I have to say having just done a very tricky project with a long DSP pipeline in gnuradio, that LLMs are a godsend for getting up to speed on gnuradio blocks. I too found it historically impenetrable, but was able to get a pipeline completed. That said, prepare yourself for a lot of debugging and work to diagnose problems if you’re doing anything of sufficient complexity.
I think it’s just that Japanese culture represents a very sophisticated antithesis to many western cultural pillars. Collectivism over individualism, incredible development of the aesthetics of simplicity and emptiness compared to the ostentatious and sometimes baroque western aesthetics, neat and tidy compared to cluttered, the fanatical emphasis on mastery of technique, or things that can’t be produced via shortcuts, etc etc. Overall I think the Japanese are viewed as much more disciplined and conscientious which is valued in the west. And a lot of these things seem like wizardry to Americans because we’re lacking the routines and building blocks that lead to it. In essence the Japanese are very process oriented, and Americans see the results and want them but don’t understand the process side of things as it’s not intrinsic to our culture. Yet the results are admirable… whether or not Americans would actually want to live the process that leads to those results is another story. That’s the gist of it I think.
> incredible development of the aesthetics of simplicity and emptiness compared to the ostentatious and sometimes baroque western aesthetics
We're just going to pretend that Japanese web design is identical to the rest of the world's, and that Japanese homes aren't half the size of elsewhere such that every horizontal and vertical service is commonly used for storage.
I live in Japan and in the rare event that there is an online option when I need to do something related to government documents, banking, insurance, or the like, all I can think is-- "wtf is this, geocities?".
It's always a clunky, awkward, boxed in mess of text that is a miracle in the case that it actually works. But that's just the way it is here. Lots of elderly, and the elderly prefer a nice helpful receptionist, I guess?
According to my Japanese GF, the elders don't understand it either. She had to setup some banking services for her dad and it was a painful mess, but doing it in person is equally annoying. So at this point it is a fucking mystery who actually likes the system and why nobody tries to change anything.
That’s not too much a concern I think. Most of the real pathogenic qualities result from known genes. Inventing a totally novel virus that is also pathogenic is basically science fiction right now. Splicing together known components that could be bad in combination is closer to current technology.
I think it’s possible this might just require reworking the transformer architecture a bit to give it an inherent awareness of the contents of its attention. Kind of a working memory workspace of sorts.
I.e. Instead of limiting autoregression to the token level, you introduce a persistent compressed global workspace latent memory vector that is fed back into the self-attention mechanism at every layer or every token step, allowing the network to attend to its own prior attentional states before computing the next token. Obviously that’s going to involve some compression steps.
Trouble is… I think the architecture there is much simpler a tweak than figuring out how to train it.
…that’s likely to just destabilize training
for not much if any gain at first. You’re probably gonna have to resort to some really clever (and currently missing) tricks to figure out how to train the network to actually use that feature.
I’m not endorsing the current administration’s policy, nor do I think it would succeed, but I find this comment really perplexing since this action is PRECISELY in line with the 2025 National Security Strategy. [1]
Also, for further insights on some of the unstated policy, I recommend reading the project 2025 document as well. [2]
Perhaps by “coherent policy” you meant “sound” or “effective”? But as it stands it certainly is coherent.
I think this is a bad way to look at it. LLMs can probably conceive of most things that are representable within the embedding space.
Ordinarily in mathematics there’s a TON of papers to write just combining low level problems with different techniques. Better still, and often considered groundbreaking is borrowing techniques from other fields and adapting them or creating analogous methods to solve problems. A lot of landmark papers have been written this way. This is also what transformers are sort of good at within other contexts. They have super human breadth so I’m hopeful they’ll become real assets in math for a long time. Though the leaps necessary to adapt a technique in a nonobvious way might be too much for a while longer. We’ll see.
Truly novel techniques are quite rate indeed and I don’t know if LLMs can represent them faithfully in their embedding space or not. My inclination is that they probably can most of the time, but I don’t know. Mathematicians would describe such thins as “alien”.
reply