We did it in the US, and filed a NOTAM with the FAA. The hardest part was figuring out how to convert what I had (we are launching a weather balloon from X, Y coords at Z time) to what they wanted (distance, direction, etc from nearest air tower). It's just a phone number you call, and they were pretty friendly (helping convert my lat / lng to what they needed).
There are rules you have to understand and follow, notifications required but no approvals if you meet certain criteria. It's mostly an FAA thing in the US, so one jurisdiction.
To me, working with AI feels a lot like the previous experiences of “working in a sprawling enterprise codebase spanning multiple systems each with emergent behavior”, except now I can outsource the introspection and validation loops to something that never gets bored instead of spending 2h hyping myself up to concentrate for a 3h stint.
At the risk of sharing an unpopular take, this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago and doesn’t seem to reflect the current reality.
(Or they’re working in an organization with lower budgets and not cranking the frontier models of today)
I fully agree about the cost/sustainability parts, but to suggest you can’t build a high quality coding/verifying/iterating loop for _most_ problems is disingenuous.
His experience is completely plausible. He’s in a niche that requires highly performant code and most complex, highly performant games do nit have source available for models to train on. It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become.
Have you actually tried performance optimisation using an agent?
With any programming language/framework that has quality profiling tooling (which is a prerequisite for most projects) I have had huge success with automated hotspot profiling where the LLM can propose theories, test the impact of fixes, convince you of which to pursue, etc.
High performance algorithms are quite well documented so it isn't unreasonable to expect an LLM to apply them appropriately when given the ability to "see" where they need to be applied.
TFA's author's experience is the opposite of your claims. Your claims may be right in _your_ circumstances, but not theirs.
> High performance algorithms are quite well documented
That may be true for bloom filters or what have you. But the author states the obvious: all recent games are closed-source. So any algorithms or techniques for real-time 3D that an LLM was trained on are going to be a long way behind the state of the art. The author makes that point extremely clearly, and they have credibility.
> this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago
TFA? No. Your comments here? Yep. Try to imagine a world where different kinds of work have different applicability of tools.
> That may be true for bloom filters or what have you. But the author states the obvious: all recent games are closed-source. So any algorithms or techniques for real-time 3D that an LLM was trained on are going to be a long way behind the state of the art.
Neither you, nor the author of the article has to use LLMs for coding. But if you want to, there are some practices you should follow to get best results, and that includes setting up your environment to give the agent the best chance for success. If you have various rules that are non-obvious, you add them to your AGENTS.md file (as we have at my corp and in my little niche). The agent will then follow those rules, and learn from the surrounding code.
I'll just be blunt here - I don't believe the author would have done anything more than the bare minimum to test his pre-existing bias that coding agents are bad. I don't believe he would have put the effort into getting it writing good quality code to suit the project he was working in.
We write high performance, low latency java for trading systems. Our codebase is highly structured around this and the READMEs and AGENTs file contain the information for how to successfully write code like this, originally for human consumption, and now agents.
Yeah, but we already have rules-based coding agents, they are called compilers.
I have a suspicion that a lot of the stuff people use AI coding agents for would be served just as well by, say, the WYSIWYG editor from Visual Basic 6. But we threw away that kind of technology long ago and settled for Reactslop, and now you need an LLM to write the Reactslop.
I am trying to not sound offensive here, but given the question you're asking, I don't think we're going to join up here. I think one of us is missing something fairly fundamental, and my hunch is it is you.
The agent will already have seen enough super optimised code and read enough material about it. It will read the entire repo you're in, and understand how the code should be structured and written to work efficiently. And anything it doesn't get at first, you write about in the AGENTS.md file and tell it.
Then it works. If you can teach a human to write specialised 3d rendering code, you can distill the same teachings to an agent in text form, and it'll work.
I am supremely confident of this, and I bet that neither you nor the author of the article has bothered to try properly.
I suppose you wouldn't train it using a traditional LLM type of training and you might not have an LLM type of model either.
American Fuzzy Lop usually manages to generate valid files of any format you want, using a genetic algorithm that the author didn't even call ML. AlphaGo trained against itself.
Such things aren't impossible when you can automate the reward function, though you might have to come up with novel techniques.
LLMs work perfectly fine for me but I’m building web app equivalents of binder keepers, like most people. Essentially store data, display data. The novelty is in what/how we’re displaying. LLMs are owning this market.
Unless you yourself are doing commercial game development, calling the author disingenuous puts your own pro-AI bias on display. It’s based on speculation. I prefer to take his very specific examples and experiments at face value.
To be honest, the optimization the author talks about isn't really high level optimization. And Claude's pattern actually calls for a more extensible design. Strictly speaking, the author's instruction could be considered incorrect. I'm not trying to dismiss the author.
If performance were truly critical, they wouldn't have been using Unity in the first place. They would have used Unreal, as mentioned earlier. And if they were sticking with Unity, they would have tried ECS.
Unity is fundamentally based on the template method pattern. The idea of pulling Update out and handling it in a single manager class is really more of a small scale indie game approach. It's a technique that scales very poorly.
In practice, there are many better optimization techniques for GameUpdateable. So I'm not sure why this particular example was used to demonstrate performance optimization.
Typically, you could use GameUpdateable with object pooling, which would be a safer approach. There are also many batching techniques available.
In other words, this isn't about performance. It's a technique used for small indie game development. By handling it directly through a manager, registration and removal no longer depend on the Unity framework and become manually scheduled by the user. This, in turn, means you have to handle many more edge cases, which creates additional work. This is a common pattern, sacrificing future extensibility for immediate performance gains.
It's a technique used in small indie games. Converting per frame Update callbacks into a central loop that iterates over all objects is where GameUpdateable would actually be a better choice. So rather than viewing this as an optimization for performance, it should be understood as a design choice made to make small games easier to manage.[1]
I’ve specifically used Opus to diagnose and fix performance bottlenecks in parallel Rust code on multiple occasions (e.g improving NPS for a chess engine) and it works well.
I’ve done plenty of performance architecting in my day-job and rule #1 is generally “you can’t fix what you can’t see/measure”. I have a suspicion that many folks aren’t investing in letting AI actually introspect iterative execution via the appropriate harness, and are then acting surprised that it is no oracle.
I understand you're trying to draw a parallel (no pun intended) between what you did with Rust and the work the author performs as a professional game developer who optimizes game code for a living (it says this in his bio).
Since, I'm assuming you are not a professional game developer, the parallel is speculative and sort of reaching. Therefore, is it feasible to you the author of the blog knows better than you what tools work for his chosen field and that he came to the conclusions he did in good faith?
> It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become.
It is a common observation but I don't buy it. AI is clearly very good at Rust, but that is probably one of the least represented languages in its dataset. Anecdotally, I've also been having very good outcomes with a rather niche combination of technologies (opencv.js + JS in a browser extension) since early 2024. I would imagine there is way more C++ game code in the training set than that particular combination.
I think the more likely reason is that certain languages, projects or technologies tend to be organized in ways that are not ideal for LLMs. Specifically, I think Object Oriented approaches are not ideal for LLMs.
My theory is the key factor for effective LLM use is how effectively you can stuff the context with only the relevant data. OO tends to result in logic spread across inheritance hierarchies and templates (and even overloaded operators /shudder) which resides in a bunch of different files comingled with a whole lot of other logic. This just tends to confuse the LLM. On the other hand, I ended up using a lot more functional programming style which let me pinpoint the exact files or snippets of code relevant to a task, and the LLM pretty much never went wrong.
These days the models (and likely the harnesses) are much stronger and need much less curation of context, and hence can power through any kind of project organization. But I suspect they are still a bit sensitive to all the noise polluting their contexts and hence can produce very inconsistent results.
> this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago and doesn’t seem to reflect the current reality.
I've been reading this for 2 years straight. "oh you have a criticism of AI? Well they fixed that in Aeternos v Y-point-Z, which after doing all of my work also gave my wife an orgasm for the first time this year, obviously OP is using the old model".
Moreso it seems like nonsense that's masked by an excellent facade of eloquence. Ironic given it's a a post ranting against LLMs.
I stopped reading when he decided to go off on some tangent about how LLMs can't do native programming or games because they weren't trained on it. Ignoring the technical issues there, an even more overt one is that, amongst a zillion other projects, even the entire source code for Unreal Engine is available and within their training corpus. Even older models were quite competent at working with Unreal and outputting idiomatic code, which is saying something if anybody's ever worked with Unreal.
I increasingly think people are writing dumb articles on purpose because it drives 'engagement' more than a straight forward and accurate post would, at least on average.
5 months ago people said the exact same things, you know. That this stuff was how it was 5 months before that, and doesn't reflect the current reality.
My favourite interview question that I received was when I was asked to write code that would serialize/deserialize a binary tree and was told it was fine to use "any programming language".
They received a crash course in the power of Haskell ADT and `deriving (Show, Read)`.
I'm so glad most of my technical interviews weren't about trying to remember things I once learned in college (15-20 years ago now) and never had to use in practice. It's been either practical "build a simplified webshop" tasks, or just an interview talking about my previous experiences. There was one where I had to do a fizzbuzz on a whiteboard but that was about it.
Those kind of abstract CS interview questions never feel that helpful to me, but in all honesty, your story sounds like a job of the interview working.
You were able to demonstrate that you were enthusiastic and knowledge about programming as a whole, which is probably more helpful than what they were originally trying to test for.
I think it's because most of us don't do CS once we graduate. We build APIs and websites which don't involve CS at all. It barely involves engineering. lol.
No one uses binary trees or some manual form of serialization.
They use "superlib.search(arr)" or (seriliazer.parse(thingy)" and that's about it. No one cares about what it does. They just know it does it faster than something they could write. The only places that care about it are at the utmost peak of scale.
Heck, I run an API that gets tens of millions of reqs a day and I'm an idiot that couldn't computer science their way of of a paper bag.
I'm not some Google/Microsoft/whatever engineer who's been working on highly scalable, highly available systems. I've mostly worked in consulting on small projects. I have never not had problems with the defaults on any of those sorts of libraries. I've almost always had to roll my own graph manipulation and heavily customize any serialization system I've used. And when I do so, because I've gotten good at doing it (in particular, I don't think of the graph stuff as being hard at all), it makes tons of other stuff inside around the project a lot easier.
I feel like people bend themselves out of shape trying to avoid these so-called "complex" or "esoteric" "only theoretical Computer Science" topics. You don't need a general purpose tree or graph manipulation library. Trying to make one ends up creating something far more complex than necessary and all you succeed in doing is making the calling code maybe two lines shorter but also not easily portable between projects, and also at a huge maintenance cost. It would be like trying to make your own, bespoke "Collection handling class" because you think trying to keep track of arrays and lists and dictionaries and sets and queues and stacks is too "complex, esoteric." No, you end up creating something complex and esoteric in the attempt to try to handle the erroneously identified "complexity."
In general, I don't think you can argue definitively on the uselessness of a piece of knowledge from a standpoint of ignorance of that knowledge.
> In general, I don't think you can argue definitively on the uselessness of a piece of knowledge from a standpoint of ignorance of that knowledge.
I agree, I think those things are definitely worthwhile learning. But I don't think that it follows we should test for them in interviews.
It sounds like you've done some pretty impressive work around graph algorithms, but if a job doesn't need you to do this, testing for it is a little silly. (If it's a criteria you're explicitly hiring for, I think that's clearly different)
Net worth or net income? Being 40yo with only 200k of worth between two when you’re halfway through your working life and have had 10y of appreciation of assets isn’t very much to go forward with.
Running with your ankle at an angle occasionally gives great strength/mobility benefits. You'd need to be putting out some serious racer pace for curb cuts to be impacting your experience in any way.
Just make sure you alternate which side of the road you are running on somewhat evenly so you are getting the benefit on both sides.
Off topic (but helpfully hopeful to someone!): In the rare cases where I get motion sick I can usually "hack" my way out of it by closing my eyes and imagining myself skiing down a mountain avoiding obstacles in an erratic path that matches the patterns of motion I'm feeling (calculated JIT).
Presumably you have to have a lot of experience skiing for this to work, but there are probably isomorphic activities (weaving around a crowd? Playing dodgeball?) that can hack your brain in a similar way.
You can endure as long as you want to, but we're all going to erode eventually.