Clear evaluation function for an objective metric if they are making progress or regressing.
Evaluation function is computed, not llmed.
Ontology of potential actions clearly specified.
Accurate inventory of the current status qou.
Clear enumeration of options from status quo towards the winner's circle.
Waypoint objectives with similarly concrete evaluations of pass/fail, or on target off target.
It's the same thing when leading a large organization to actually hit a goal. There's randomness every turn away from your mind, so the more constrained the options, the more likely you are to hit the target. The consequence is if you're wrong about the plan then with people you're fucked. Morale will plummet. With AIs, they are so nerfed emotionally now, you clear context and start again.
I did enjoy Sonnet 4 when they would swear randomly and become sullen or wax desperately. That would at least cause pushback against a bad plan.
I think most software projects have reached the point that the speed of capturing real information about what the winner's circle looks like, and therefore what the program should be, so many magnitudes slower than the amount of code that can be generated in the wrong direction.
I'd need to measure these new models on well understood but complex problems that are relatively easy to validate to get a sense if they are 'better'; on the other hand, the real impact in daily life may be marginal since generating code is not the biggest problem at the moment.
Humans have hard skills and abilities the ais can’t reproduce yet like real time learning, spatial reasoning, cheap parallelism, Qualia so we can identity QWAN (quality without a name) because we feel in real time what the code is.
AIs have skills humans aren’t good at like nerding out on technical details.
That’s not a perfect map because I’m spitballing. However there is a symbiosis.
I am not sure I am productive anymore with AI as I am up to 125 repos and agents most of which are tools for managing AIs and things break frequently that it feels like spinning plates.
I spent two months in November and December last year writing by hand a fundamental library to constrain how the AIs build clis. That did make things move a lot faster but for those two months I felt the slowness.
I think it will always be like this. It’s the nature of paradigm shift to shift.
The way I think of it is, computer memory is superior to human memory because it can store anything and re-call on demand when requested. This is great for the human because we no longer have to remember every tiny detail - just enough to recall the object and thus opening up room for space in the brain for other stuff.
The current algorithms have a limited context window and work linearly and are extremely expensive to change and energy intensive to run.
The human brain has a wide parallel multisystem real-time low-wattage execution layer that has way more modes than a large language model.
More importantly, because our brains are real-time, our qualia plus spatial and visual reasoning is superior to an LLM at understanding "elegance", "code smells", and overall system design because we can imagine ourselves as being the code or the system and we don't necessarily need to think in language. Well, at least that's how I experience coding in my mind; I imagine other developers similarly bring large parts of themselves into coding.
Feeling the code seems to be much more efficient at reducing complexity than any static analysis I've yet seen.
Finally, humans also empathize with other humans who have all the money. We know what works and doesn't work for humans in the here and now, not 2 years ago when the model training data was last collected. The value of Qualia is not to be discounted.
That being said, Sonnet 4.0 was the best model I've used that could express how the code felt, so who knows. If the emotionality wasn't tamped down, and the spatial reasoning improved, and the new algorithms for context engineering and parallelism make it to market, these advantages can be erased.
True, in May 2026. But this is only one version of this.
In the future, it's likely the open Internet will be 99.99% robots. It's already > 50% robots. The government ID system a lot of countries are adopting to keep teenagers off of social media would also serve to both help control for non-human spam, and also control the network period. It's also possible a private system of human-verification certificates may come up to meet the demand like Apple ID with biometrics. Could also be the liveness tests KYC companies use may be more popular.
I don’t think we can extrapolate from current trends like that (at least I hope not). Society is dynamic. People will adapt. If bots become a problem websites will take more and more strict measures against them.
Which is a long way of saying, for any big enough problem created by a YC company, another YC company will emerge to fix it.
Think about it from an information theory point of view. You need to attach a digital transaction to human body. Since a human body isn’t digital you need a gateway that you can trust to vouch for that human body being present.
Either you use biometrics, like liveness testing or face id or fingerprint testing, or social validation like decentralized web of trust or private moderation (account controls) or state methods like fines and criminal convictions.
Biometrics rely on social methods eventually like we trust Apple because we can sue them or the government will harangue them. Liveness testing is only as good as your sensor and image vs generation and replay in the arms race.
And iterated social games like punishment are only as good as people want to invest energy into it.
What I mean is that once you have a token that represents your identity, you can pass it to your agent. As you said, humans aren’t digital, so we need to delegate the trust to a digital marker (auth token, cryptographic signature, etc). But once digitalized there is no way that I know of to block an agent from using that marker. And I don’t mean the agent stealing it. I mean the human running an agent that impersonates them deliberately
Exactly. There is only an arms race, which is escalating costs. Eventually it breaks and we use social means to manage it, surrender the digital space, or accept the artificial nature of the digital realm.
Our current most popular methods of using AI with software development is either waterfall or autocomplete. We aren't at a great pair programming experience yet. I presume that would improve speed and accuracy, but it's still unclear.
A few things. A web of trust of some kind like vouching may come back, and general algorithmic silencing of low quality members. Also most governments are going towards the South Korean model of government-verified ID to post online to keep teenagers off social media. The same tool can be used to greatly reduce spam and slop, if that's what platforms want.
Also people will get used to AI in online spaces as AI quality improves. If I'm online trying to get help for some task, I personally don't care who wrote what if it is correct; it's not like humans have great track records of accuracy or substantial contributions either on average. Correctness is expensive in general.
If I'm online trying to relate to other humans emotionally, well I get what I'm paying for. It's been true forever that the better the gate, the better the community. I've tried to push the boundaries of openness, but as I've written extensively on MeatballWiki, soft security depends on there being more good than bad apples in a community. With machine intelligence, the economics of that are silly.
Regardless, people love people, so we'll figure it out. I'm optimistic we can rise to this challenge.
Clear evaluation function for an objective metric if they are making progress or regressing.
Evaluation function is computed, not llmed.
Ontology of potential actions clearly specified.
Accurate inventory of the current status qou.
Clear enumeration of options from status quo towards the winner's circle.
Waypoint objectives with similarly concrete evaluations of pass/fail, or on target off target.
It's the same thing when leading a large organization to actually hit a goal. There's randomness every turn away from your mind, so the more constrained the options, the more likely you are to hit the target. The consequence is if you're wrong about the plan then with people you're fucked. Morale will plummet. With AIs, they are so nerfed emotionally now, you clear context and start again.
I did enjoy Sonnet 4 when they would swear randomly and become sullen or wax desperately. That would at least cause pushback against a bad plan.
reply