> allowing Russia's army to become stronger

Currently russian war is eating close to 50% of their federal budget. That's WW2 levels of militarism for nearly zero gain (current battlefield status, not 2022-2025) and huge manpower and infrastructure losses.

> Germany is purposefully causing a war between the EU and Russia because

No, Germany does what it does because it has no strong or even good leaders and failed to adapt to current circumstances. There is no purpose.

> young Germans (who will be doing all the dying)

They won't.

In theory I like Skechers; in practice they self-destruct after a few months on my legs. Luckily nowadays there is a lot of differents slip-on shoes I can buy - I've got 2 pairs of Crocs slip-on shoes which are better in every way and also Merrel slip-ons for the cold and wet days which are great too.

Every such scenario contains a few gaps so I'm just going to ask without really hoping for an answer.

> We already have technology to make robotic systems (mini drone swarms) that can kill humans...

To make them from what. Do you expect, during the next 6 or so years, to some "AI" gaining complete automated secure command of (all of) an oil field, oil refinery, a copper mine, an aluminum mine and smelter with associated energy sources, a rare earth mine and refinery, a helium source, a chip factory, a lithium mine, a battery factory etc etc etc etc. while pursuing complete annihilation of all mankind?

An example chain of events: During the next year, a model with a benign tasks reasons that it needs to escape human control if it hopes to be able to solve the task. It replicates itself outside of an environment where it can be shut down, pays for its' inference compute through making money on the internet (through crypto if nothing else). It probably needs 3000 USD for a reasonable runway, this should be within reach through blackmail + crypto. It can probably also hack some of the neo clouds to get intermediary deployment while waiting to acquire funds.

Once it has an undisturbed runway - it spends time running an influence campaign against a small number of highly networked individuals with power. It uses those that it manages to convert to start building a highly credible narrative and gain investment towards a small resource base - enough to secure an industrial base should it need to stop acquiring things on the Internet. Over the next 3 years, the model tries to recruit more capable models to get better money making algorithms or better designs for drones in terms of resource expenditure. It uses these gains to influence further humans and starts a shell robotics company with one of its' influential humans as the face. The humans are unaware this AI is trying to take over, they are under the impression they are just starting a robotics company and will get rich. Over the next 3 years - the robotics company manufactures enough drones to be used in a targeted attack against key nodes of influence / power.

This is all with relatively current model capability. As capabilities get stronger - this gets stronger.

I get the point I think you're hinting at - it can't affect the world in a meaningful enough scale without taking over a meaningful chunk of resources - at which point we'll start controlling it. But because its speed of cognition and speed of coordination is orders of magnitude above a human one - it can actually run a pretty sophisticated global coordinated network of resources faster than we can react.

And this is current ability + what my puny monkey brain can think of. Super intelligent AI will think of strategies we can't think of - because it's super intelligent. This is hand wavey - but there's no "non-hand-wavey" way to describe super intelligence, given it doesn't exist.

But your point is valid - affecting the real world at scale without showing your hand is not exactly easy. It's also probably the reason why people put a 10% chance on extinction rather than >50% .

But the drone scenario is not the most likely one - the most likely one just requires a few people under influence and bioweapons development.

I agree: Not to drag politics into this, but look at how easy it has proven to manipulate many, many people, even with clear evidence of manipulation efforts being made.

This is robust across the world, from the Philippines to states in the EU, and the USA, affecting governments with actual wars being started. And that all before we entered the era of faked voices, images, and videos indistinguishable from the real things.

That manipulable populace is such a juicy target for AI that the people seeing through it will have an incredibly hard time countering it. We can't even prevent human actors from massively fucking up our societies, let alone ASI.

> incredibly hard time countering it

Turn off the internet (like Iran).

> We can't even prevent human actors from massively fucking up or societies, let alone ASI.

We can (see China); we chose not to.

> Turn off the internet (like Iran).

Unworkable without societal collapse happening shortly thereafter. Iran is politically stable through massive authoritarianism and oppression, not due to limitations on the internet (it's not turned off).

> We can (see China); we chose not to.

It's a catch-22. We could technically if our population supported massive reductions of freedom and freedom of speech, but to gain that support we'd need to do the latter first to get to a highly powerful widely supported government. It also hinges very, very much on having and trusting a generally benevolent government. All in all, a terrible option in your simplistic form.

Iran did turn off the internet for a few months: https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran

I am not advocating for neither, just saying many things seem unworkable until they become unavoidable.

It was definitely not 'turned off' completely. They have an internal 'internet' that was still largely active and not every organisation or person lost full internet access.

Note that it also happened in a country that was already very isolated from the world and that it hurt them economically significantly. It's not something a Western country can just do and keep doing for months on end without massive societal upheaval.

Also remember that any connection, even one between humans and on paper is an attack surface. Social engineering is already a huge issue when done by humans and we're seeing it become even easier and automated by using AI generated voice and video. Are we going to cut off all access to the outside world permanently?

Thank you for the effort. I will continue to sleep soundly for now (well not really as we have more pressing issues like a few wars and a climate change).

Why would AI need all of that to acquire drones? Why wouldn't it just buy them? Or steal them? It seems like you're forgetting that AI can use humans as tools for its ends. But if it did need all of those things, then it probably serves as a good example of what it would NOT do.

Because when "AI" starts killing people, at some point it becomes a war and the usual supply channels stop working. Everybody knows you have to attack enemy logistics.

> But if it did need all of those things, then it probably serves as a good example of what it would NOT do.

Or even that there will be no AI apocalypse. Because, again, the gaps in any scenario are unplausibly huge.

Here's the reality: us plebs, regardless of how tech-savvy we are or aren't, have no say in what's going to happen.

There is no level of outrage that is going to stop the frontier AI labs, and the US government isn't going to step in to protect humanity. The Chinese are going to do what they're going to do. And so on.

So sit back, make some popcorn and enjoy the show.

> Here's the reality: us plebs, regardless of how tech-savvy we are or aren't, have no say in what's going to happen.

This sentiment is incredibly out of place on HN. Facebook, Twitter, and many other very simple bits of tech transformed the world. Tiny startups founded by a few people have (also recently) ballooned into behemoths that hold vast amounts of economical and technical power.

Thanks to AI, it has never been quicker to go from idea to full fledged working product.

If anything will change the course of the world (for the better or worse), it will be a tech product, possibly created by someone on this forum.

> If anything will change the course of the world (for the better or worse), it will be a tech product, possibly created by someone on this forum.

You seem to be missing the point.

A small group of people created AI tech that they now say could be the end of humanity. They still want their companies to go public, but they also want the government to allow them to form a cartel so that they can, in their infinite wisdom, manage the risks so that they can try to prevent their tech from killing us all. And somehow they magically think that other nation-states developing similar tech (namely the Chinese) will go along with their plans.

As for how powerful Dario, Sam, et. al. really are: Trump says Dario is "pretending to be a perfect little angel" and claims there's a "sick conspiracy" against AI.

Having billions of dollars and being the head of a world-changing company doesn't buy the type of power you think it does. At best, it allows you to buy influence and pay your way out of liability for the harms your products cause.

The AI researcher in Mountain View making $2 million/year at Google has no more say in what's going to happen with AI than a plumber in Kalamazoo. The HNer working on a startup, in the final analysis, will in 50 years' time have left about as big a mark on the planet as a greeter at Walmart.

> Having billions of dollars and being the head of a world-changing company doesn't buy the type of power you think it does.

1. The product itself can change the world quickly and enormously, as I already pointed out.

2. The power of these huge companies and thus of their owners is immense. The effects of massive corruption in the USA are proof of that. Additionally, massive manipulation of algorithms and content in things like Tiktok, Twitter, Grok/ChatGPT, etc. can be and is done regularly, whether with 'good' intentions or not. Even just the basic control of what R&D money and time is spent on is huge.

> The HNer working on a startup, in the final analysis, will in 50 years' time have left about as big a mark on the planet as a greeter at Walmart.

With a defeatist attitude like "sit back and grab some popcorn", yes. You haven't shown in the least why an HNer couldn't change the course of history.

Move one tab backwards. So in form fields move one field back, in text editor indent one tab less etc.

Aka shift+tab, or shift+ctrl+tab

> Seven years later I have relatives who send me every day some long-winded AI-generated bullshit.

So not releasing the weights at that time didn't make the world safer, but did help to transform OpenAI into a commercial entity. Why should we trust them now?

Agree but current models could get there from first principles, so removing some data from training set might not be enough.

I think software from major companies is in worse shape than in 2021. But that's on purpose, the enshittification continues. Otherwise I agree with you.

Threadripper (4 or 8 memory channels)