I have no idea about timelines, but the current LLM architecture has no mechanism which could emulate online learning in a way similar to how it happens in humans and other animals. As another commenter wrote, there is no neuroplasticity while these systems interact with their environment in normal usage; adding information/constraints to the context partially mitigates this, but it's a completely different mechanism.
The RL phase is the most similar mechanism I know of that comes to my mind, but I'm not sure it could be adapted to fill that gap.
My totally unsubstantiated theory is that this is the missing link towards what most humans would consider AGI. I don't see this as intractable, but it may require some substantial change in architecture.
We've had that concept for quite a long time now, in the form of Lora [1] and similar fine-tuning techniques.
It first got popular for StableDiffusion to teach the image generation models new concepts.
We could easily live in a world where you can train / build Loras to encompass your entire code base history, company knowledge base, new skills, etc.
Then the models would start with a baseline that already has all the important knowledge without needing to cram it into the context.
This still isn't on the fly learning, but you could imagine daily or weekly training runs to regularly incorporate new knowledge.
I think the main reason this hasn't happened yet is that the shared batch based efficient serving architectures used today wouldn't support that structure well.
Yes, that could be a way to achieve that. After all, even humans have short term and long term memory, and it looks like sleep is a very important "tick" to connect the two, so it's not all just continuous.
For sure, doing it for all users would be economically unfeasible. I wonder if the labs are experimenting with something similar, though.
Chain of thought reasoning is really good, true, but agents really took off with tool calling and being able to integrate with a bunch of stable tools.
In some sense, creating good new abstractions externally and learning those is a form of "learning", on a very large timescale. The AI model is not necessarily doing the whole "look at the whole space holsitically and find a key invariant", but if you let other people do that you can enable new capabilities that were previously unknown.
Given how important matrix multiplication with a huge number of fixed parameters is becoming, there is an enormous incentive to design much more efficient architectures where this very simple compute is colocated with memory. Inference cost would come down a lot.
With the size of these matrices I don't think they are even meaningfully colocated with themselves in memory. You'll end up with some dataflow TPU architecture anyway because you'll have to stream the second matrix to multiply against.
Exactly where I see this going as well. Sure, a smartphone might be a nice place to introduce such tech. But matrix multiplication is literally where all non-labour jobs are going - this is the bedrock for efficient (time, energy) machine learning and inference.
If AI really is going to eat all our jobs, then matrix multiplication in memory is almost a requirement.
Memory fabs take years to build, they cost a lot and there is no guarantee that the demand will stay as high years in the future. It's all very complex.
30-40 per day means between 10,000 and 15,000 Palestinians killed per year. Just think about that number for a minute. I think that you're giving Ben Gvir way too much benefit of the doubt here.
And a 1,277,500 in ten years! Except no where did he say that this is the plan until the end of days, or until the end of the year. He said what he think Israel should do now.
Again, to clarify, I do not support him nor wish for any of his plans to become real, and I'm deeply said and ashmed of every plan of his that did become real. What I was commenting on what about what he said, though, which is already a deviation from the original post in discussion. If we're going to start talking about Ben Gvir in general, for all the I care he should be sitting in jail, already since the 90s.
I'm sorry, but what does the current length of the war has to do with this? During an armed conflict one can be asked what they think needs to be done, and it doesn't mean that this is what they think needs to be done even one week from now. And again, even if he thinks so, it is clearly not what he said. If we're trying to guess what Ben Gvir really wants, I can only agree he probably wants terrible things. If we're talking about what is said on Israeli TV, then that's simply not what was said and I'd like to believe that it's not something the general public would accept.
It's legitimately weird to see anybody defend them, as the immorality is out in the open.
The past few years have really opened my eyes to the nakedly tribal nature of democratic politics. Polybius was largely correct when he said that it inevitably degenerates into oligarchy or mob-rule -- but what's especially unfortunate is that the US and Israel appear to be getting both at the same time: A naked oligarchy with just enough mob rule to keep the animal spirits of the populace sated.
I don't view Ben Gvir's rise to a position of powers as a product of democracy, as much of a product of artificial states, neocolonialism, ethnostates, and apartheid. More directly, his rise to power is a direct result of the financial support of Israeli settlements over the past century which could not have existed in the absence of such financial support: settlers were paid to live in a specific strategic locations, in a specific way that would further the Zionist mission (first exclusionary, later apartheid).
Any power structure will be twisted and corrupted in the presence of these state systems to give people like Ben Gvir (or Stephen Miller, if you'd like) the platform and power to unleash their naked evil on the world. This is not unique to democracy.
I think it's upon us - all of humanity - to denounce artificial states, neocolonialism, ethnostates, and apartheid. They are founded on principles that are misanthropic, asocial, and unhumanitarian. They have always led to an immense amount of suffering. But they are not inevitable, termination mode of democracy.
> I don't view Ben Gvir's rise to a position of powers as a product of democracy, as much of a product of artificial states, neocolonialism, ethnostates, and apartheid.
From my understanding Ben Gvir(who has very little overall support in Israel from the population at large) getting elected is more of a consequence of Israel having nationwide proportional representation which allows extremists to get elected.
Ben Gvir then managed to become a minister due to Netanyahu having insufficient support from moderates to form a coalition without having to work with extremists like Ben Gvir.
> I think it's upon us - all of humanity - to denounce artificial states, neocolonialism, ethnostates, and apartheid.
So most states in the middle east should be dissolved? It's not clear what you're actually advocating for in practice.
> They are founded on principles that are misanthropic, asocial, and unhumanitarian. They have always led to an immense amount of suffering.
So we should dissolve most states including the United States due to flawed founding principals?
> From my understanding Ben Gvir(who has very little overall support in Israel from the population at large) getting elected is more of a consequence of Israel having nationwide proportional representation which allows extremists to get elected.
The existence of the support for those extremists has the same root cause that I stated in my original message: the artificial and strategic neocolonial state propped up by outside financial support with the expressed goal of establishing an ethnostate that eventually and inevitably led to apartheid.
> So most states in the middle east should be dissolved? It's not clear what you're actually advocating for in practice.
You can't unscramble an egg. Dissolution of dysfunctional but non-anarchic states is obviously a hyperbolic solution that I think you're aware brings nothing to the table as a suggestion.
What I'm advocating for in practice is quite clear in my original post, but I'll state it again: we need to denounce artificial states, neocolonialism, ethnostates, and apartheid. Much in the same way that we denounce war crimes, sex trafficking, and slavery.
> So we should dissolve most states including the United States due to flawed founding principals?
Again, you're bringing nothing to the table here with hyperbole. But in a way, yes, during Reconstruction a more heavy-handed approach should have been used to eliminate with great fury and prejudice the cultural desire for an apartheid state in the South, and maybe prevented or softened 150 years of suffering for black people and their descendants.
> The existence of the support for those extremists has the same root cause that I stated in my original message: the artificial and strategic neocolonial state propped up by outside financial support with the expressed goal of establishing an ethnostate that eventually led to apartheid.
Do you attribute Palestinian support for extremism to a similar root cause? Opinion polling indicates the extremism problem is far worse on the Palestinian side, Israeli extremism trends tend to be fairly reactionary to Palestinian extremism(i.e. Second Intifada) historically.
> What I'm advocating for in practice is quite clear in my original post, but I'll state it again: we need to denounce artificial states, neocolonialism, ethnostates, and apartheid. Much in the same way that we denounce war crimes, sex trafficking, and slavery.
So you're just advocating just for more virtue signalling instead of actual concrete actions? I don't see how that would meaningfully change the situation on the ground as I don't see how denouncing the right for Israel to exist is going to do anything except push Israel towards a more extreme position.
> But in a way, yes, during Reconstruction a more heavy-handed approach should have been used to eliminate with great fury and prejudice the cultural desire for an apartheid state in the South, and maybe prevented or softened 150 years of suffering for black people and their descendants.
Are you suggesting Israel also take a more heavy handed approach when it comes to eliminating the widespread practice of teaching of hate and antisemitism[0] to Palestinians?
> Do you attribute Palestinian support for extremism to a similar root cause?
Defending your home and against genocide is not "extremism". Anyone who's familiar with the history of Palestine will understand why they fight for their homelands and families:
> Defending your home and against genocide is not "extremism".
The Nakba is not genocide, it might be considered ethnic cleansing but that's not remotely the same as genocide. It also wasn't just one side doing ethnic cleansing either, Israel has a large Arab minority population with full citizenship while the same is not true for Jews in other Arab countries or Palestinian controlled territories(i.e. Area A in the west bank).
Massive amounts of Jews were driven from their homes around the same time as the Nakba throughout the middle east and they are not fighting to take back their original homes. This idea that everyone can somehow go back to their original homes or ancestors homes only serves to perpetuate the conflict.
People flee from wars, sometimes that's simply the least bad option, I have ancestors that have lost their homes due to their home country losing a war of aggression but I am certainly not about to go fight to try and take back my ancestors homes. That's all settled history at this point.
> Anyone who's familiar with the history of Palestine will understand why they fight for their homelands and families
There's much more to it than just the Nakba, it's also the many years of Palestinians being used as a tool by Arab countries to fight Israel.
Keep in mind the UNRWA(UN agency which is in charge of Palestinian refugees) unlike the UNHCR(UN agency in charge of non-Palestinian refugees) does not have a mandate to resettle refugees which deliberately perpetuates the conflict. Arab countries also coordinated a refusal to grant Palestinian refugees or their descendants citizenship as a way to perpetuate the conflict.
Then there's the coordinated antisemitic terrorist propaganda taught to Palestinian children as well.
> The Nakba is not genocide, it might be considered ethnic cleansing but that's not remotely the same as genocide
It was absolutely an ethnic cleansing and the killing never stopped. Israel has been murdering Palestinians (and it's Arab neighbors), since its inception.
All of the things you're saying have been said by Zionists a million times, and presumably are being pumped out by the think tank in the OP to flood the internet with trainable propaganda. It will not work though, the truth is known by too many people.
> Do you attribute Palestinian support for extremism to a similar root cause?
Yes. But we typically don't call oppressed peoples "extremists". For example, I wouldn't call those who rebelled in the Warsaw Ghetto "extremists". I wouldn't call a 14 year old boy who has only known struggle his whole life under apartheid an "extremist". Apartheid also has had a lasting effect on oppressor / oppressed dynamics in South Africa in a sad and complicated way. There is a wrinkle that Muslim extremists fund Palestinian groups, but I view this as yin to the yang of Israeli settlement funding (also extremist). Two sides of the same apartheid coin.
> So you're just advocating just for more virtue signalling instead of actual concrete actions?
Calling everything one disagrees with "virtue signalling" is a tired trope. There is no solution that will occur in our lifetimes, because there is no benevolent and omnipotent authority that can snap their magic finger and cease the suffering that's occurring because of this apartheid ethnostate.
With my suggestion, I'm trying to prevent the next ethnostate disaster. Ideally, I would want our descendants to recoil in disgust when thinking of the concept of funding an ethnostate in the same way that we recoil in disgust at the concept of funding a sex trafficking ring. I will teach my children, and my children's children, that "A State for [] People" is disgusting, no matter what [] is - unless it is "A State for All People".
> I don't see how that would meaningfully change the situation on the ground as I don't see how denouncing the right for Israel to exist is going to do anything except push Israel towards a more extreme position.
It's not that Israel doesn't have a right to exist, you're putting words in my mouth. South Africa was once an apartheid state, and it continues to exist. Similarly, ethnostates need to find ways to transition away from being ethnostates. Being an ethnostate can only lead to crimes against humanity.
> Are you suggesting Israel also take a more heavy handed approach when it comes to eliminating the widespread practice of teaching of hate and antisemitism[0] to Palestinians?
If only I could trust the state of Israel to actually design a "heavy handed approach" that wasn't rape, pillage, murder, and terrorize. The Union could be trusted, but only barely. The Union famously allowed black women to testify against the Union soldiers who raped them. I could never imagine the state of Israel allowing a Palestinian to testify against an Israeli soldier who raped them. Not yet, at least.
> But we typically don't call oppressed peoples "extremists".
Many groups are oppressed to various degrees for various reasons but restoring to terrorism(especially against civilians) is arguably an extremist response. The two are not mutually exclusive.
> For example, I wouldn't call those who rebelled in the Warsaw Ghetto "extremists".
They were being exterminated. I don't recall there being much terrorism targeting German civilians during that rebellion either.
> I wouldn't call a 14 year old boy who has only known struggle his whole life under apartheid an "extremist".
It's not oppression that would make one an extremist, it would be for example holding extremist views such as the supporting of terrorist attacks against civilians. One can teach extremist views to a 14 year old boy, in fact it's often much easier to teach extremist views to children than to adults.
> Calling everything one disagrees with "virtue signalling" is a tired trope.
Advocating for more condemnation without proposing actual solutions is rather pointless unless your end goal is endless war.
> There is no solution that will occur in our lifetimes, because there is no benevolent and omnipotent authority that can snap their magic finger and cease the suffering that's occurring because of this apartheid ethnostate.
So we should just give up and advocate for policies that will clearly never lead to peace? Also an occupation is not the same as apartheid. Apartheid is race based discrimination while the situation with Israel/Palestine falls mostly under the category of citizenship based discrimination(which most countries practice to various degrees).
> With my suggestion, I'm trying to prevent the next ethnostate disaster. Ideally, I would want our descendants to recoil in disgust when thinking of the concept of funding an ethnostate in the same way that we recoil in disgust at the concept of funding a sex trafficking ring. I will teach my children, and my children's children, that "A State for [] People" is disgusting, no matter what [] is - unless it is "A State for All People".
The middle east is full of ethnostates, compared to most states in the region Israel does reasonably well at protecting the rights of minority ethnicity citizens.
> South Africa was once an apartheid state, and it continues to exist.
Sure, but I don't think black South Africans were advocating for the destruction of South Africa as a state during apartheid.
> Similarly, ethnostates need to find ways to transition away from being ethnostates.
How do you propose states do that? Israel is clearly never going to give Palestinians full citizenship when the Palestinians at large do not believe in equal rights.
> If only I could trust the state of Israel to actually design a "heavy handed approach" that wasn't rape, pillage, murder, and terrorize.
So should a different country or organization be trying to de-radicalize Palestinians?
> I could never imagine the state of Israel allowing a Palestinian to testify against an Israeli soldier who raped them. Not yet, at least.
Israel has allowed for testimony from Palestinians in cases to some degree AFAIU but I don't think simply being allowed to give testimony is really the main issue here. The real issues are more tied to the way justice is administered, systematic double standards and the way military law is applied to non-citizens. The lack of any sort of final peace agreement between Israel and Palestine is what perpetuates this system however.
Think we're done here. I don't see a cohesive theme that deserves a response in between your responses to very specific pieces of my argument rooted in deflection and whataboutism. I feel I've expressed my viewpoint sufficiently. I don't feel like debating the definition of "terrorism", "extremist views", "race", "South Africa", etc.
Thank you for holding your ground here. As frustrating as it can be to wade into these threads, pushing back brings some much-needed sanity to a conversation that otherwise gets drowned out by propaganda (or at least the one i'm used to as consumer of regular western media). Beyond the main points, this exchange is a useful case study in Hasbara and bad-faith debate tactics.
(whataboutism, shifting goalposts, and semantic traps designed to derail the discussion; demanding an immediate, comprehensive policy solution just to validate a basic moral critique is a funny double standard given that the Israelis themselves offer no peaceful policy solution or path toward equal rights)
Kudos for calling out these rhetorical games in the open
Well you are implying Israel should be dismantled , in the most optimstic scenario that still involves a large chunk of Israeli Jews finding another place to live.
illuminating thanks for sharing this point of view.
funny that recently I came to a similar conclusion regarding power structures being corrupted anyway at the end, independently from the starting political form, is like we humans tend to be corrupt and violent at the end.
how many "democracies" are currently at war? or are doing violent covert or non covert operations/strategies? who controls that?
"I think it's upon us - all of humanity - to denounce artificial states, neocolonialism, ethnostates, and apartheid. They are founded on principles that are misanthropic, asocial, and unhumanitarian. They have always led to an immense amount of suffering. But they are not inevitable, termination mode of democracy."
this is gold, we can close the conversation here :)
> we humans tend to be corrupt and violent at the end
On a big picture "we" the west are perpetuating a type of colonialism. On the invisible "other side" in the global south, the outlook on a statement like "every conceivable political system always leads to corruption and violence" is quite different, better things are imaginable to them because it means liberation, freedom from oppressive forces, and so on. In the west it is the opposite, its unimaginable because an end to the oppressive forces means "we", means history literally ends, it's circular it's why the west views itself as the peak of civilization nothing that came before or after is even conceivably better than us.
I’m genuinely surprised they aren’t here in this thread doing exactly that. The defenders pop up in every corner of Reddit any time anything related is mentioned. That there isn’t a similar effect here is quite interesting.
paul Graham is a very brave and vocal critic of apartheid policy of Israel. I legit almost cried when I saw one of this posts... Honestly it was just so unexpected. This place tows no line, and weeds out the bots
It really disturbed me discovering how many nominally liberal people who would react in horror at casual use of the n word are quite comfortable with genocidal racism.
As you said, nominally liberal. What Israel is doing is deeply illiberal, and actual liberals, if there are any left, don't support the Zionist project.
That's why support for Israel is such a good wedge issue that leftists keep hammering on. If a nominally liberal politician is willing to support such an obviously illiberal position, then what other fundamental liberal values are they willing to give up?
Use-mention distinction is lost on a lot of people. It's a useful test for whether someone is reacting to a word (e.g. "damn") out of pure emotion and conditioning, or whether they're acting rationally.
Unfortunately there will always be people across the political spectrum who pearl clutch over media they personally find distasteful. The question is do any of them care when the rubber meets the road in reality. Christian conservative, progressive atheist, all are capable of failing that test.
Not just “a minister”, but the Minister of National Security. I mean, yeah, the “you can’t judge the intent of Israel’s security policy by Ben-Gvir” would maybe hold some water if he was, say, Minister of Tourism (though if a Tourism minister was telegraphing genocidal intent that wasn’t that of the government, you still wouldn't expect them to stay around long, either) but...
I've been using it to vibe code a personal application while I'm using a more "powerful" setup for my main agent assisted coding, and I have to say that it's been very good so far. I just wished that it was easier to hand off some tasks to codex web, like I tried to do today as I was starting a boat trip.
Come on, it's not just MBAs. I remember when patio11 (nothing against him personally, it's just the best example I can remember) became a star here on HN by saying that the solution to every problem as a consultant was to hike your rates. The idea of just asking as much money as you can get away with, which was seen as reprehensible by most of the population (my guesstimate, I don't have hard data) until more or less the eighties, Reagan and Thatcher era, has now permeated Western society at most levels, not just MBAs.
PS Chinese companies were (and probably still are, when they can) stealing IP, just like the USA did when it had more IP to steal from Europe than to protect from others. Western companies were very happy to let that happen when in exchange they could make more profits in the short term by using cheap Chinese labor and infrastructure.
Yes, which bet? I agree. There's no bet here. The comment correctly outlines why the US is doing this and how it benefits. And instead of concluding that "this is why they're doing it", they're making some weird hint that they're shorting the Euro and that somehow this doesn't pay off if the trade isn't favourable or something else? They benefit by BoJ not selling US treasuries. It doesn't matter how EUR is valued against JPY. It's not a bet between these two currencies. Europe would also benefit by their currency getting weaker, since it's filled with export-heavy countries like Germany.
reply