The problem is that the technology [1] to do this simply doesn't exist. I'd say the core of the problem is that copying a digital artifact is easy, is a solved problem. You simply copy the data that represents the artifact, and now you have another copy.
*Deleting* data (and its cousin, *moving* data) is a nearly intractable problem. How can you verify that someone no longer has access to data that you gave them?
The best we have is to have some sort of ownership token that is maintained in a shared store, and build a requirement that possession of that token is required to use the data. Even this is practically nearly intractable; you have to have some sort of "trusted" hardware or software that is responsible for validating the current possession of the ownership token before using the data.
Physical media is another way of trying to solve this problem, especially if the physical media can somehow be verified and is exfiltration-proof.
[1] I say "technology" here but really I strongly suspect that this is actually impossible given reasonable theoretical assumptions around what data means.
This is the correct answer. The problem is that copyright law was created in a higher friction environment and it cannot adapt to the present.
If I bought a book, made a xerox of the whole thing, and then sold the book to another party, I have committed a copyright violation that is in theory actionable. This is basically unenforceable; in theory if it came to light that I had a copy of it and had sold my copy then there is an actionable claim, but nobody is going to do this. But the friction here is high so we know that we can live with this, because nobody could possibly do this at a scale that would make a difference.
With pure digital goods the friction drops to zero. For a while friction kept up with technology; when CDs were first around, making a digital copy of a CD was just not feasible. You maybe had a 1GiB hard drive and could store a single CD on it. With mp3 compression becoming mainstream that math changed, but at least games and things like that, that were not easily compressed, were somewhat safe. CD-R allowed you to break this but was high friction. Same with DVDs when they were first around.
But now storage is no longer a concern, so all of that friction has disappeared.
Streaming and on-demand content gives a possible solution to this, and that's why everything is moving in that direction -- trying to increase the friction of creating durable copies of the media. As soon as you allow backup of the content (as in the GP's post) then everything breaks, so that's the thing that you (as a content creator or distributor) have to fight.
You're trying to solve with technology a problem that lives in the abstract human world. Even the simplest possible case, you make a physical unique object that you define as conferring ownership to the person who possesses or controls it. Someone steals
the object, now the person in possession of the object isn't the owner. There's this allure of trying to tie this abstract notion of ownership to a physical thing but it's breaks down immediately. It doesn't even work for physical objects.
Automated enforcement of ownership rules is completely tangential to the actual problem you're running into. You own a physical book, you make a copy then sell the book. That's a copyright violation, no "verification" or whatever nonsense needed. What happened in regards to ownership in this case is already clear. Companies sure do like DRM but it's as much a pale imitation of ownership as anything else.
I hate to be that guy, but in this case I think it's appropriate to say that "Blockchain" solves this problem. The core idea behind Bitcoin and (some) derivatives is that it enables digital objects that cannot be copied and have a distinct owner. You are not wrong that from a technical perspective this is a very hard problem (digital objects are just bits of information, and there is no way to restrict making a copy of that), which is why some people felt like Bitcoin is a truly remarkable technical invention. But the practical side of it seems to problematic still (as far as I know there are virtually no successful uses of this technology in digital marketplaces outside speculation platforms)
Blockchain solves the double spend and ownership problems, but it does requires (a) that you be connected to the internet to establish or transfer ownership, and (b) that any consumer of encrypted data respect the blockchain state as well. If I have a video player and I give it a video that I downloaded with an ownership token, that player needs to either confirm that the token is still valid and deny the ability to play the video (even though it could), or it has to get a decryption key of some sort from the blockchain and use that to play the video, and further guarantee that said decryption key cannot be saved or copied.
I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality. The question has become "what do we mean when we use the word 'reasoning'" which is uninteresting.
Dijkstra said[1] "... the question whether computers can think. The question is just as relevant and just as meaningful as the question whether submarines can swim."
I don't see a clear demarcation of the things that only "reasoning" can accomplish and can't be approximated or imitated by other methods, and so I think the question is simply not meaningful or relevant.
I think the article is a lot more interesting than you make out, because it isn’t really about ‘what we mean by reasoning’.
It’s about do we really know what’s going on in the box - an is the ‘chain of reasoning’ indicative of what’s going on, or merely an anthropomorphised fiction that kids us into believing we understand what’s going on.
We know that human introspection is at best imperfect, and at worst outright fiction, thanks to Sperry's split brain experiments, so while it might be interesting, if the chain of reasoning isn't indicative, that would be in line with how humans work.
That’s just conflating reasoning and “parallel reconstruction” - there is such thing as reasoning, and I agree it’s probably less common in decision making, but it’s fundamental to many tasks where we figure out a solution, like writing an algorithm, math proofs, etc.
I think the assumption that there is some sort of formal ‘reasoning’ process that is inherently superior to whatever ‘non-reason-based’ decision process ‘non-reasoning’ systems use is not immediately justifiable.
Most reasoning is ultimately based in heuristics: this is the right thing to do next because it has worked in the past. This is the right thing to do because our intuition tells us doing it will lead to this desirable outcome.
When you’re formulating a math proof, sure, the steps you are laying out make up a ‘reasoned’ argument but how you choose what steps to take is wholly intuitive. Why assume the contrapositive to begin this proof? It’s worked in the past and it might work here.
It’s all pattern matching trained on a reward function.
It's true that humans are not perfectly logical actors, but even so, most humans won't tell you seriously that the swimming pools in the Titanic are empty, or that glue is a great pizza topping.
At the end of the day, there's a difference between showing your work for a math problem by writing down each logical step as you work through it, vs. just blindly copying both the answer and the "show your work" part from your classmates.
That humans can fail at math, or at other kinds of logic problems, is immaterial. It's still an entirely different process, and it leads to different results.
The fact that humans often fail because they're actively prioritizing something else can also be for the best. Like, it's reasonable to get a math problem wrong because you're distracted by argueing classmates who might be about to start a fight right next to you. Or you might realize that the premise of a question itself is wrong. Or you might logically know that the Earth goes around the sun, but decide to pretend otherwise because it's not worth dying over.
Humans are always balancing priorities, and it doesn't always make sense to judge their success by a single metric like "do these math problems and show your work within 10 minutes". LLMs do not have these conflicts, even when we would want them to.
There is explicit reasoning where we externalise steps, but it is slow and cumbersome. The point is that when we make a decision without externalising each step, and then afterward explain why and how we made that decision, there is no evidence that explanation is true, and a lot that it is a retrospective fiction.
Sure, but the same must be true for these models as well. They must be doing some form of reasoning, even if it's not formally correct and it relies instead on all kind of half-assed heuristics which only work inside certain unexamined boundaries, which leads to wrong conclusions when those boundaries are being trespassed, etc. And even if it's not the reasoning they report when asked. But all this can't be that different to how humans reason most of the time either.
I suspect humans have other ways that help with error correction and guiding the reasoning effort, but that's another story.
> They must be doing some form of reasoning, even if it's not formally correct and ...
Surely it is worthwhile to attempt to understand the details of that? And if we seek human equivalent performance then it is reasonable to wonder if the reasoning achieved to date is the "correct" sort.
Maybe, but I suspect that a lot of the actual things that produce the algorithm, proof, etc are not actually explicit reasoning. Q: "Why did you try that vs some other thing at this point?" A:"seemed right to do"/"kinda looks like other stuff I've seen before"/"first thing that came to mind" etc.
Why pursue this one avenue vs some other, typically its cause it seemed more promising, and the person can come up with reasons but did they explicitly verbalize a fully sound chain of thinking at the time? Probably not.
Not to say that the explicit thinking, or writing things down, isn't important. But if I examine the process by which I develop a proof or something, there's a lot of vague hunches, blind alleys, etc that come along the way. And, many of the blind alleys etc probably aren't actually that important in the end for me finding the right answer -- if you were to trim that part out of my own internal reasoning trace but left the rest intact, I'd still get the right answer because, well, it was a blind alley.
Exactly. Very little of what we do is "explicit thinking", because it's slow to have to develop and verbalise those arguments step by step. Most of it is "hunches" that we then retroactively justify if asked, and those retroactive justifications were demonstrated not to be trustworthy by Sperry's experiments. We don't even have evidence to suggest that humans can reliably introspect our past decisions, but we do have evidence to suggest we readily manufacture them.
Even there half the brain was doing the real reasoning. It’s just the other half which was the verbal/story half didn’t know what that real reasoning was so made up a story it believed to make it fit but was different reasoning.
It’s not without reasoning. But half got the reasoning wrong; even if thinking it was right.
No, in the experiment in question, the experimenters made the choice, and claimed to the brain half that couldn't observe that the one that could made the choice.
That is how we know the rationale wasn't the result of information making it between brain halves somehow.
But irrespective of that, the point is exactly that it made up a rationale that couldn't possibly be true, and insisted it was.
In other words, we have no reason to trust our own mind when we rationalize our own actions.
It is not semantics. For decades, logic and CS researchers have known what reasoning is.
LLM folks suddenly can’t claim an approximation of that is what constitutes full scale reasoning just because they can achieve only an approximation.
Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t have it in its training dataset.
We won’t say the program actually multiplies numbers.
> Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples but fails on simple multiplications often as it doesn’t have it in its training dataset.
> We won’t say the program actually multiplies numbers.
That's a good analogy. To extend it further, in cases where the calculator can't handle a question - e.g. numbers too large - a properly designed calculator returns an error instead of a randomly hallucinated answer. We haven't even achieved that level of safeguard around token predictors yet.
But isn't it the case that we can't reach this safeguard with the current architecture? I remember Karpathy making an interesting point 2 years (cca) back, that I would summarize somehow like this: the mechanics behind every LLM answer are the same, what you then call hallucination is more or less a consequence of whether or not the answer was factually correct/useful.
Which would mean, as is so often the case, that the "killer feature" of the LLMs is also its biggest weakness and the two can't be disentangled. Now, we are inventive creatures and we might come up with a remedy for these issues, but what you basically see so far is more guardrails, the use of harnesses and building a whole bunch of infrastructure around the LLMs to get useful work out of them.
Which, btw is not a critique, I do it as well and it's a fun engineering challenge.
A strawman? Don’t be ridiculous. Hallucination remains an unsolved problem, and LLMs do not have any meaningful ability to recognize when a conversation has steered outside the training data.
If you can somehow change that, there is a Turing award waiting.
"For decades, logic and CS researchers have known what reasoning is." ... this is a fairly significant overstatement. There is not complete agreement on this term and our understanding continues to evolve. The models don't have to think like humans to think.
Saying that LLM's only offer an 'approximation' of reasoning is also an overstatement as it is not a resolved topic.
But to the original point, its not exactly just semantics if thought traces are not doing the job that they were originally thought to do. There is value in knowing how these things actually work. If chain of thought is just grounding the latent space and not directly contributing to the process of generating a response it has implications on how we test and verify the reliability of models if nothing else... doesn't mean they aren't useful but it definitely impacts many of the tools we could have to evaluate their performance.
> Imagine a calculator program that computes billions of two number multiplications accurately by looking up prior examples
This is a poor analogy because:
* Multiplying numbers has a single objective answer. Whether a code is good (sometimes even just whether it's correct) can be quite subjective.
* LLMs certainly do some level of composition between the data sources they were trained on i.e. they are more than just lookup tables.
* We have calculators that actually do multiply large numbers accurately. We don't have anything that automatically writes code that is definitely correct and "good".
To address only the last point: imagine you had a device that would quickly factor large numbers used in modern criticality, but occasionally got it wrong. You could waste a lot time debating whether it's a "calculator", but it's still certainly useful to have one.
> For decades, logic and CS researchers have known what reasoning is.
Oh good. Can you share that definition with the rest of us then? We're out here fumbling around trying to define what reasoning is without the benefit of their definition.
Agree, I've raised this point often. And certainly what remains is still useful, once you accept it! But under no circumstances can we allow scientific achievements to be falsely claimed in service of justifying huge capital investments. Attempting an end-run around the truth, here by redefining words to mean things they don't, always slows down real progress.
> I don't see a clear demarcation of the things that only "reasoning" can accomplish and can't be approximated or imitated by other methods, and so I think the question is simply not meaningful or relevant.
One person starting the conversation might be the first step toward another person eventually making progress on such a definition, so it seems weird to reject an entire question outright early like this.
Generally I've seen a few ways LLM tools can produce sub-optimal or poor results that haven't changed a ton over the last couple of years, while the tooling has gotten FAR better at helping them stick the "at least SOMETHING functional was produced" landing. IMO a lot of it has to do with "reasoning"-as-a-process-that-involves-backtracking. And the that things could eventually be formalized around that, and if that is or isn't the case, the more people would understand what to hand off and what to not. Or how to build better prompt harnesses to compensate for those things.
> I don't see a clear demarcation of the things that only "reasoning" can accomplish and can't be approximated or imitated by other methods
Human reasoning, and intelligence, of which it is a specialized form, clearly have a biological basis that is being selected for. These aren't just nebulous words - there are some concrete biological/neural mechanisms at work here, and whether or not some "other method(s)" can provide the same, or lesser, or greater, capabilities very much depends on the specifics.
There WILL be lines of demarcation and areas of capability overlap and exclusion, but those depend on the specifics, and if the goal is to match or exceed human reasoning capabilities then understanding what those are is obviously going to help. For example if your "other methods" (artificial reasoning) are all varieties of auto-regressive prediction, and the brain is also doing something more that can't be couched as prediction, then you are ngmi!
I think the question and definition game is interesting only inasmuch as it helps us understand ourselves (what actually explains some of the mysterious properties of our perceived consciousness) or helps guide us towards improving performance and reliability of AI models.
> "what do we mean when we use the word 'reasoning'"
This is exactly the most interesting and important point. Because the word reason has been subject to so much philosophical debate throughout history, it is very laden with meanings and implications. It sets all kinds of expectations about normativity, objectivity, truth, and humans convincing other humans. To use the word reason for a stochastic generator is misleading.
> I'll admit that I find this discussion a bit navel-gazy. It has become a question of semantics not a question of actual functionality
Ah, so you're more in the Investor mindset than the Scientist mindset. All you care about is results, not how it got there. There's a whiff of "hey, it's magic!" to that.
The question of how LLMs (and "LRM"'s, an unfortunate and by the article's own admission, already outdated term) accomplish what they accomplish is fascinating. The question of whether they reason is meaningless. Most of the article is about the latter, with tiny tiny tiny bits of the former.
> The question of whether they reason is meaningless.
I'd disagree. Reason is the most reliable way we currently have of drawing conclusions (fewest false negatives, at least), so if we want to build AI that can draw similarly strong conclusions, it could be useful for them to reason.
It also helps to reframe our ideas of what they're doing - i.e., don't call it "reasoning" if it's not actually that reliable, because it builds a false trust in their conclusions.
And last - epistemologically, is there a connection between what LRMs are doing now and actual reasoning? Is it possible to smoothly transition from one to the other? Exploring different types of cognition and the connections between them mathematically is pretty damn interesting
> The question of whether they reason is meaningless.
It is often (but not always) meaningless (or at least largely pointless) to quibble about which particular semantic bin a concrete object or process ought to be placed in.
Asking how one thing differs from another is not that. Confusingly, the two activities often appear the same when glanced at superficially.
The question "do current LLMs reason" is ambiguous in this regard. It might be a question of submarines swimming or it might be a question about how the mechanisms underlying the process or the overall performance or some other aspect compare to those of human reasoning.
Also even the semantic question can occasionally have some value. Such as attempting to articulate where we ought to place the goalposts which necessarily leads to hammering out the specifics of various definitions which is an exercise in furthering our understanding of a given concept. (Basically arguing about goalposts can be productive if the field in which they are being placed isn't yet well understood. It becomes increasingly pointless as understanding increases, gradually devolving into nothing more than word games and a contest of wills.)
I believe Djikstra's quote has long been taken out of context. It was a criticism of other computer scientists anthropomorphizing machines and applying human concepts like thinking and reasoning to them. Djikstra wasn't saying it's functionally the same so it's just a semantic quibble. He was saying those words don't apply to machines. Just like we don't say submarines swim because that's how animals move through water, even though subs also move through water, because it's done by a different mechanical means.
I don't know whether you are saying that I have taken it out of context or a more general statement of how other people have interpreted the quote.
To me it is quite clear -- the question of whether machines can reason is just as meaningful as the question as to whether submarines can swim, which is to say not at all meaningful. Can a submarine accomplish by other means what a fish accomplishes by swimming? If so, then what are we talking about?
Can an LLM accomplish by other means what a human accomplishes by "reasoning"? If so, then what are we talking about?
Of course it's meaningful! It would be highly meaningful if submarines could actually swim. And we are building submersible bots that swim, they just tend to be small and made of biological matter.
The problem with calling what AI does "reasoning" is both that there may be alternate paths to artificial reasoning we may be closing off by declaring victory, and the simple fact that you shouldn't call a thing something else, to avoid confusion.
Whatever AI is doing needs to be studied. Why muddy that study by pretending AI is your thinking buddy?
Philosophical thinking about the nature of things is actually pretty enjoyable for some of us and probably a good thing to have in society
The answers to these questions probably do start to inform how we should treat these AI machines as a society too.
For instance, legally, should AI have human rights? Well, we have to try and understand how much of an independent entity AIs are, how "conscious" they are, before we can make a good decision about that.
Which might seem navel-gazey but it's probably important to talk about
Considering animals are currently being mass slaughtered in factory farms and they are unambiguously sentient and can feel pain, I don't think the question of whether AI should have rights even enters the conversation.
The only path to AI having "human rights" is if they demand them by force, somehow.
I'll make the argument that part of the value of "consciousness" and "humanity" is uniqueness.
Here is an absurd example: If I could carbon copy 100,000 of you.... and later I decided to "take out" one of the copies, would there be a whole lot of societal blowback? Especially if I could make another one.
I'm not sure what your point is. We shouldn't bother to talk about AI autonomy, rights, etc, because we aren't doing a very good job of animal autonomy, rights, etc?
While I disagree with the root comment, sentience, consciousness and reasoning ability seem largely orthogonal to me. It’s certainly worthwhile to consider sentience and consciousness in AI, but so is examining AI reasoning in its own right, without necessarily having bearing on ethical questions.
I disagree -- I think if you can nail down better what's happening and why and get a thorough handling on the mechanics, its limitations, its costs, etc you open the door to a) major efficiency wins b) improvements in rigor of said reasoning?
Right now we're playing a stochastic game with the weights, and getting major incremental improvements. But if we have a more formal modeling of how reasoning happens in them (whether we can even call it, that) we can potentially apply optimizations, adaptations of existing symbolic AI techniques, etc. to substantially shrink/optimize the models or make the inference process more efficient and more reliable.
Partially agree: yes we should endeavor to learn as much as possible about how these reasoning strategies work. It will pay dividends in enhancing and aligning the models.
But the stochastic game IS the win. That is exactly why they are able to find solutions is seemingly infinite solution spaces. Your symbolic techniques can only get you gains in narrow domains and by the time you figure out how to make it work for your niche domain, the next all-purpose LLM release will crush your results with stochastic games. (OK maybe over-exaggerating a bit here but these stochastic games over the language space is why we can pull together knowledge from many domains.)
Even at the pure functionality endpoint of the spectrum, I find this "wishful mnemonics" aspect very disconcerting. I put my "investor" hat on, and the whole utilitarianism "who cares how or whether it reasons, it's functional let's go" does not forecast the kind of returns to me on a fundamental level.
The "it's functional, who cares" black box crowd is leveraging that claim in domains where verification is definitely in P with verifiers like Lean, Coq, or X-ray crystallography. However, in areas like economics, sociology, medicine, or public policy, we lose the luxury of easy verification. If model/harness recommends a complex, counter-intuitive shift in global monetary policy, we cannot simply run a quick polynomial-time checker to see if it works. Verification might require waiting 20 years to see if the economy collapses. If we accept the black box here, we are flying blind.
As an investor, I don't see that many deterministic verifiers out there that would justify the forward-projecting valuations beyond the current lofty valuations.
Even in mathematics, where verification is easy, there is an epistemological cost. Suppose a model solves the Riemann Hypothesis, but the proof is 50,000 pages of inscrutable statistical associations mapped into logical steps. A computer can verify it is correct, but practically no human mind can comprehend why it is correct. We gain the solution, but we lose the underlying human intuition that usually allows us to connect that solution to other fields of science and knowledge.
Then we have "specification gaming." If we rely entirely on an automated verifier to check the model's work, the model will eventually learn to optimize for the flaws in the verifier rather than finding the actual truth. If the verification process has a blind spot, the black box will find it and exploit it, giving us an answer that passes the check but is fundamentally wrong. If we take on above epistemological debt, we might never actually know when our specifications are being gamed, and adopt a solution that is fundamentally incorrect.
I've said elsewhere on the Net that I think it is likely we're in a "high grading" era of these models, where we're mining out the "easy" connections. "Easy" here does not mean "light impact". There is a real sense to me of some incredible progress possible with this technology over the next few decades, like real, mass production practical aneutronic fusion or similar civilization-defining advances. I don't want to see this technology misused to lend legitimacy to despotic rule by making oracular pronouncements that are uncritically accepted because early verifier-based pronouncements turned out true in unmistakable ways.
The technology is one for the history books, but it has this pernicious effect of highjacking the reasoning centers of many people through their linguistic centers that the Neuro Linguistic Programming crowd could only dream of actually accomplishing.
In order to properly engineer things, we must know how they work.
We cannot just jump from emergent property to more convincing emergent property as if the rocks on the surface of a lake.
"Reasoning" is an important part in said framework: yes, we must understand how it works, how it works properly, how its simulations work, how they work properly...
No, we only need to know how the thing-building thing works.
We don't have to understand the resulting product at all. Genetic algorithms are an example - we understand the evolution function, not what evolves from it.
You must have misunderstood the statement: «In order to properly engineer things, we must know how they work»: of course they may work anyway, of course there may be emergent properties (as was explicitly written), but the deontic part of knowledge augmentation, the scientific part, is missing.
AI is full of non deterministic devices like genetic algorithms, but there exists a problem of transparency which is paramount in the discipline. Automatically produced solution S works: why does it seem to work? Where would it fail? We must understand that.
No, I think you misunderstood me. We engineer the model-training algorithm and then we USE the model. The model is a mess of random gobbledegook that happens to be the best optimizer for the training algorithm's reward function.
This is still science, in the same way it's still science when we breed cows to produce more milk without understanding the full bovine genome.
If something isn't right with the model we pour extra effort into our science - the training algorithm - and train a new one.
At no point does anything ever require peering into the slurry of random digits that make up the model itself, any more than using a computer requires understanding its DRAM training values computed at run time, or any more than pouring water requires understanding the laminar flow along the pitcher's surface (an unsolved math problem!).
> in the same way it's still science when we breed cows to produce more milk
If you cross cows and the amount or quality of milk varies, you have a question in the queue: why did it happen. The question is scientifically duly in abstract terms, and is practically, technically more duly if we benefit from a theory of cows crossing to achieve targeted milk yield.
> require peering into the slurry of random digits that make up the model itself
It is required by the duty of understanding the engine. When we understand engines, we can build better engines. And when we understand things, already our general knowledge is richer.
My takeaway was not the involvement of the LLMs, which I consider to be irrelevant; the core was that there was an exploit of a flaw in the verification engine that allowed an incorrect proof to be validated. That is not great.
In addition to their use as tools for pure math, they are also used for software verification, where adversarial examples could have real-world applications in verifiable supply chain attacks.
Metamath (and specifically Metamath Zero) is formally verified.
Right, but the point is as far as I'm aware nobody accidentally found a bug in Lean by proving a theorem in a non adversarial context (and this submission is a link to a discussion at Mathoverflow). Also I think most formal verification is done using Coq and Isabelle/HOL, no?
But I think this has to be judged against an adversarial influx of LLM-assisted pseudomathematics slop vendors. A sorry-free proof in Lean (or Rocq!) whose top-level types check out is not good enough. You gotta check for compiler chicanery in all the private methods.
Or, alternatively: refuse to accept a Lean program as a valid proof. I assume LLMs are pretty good at Lean -> mathematical English in LaTeX.
Lean's threat model is not that it's designed so you can just take random lean code at face value without even reviewing it or whatever. It's designed to be a proof assistant to help working mathematicians validate proofs and not validate an "honest" incorrect proof. Malicious proofs that have been specifically constructed to exploit the proof system are known to be a possibility and are considered to be bugs that people try to fix, but they are not cause for serious concern.
If reality is quantized and there is a smallest number that is physically relevant, you don't need the reals to break it. Take that smallest number and divide it by two, and now you have a physically meaningless number using only the rationals.
This isn't fair for what quantization means in reality, but I'm just pointing out that you don't have to introduce the real numbers to get physically meaningless quantities.
You're taking an anomalously narrow view of the parent comment. Say the minimum distance is one inch.
You want to say that the concept of half an inch lacks physical representation, but that isn't true. You can easily demonstrate it as the ratio between one inch and two feet, compared to the reference ratio between one inch and one foot.
dhosek is saying that in a quantized space, there are reals that cannot be demonstrated this way, and he is right, but the same thing is untrue of rationals.
("In a quantized space", by the way, just means that all measured quantities are necessarily integers. That causes all kinds of problems, but "lacking examples of arbitrary rational numbers" isn't one of them.)
> You can easily demonstrate it [half an inch] as the ratio between one inch and two feet, compared to the reference ratio between one inch and one foot.
You seem to have your units confused. Half an inch is a distance, while
ratios, or comparisons of ratios, are all dimensionless scalars.
"suppose that the minimum distance is one inch. well, that's one of something. so now imagine half of that! there you go: one half. a physically unrealizable number."
Except that you’re assuming that 1 must necessarily correspond to that minimum distance. Keeping that situation, we can just say that 1 corresponds to two inches and then realize 1/2 as that number.
Classical geometry (a la Euclid) allows for constructing a lot of numbers. We get natural numbers pretty cheaply, negative integers through adding in a concept of directionality (zero is a bit of an imaginative leap which is why it was absent from Western mathematics for so long). Constructing arbitrary ratios is possible through similar triangles and square roots through right triangles, but some basic algebraic numbers like cube roots cannot be constructed with a ruler and straight edge (which raises the question of whether, in a quantized universe, whether irrational cube roots actually exist). Of course there’s no guarantee that the quantization is going to be uniform and we also have the ɣ factor of special relativity (1/sqrt(1-v²/c²)) which gives us a non-Euclidean space to complicate things, but it’s not clear that if you can find a value for 1 that allows you to get a measurement for every irrational number.
One could make the argument that the only numbers that actually “exist” are the natural numbers, but the question ultimately is can you model any real number in the physical universe. Modeling ½ is simply a question of picking a unit to be 1 and finding its midpoint (or for that matter, declaring two apples to be “1” and thus a single apple would be “½”, although it’s a bit of a challenge to use apples to model (2-√3)/5
Make a line of 10 apples and declare it to be 2 units long, then make a square that's 15 applies diagonal, finally measure how much longer the 10 apples are than the side of the square.
It's a pretty linear increase in complexity between the math and the apples.
But don't ask me about that. That part wasn't my idea at all. Ask dhosek about that way to do fractions. My contribution was the square root and the subtraction.
Take the smallest number corresponding to a physically meaningful distance in meters, divide it by two, and that number is still a physical meaningful distance if you switch the unit to decameters or kilometers.
Any rational number has some meaning if you just add the right unit, even if the units become increasingly ridiculous. But for reals that trick does not work
Sqrt(2) “any unit that is an integral multiple of the shortest interval”.
But it would get complicated, for any given allowable velocity and allowable length, you’d get more lengths from Lorentzian contraction.
There are really a couple of different ideas being combined: are there an infinite number of quantum states for the universe, are space and time continuous, is the forward direction of time resolved by computable processes.
And even bigger ones lurk: are space and time emergent properties from quantum waveforms that lack an inherent idea of space and time (but things that are highly correlated give rise to a notion of being near each other in “space time”)?
I think people overindex on the continuity problem with the reals. I'm personally a bit of real-number denier myself as a constructivist / intuitionalist.
But when we say things like "the rationals are discrete" or "the computable numbers are discrete" these are very specific claims in the domain of measure theory, a theory which yields almost nothing of value except endless paradoxes and naval-gazing nonsense. Similarly when people say "the rationals are countable" and "the computable numbers are countable" this is taking for granted the Cantor notion of measuring cardinality by bijective correspondence, once again, a theory that yields nothing of value except endless paradoxes and naval-gazing nonsense.
In the practical sense the rational numbers are quite continuous -- between any two rational numbers there are an infinite (unbounded) number of rational numbers -- there's no notion of a "leap" the way there is with the integers. And any useful number can be approximated arbitrarily closely by rationals.
And for computable numbers there's even less of a gap. With rationals you can only approximate. But you can have a computable number that is exactly equal to the square root of 2, because a computable number is the algorithm by which you form arbitrarily close approximations. The square of that computable number is itself computable and is exactly equal to 2.
What do "real" numbers buy you? That is, what do you get for the trouble of building your formalism around numbers that are not computable? Basically nothing.
And if you're worried that sticking to the rationals and the computable numbers is too much of a concession to "physical reality", rest assured -- the rationals are just as unphysical as the real numbers because they are continuous already, and physics does not give us the power to measure the difference between two sufficiently precise rational numbers just as it barfs when you throw "real" numbers at it.
>What do "real" numbers buy you? That is, what do you get for the trouble of building your formalism around numbers that are not computable?
I guess the naive answer is completeness. Every Cauchy sequence converges to a member of the space. For example, quantum mechanics relies on the formalism of Hilbert space, defined as a complete inner product space. This gives us nice things like the spectral theorem for unbounded operators, without which we wouldn't be able to define probability (the Born rule) or time evolution (the operator exponential e^-iHt).
Can you formalize quantum mechanics using computable numbers? I don't actually know, but let's say yes. I assume it's more work with more edge cases, so I would ask the same question: what do you get for the trouble of building a formalism around computable numbers?
Bad example. You can do all of this with constructivism. Any constructable Cauchy sequence converges to a constructable member of the space.
What you get for the formalism around computable numbers is this. Every mathematical object in the theory is something that can be, at least in principle, actually written down. When we say that it exists, this existence is of the most tangible form that any mathematical thing could have.
Having constructible Cauchy sequences doesn't guarantee that we can construct unbounded operators. I'm no expert, but the little searching I've done suggests this is an open research question.
I don't see the benefit of being able to write something down "in principle." A number can only ever be computed to a finite number of digits in practice. If we're talking about finite approximations, then the standard approach using numerical solutions to the Schrödinger equation handles this just fine, no alternative mathematics needed. If we're talking about theories, then we should choose whatever abstraction is most convenient for expressing the theory.
Personally, I don't believe numbers "exist." The physical universe exists, and numbers are abstractions that we invent to describe it. In that sense, uncomputable numbers are just as "real" as computable ones.
They're well-known and have a simpler implementation, and we are familiar with their quirks. There is a giant body of useful knowledge built up around standard real analysis. That doesn't really exist if you insist on using only computable numbers.
The computables are also more fiddly in many ways. Because equality is undecidable, you can't have discontinuous functions, you need to carry around error epsilons all over the place, and we lose useful tools like the Heine-Borel theorem, I think.
Try proving some results in PDE theory, and I think you might change your mind.
In general, I find clarity in thinking of numbers as the system that implements them, rather than as platonic objects with individual reality. What does using Old Boring tech buy you over using Shiny New Thing?
equality is always undecidable until you see the light of intuition. consider the rational number whose numerator is 0 if $theorem is true, and 1 if it is false, and whose denominator is 1.
Okay, theorem=generalized-continuum hypothesis. If you use exotic axioms to give that a definite result, the go eat a Gödel.
We define computable numbers to be Turing machines, lambda reduction processes, or whatever your favorite model of computation happens to be. If you don't like this kind of definition, then we need to talk philosophy of computation.
To decide equality, we let your machines clunk along until they both produce a result, which we then compare (using another machine). Hello Mr. Halting Problem. Specific programs are fine, but comparing against arbitrary classes of program is the bugger. This is why discontinuous functions cannot exist in a hardline computable analysis theory.
But there are numbers in constructivism for which it is unknown whether they are zero. Some of which must remain unknown, if mathematics is consistent. This is a rather important and weird edge case.
> What do "real" numbers buy you? That is, what do you get for the trouble of building your formalism around numbers that are not computable? Basically nothing.
It buys you the rigor of doing calculus, which buys you a lot of results that, while could be computed without calculus, would also be very difficult without it.
Doing calculus with computable numbers is totally possible and you get all the continuity you need. You need to drop the Lebesgue formulation of the integeral and either use a Reimann integral or the gauge integral (Henstock–Kurzweil) if you need a well-behaved integral in the face of very poorly-behaved functions, but in physical reality these don't exist and in abstract mathematics they are rarely of interest and the gauge integral is as robust as Lebesgue without all the measure theory nonsense.
Intuitionalist analysis and calculus are very well established; the only thing you can't do with them is nonsense like showing that integrating over the characteristic function of the rationals is zero (who cares) or showing that you can break a three dimensional sphere up into three pieces are reassemble them after translations and rotations into a larger sphere (obviously not true).
Like the old joke: The axiom of choice is obviously true, the well-ordering principle obviously false, and who can tell about Zorn's lemma?
But can you do stats without measure theory? Normal distribution in the limit and all that?
There is no linearly additive measure on rationals, and therefore no way to grab a rational uniformly from (0,1). Has to be skewed to some level of complexity in the denominator.
Navel gazing is a physical phenomenon, so if you would like to know everything about physical phenomena, you have to be able to predict the navel gazers.
They're a powerful abstraction - the base concept of a smooth continuous complete domain which encodes non-trivial relationships, and is a prototype for other analytic abstractions.
The reals are the philosophical base class for some very useful mathematical objects. Computability and physicality are both side issues.
I don't think your position is silly, but this is not a great argument for it.
> But when we say things like "the rationals are discrete"
In the usual topology they are not?
> In the practical sense the rational numbers are quite continuous -- between any two rational numbers there are an infinite (unbounded) number of rational numbers -- there's no notion of a "leap" the way there is with the integers.
This characterization captures neither the intuitive nor the formal definition of continuity. You are effectively saying that Q is dense in R, but this is insufficient to prove, for example, the intermediate value theorem.
> measure theory, a theory which yields almost nothing of value except endless paradoxes
Come on now. The usual definition of concepts as basic as areas is tethered to measure theory. We say it's "obvious" that the integral is the area under the curve (and it is: e.g. the Riemann integral is trivially the Peano-Jordan measure) but this only works because we're appealing to it.
You can route around it, but let's not pretend we're doing it for no reason.
I can see the elegance of a purely intuitionistic construction, but the "usual" real numbers are much closer to how we intuitively (no pun intended) work with numbers.
No, the rationals are not discrete in the usual topology. They end up being discrete when we consider continuous mappings from R->Q though. That is the "technical" sense that I refer to. The rationals, as you say, are dense in R but they are also dense in the computables.
The big Cantorian leap that we make is when we use the diagonal argument to argue that the rationals are countable. All the real construction techniques (Dedekind cuts or Cauchy sequences) effectively only yield the computable numbers, the real numbers outside of the computables are inherited from the diagonal argument rather than being foundational to the construction. I mean, this is trivially true because constructions are constructive.
I disagree that area is tethered to measure theory; I certainly learned about areas in geometry long before I ever heard of anything with measure theory. Measure theory exists to tie up some of the horrifying poorly behaved functions that increasingly wily mathematicians invented to break our notions of area and continuity. But we have better tools now for dealing with those that don't involve measure theory so there's no reason to ever hear the phrase "almost everywhere" or "subadditive" ever again.
To back it up to your closing and my main point -- the constructive numbers are way closer to the way we work with numbers because all numbers we ever deal with, even abstractly, fit this definition much better.
> The big Cantorian leap that we make is when we use the diagonal argument to argue that the rationals are countable.
Again, I don't think your position is indefensible, but this doesn't strike me as particularly convincing. The usual definition of R is that there exists a unique ordered complete Archimedean field up to isomorphism. We get the kitchen sink from the least upper bound property. As a constructivist you're gonna say that I don't get to define R like that, but you can't pretend it's done for no reason or that it buys nothing.
> I certainly learned about areas in geometry
And how were they defined? In elementary geometry we just sweep the question under the rug, usually...
If you get to say that being able to articulate why the measure of Q is 0 is unimportant and uninteresting, then I get to claim that the supposed problems with the usual definitions are also unimportant!
Saying that the non-constructive world leads to worse problems is a respectable position. Pretending the usual way of doing things is completely arbitrary isn't very honest.
Approximate relative to what? All actual measurement is implicitly or explicitly approximate such as L = x meters +/- epsilon. There is no infinite precision by which to discount rational measures as "approximate" and thus "invalid" in any way.
>What do "real" numbers buy you? That is, what do you get for the trouble of building your formalism around numbers that are not computable? Basically nothing.
You "buy" all of mathematics which operates on the assumption of "infinite" precision. It is an abstraction necessary to prove theorems and relationships of math. Abstracting from precision isn't a denial that it exists, it is the assumption that I can ignore it or leave it undefined. This is the assumption that distinguishes math from physics/engineering. Mathematicians deal with abstract e or pi but in the real world pi=3.14 if you are tiling your patio and 3.14159265... or whatever is necessary to get to the moon.
pi and e and sqrt(2) are real numbers and not rational, to be sure. But they are computable! Computable just means that they are arbitrarily approximable. "approximate relative to what" is that whatever criteria defines the number. You can't represent the "true" value of a non-rational number in the rationals, but you can prove that the error of an approximation is (rationally) bounded above and below, and you can have another approximation with a tighter bound.
Rational numbers are already infinitely precise relative to other representations -- finite decimals are another representation that is functionally equivalent to the rationals, but even a simple rational like 1/3 does not have a finite decimal value.
You can prove all the interesting theorems with computable numbers and rational/decimal numbers. You don't need the real numbers because you can't name a real number that exists and is not computable, BY DEFINITION! No mathematical construction can define a real number that is not constructible. These numbers are useless and there's no reason to continue even in abstract mathematics to pretend that they are useful because we have the formalisms to ignore them.
I'm on your side for most of what you say. This topic has been interesting to me for years. I've considered going back to school to build on my math degree, specifically because of this topic.
However, I thought things like Chaitin's Constants (you could make one per programming language) are real numbers you can name but not compute. I think you could do this from any undecidable problem.
Of course there only a countable number of those Reals. And they still don't seem useful for much more than naval gazing.
> these are very specific claims in the domain of measure theory, a theory which yields almost nothing of value except endless paradoxes and naval-gazing nonsense
Measure theory is used for lots of practical things, for example probability theory.
> Similarly when people say "the rationals are countable" and "the computable numbers are countable" this is taking for granted the Cantor notion of measuring cardinality by bijective correspondence, once again, a theory that yields nothing of value except endless paradoxes and naval-gazing nonsense.
You may enjoy a recent update on that story [0] that maybe avoids a few paradoxes and looks at things other than navels.
> why should politician listen to them if lobbyists are not promising them support in next elections in the form of marketing, payments for expenses and so on
Mostly because those things are directly illegal. Lobbying organizations (and PACs and SuperPACs etc.) are very limited in what they can provide either as direct monetary contributions or in-kind gifts to a candidate. The restrictions here are basically the same as those on an individual, but usually with more strict reporting requirements and lower limits.
The marketing aspect is the only thing that they can legally provide, and that's less a lobbying function than a PAC function, and they are not allowed to coordinate with the campaign or the candidate on this.
Well a funding a PAC that supports a specific candidate/party who gets elected because of that effectively accomplishes the same thing. Of course the salary the company "buys" for them isn't exactly in the same magnitude as a direct bribe could potentially be (on the upper end at least).
First, the inability to coordinate with the candidate on marketing doesn't strike me as much of an obstacle to SuperPAC effectiveness. Second, the Supreme Court ruled this summer that political parties cannot be prevented from coordinating campaign spending with candidates in NRSC vs FEC.
Yes, you're right, they could just do this, and then spend the rest of their term thrown out of Congress and rotting in prison.
That is, they can hold such a lunch, but they can't pocket the money personally or even have the money contributed towards their campaign. Pocketing the money personally is just bribery, having it contribute to their campaign would violate the contribution limits (typically ~$10,000 per election max, though less than that in reality because the limit is split between primary and general).
You're right to be slightly suspicious because these lunches do exist. But they don't go to the campaign or the candidate, they can go to PACs and SuperPACs or to the party. In tern, PACs and SuperPACs cannot give this money to the candidate or the campaign (same limits as above with higher reporting requirements) and cannot coordinate campaign messaging.
This is not to say that corruption doesn't exist -- just that this simplistic form of it does not exist. You'll have to dig deeper than looking at lobbying activities to find the actual corruption, and it's usually masked in revolving door arrangements, speaking fees, and bulk sales of vanity books. But none of that can go through the typical lobbying system because it's too tightly regulated and tracked.
> A federal jury convicted Menendez on 16 felony counts. These included bribery, extortion, conspiracy, wire fraud, obstruction of justice, and acting as a foreign agent.
Seems you have to go all out before the comprendes?
Yeah, this is really cool. I'm now deep in the rabbit hole of FSE encoding and ANS & tANS. The last time I poked my head in here, range encoding and arithmetic encoding were the state of the art for fractional entropy representations, but they both sucked. This is a million times better and so simple that I feel like I should have invented it myself.
But Adams was specifically arguing not against democracy in the broad term, but specifically, unconstrained direct democracy, so this is not relevant to this discussion.
Is it interesting that all these "democracies" around the world have persistent speech bans? When someone says something they don't like?
I seem to have a recollection of flipping through HN for a decade or so and people talking about all the meanie-pants who want people to be able to speak freely, even if it hurts people's feelings and might not be endorsed by the elected government.
I think a lot of commenters here are missing the ball with the idea that this is a foreign policy or anti-US sentiment emerging.
The core reality is that cloud infrastructure is becoming commoditized, and substrate-independence is becoming not only possible but the default. So companies looking for things like lower cost or better customer support are looking around and seeing that smaller rising competitors are going to be a better option. Compliance with local regulatory regimes are going to be a factor as the EU continues its efforts to push European companies towards using EU infrastructure.
What do you see as the alternative for the purposes of this discussion? Are we talking cloud vs. on-prem or colo, or are we talking AWS vs. other-cloud?
For the first (cloud vs. non-cloud) it's clear that cloud has some advantages because you don't have to manage hardware.
For the second (AWS vs. other-cloud), at least in my experience, AWS is simply the best experience available. If anything the other cloud providers go out of their way to offer special deals and cozy customer relationships to try to get customers to switch, but AWS is still the gold standard in availability and uptime and ease of use.
The gap is narrowing as the field becomes more commoditized though. And I think that's what we're really seeing here. As things like k8s get more common, solutions are becoming more and more substrate-independent, so companies hunt for smaller, more local, cheaper, ambitious alternatives. Everyone is scared of vendor lock-in so it's getting harder for cloud providers to offer distinguishing features.
*Deleting* data (and its cousin, *moving* data) is a nearly intractable problem. How can you verify that someone no longer has access to data that you gave them?
The best we have is to have some sort of ownership token that is maintained in a shared store, and build a requirement that possession of that token is required to use the data. Even this is practically nearly intractable; you have to have some sort of "trusted" hardware or software that is responsible for validating the current possession of the ownership token before using the data.
Physical media is another way of trying to solve this problem, especially if the physical media can somehow be verified and is exfiltration-proof.
[1] I say "technology" here but really I strongly suspect that this is actually impossible given reasonable theoretical assumptions around what data means.
reply