Say you've got a process that spans a bunch of services and touches the real world. Some ecommerce check out for example.
You need to reserve some inventory, charge a card, eventually email a customer and steps can go wrong. So you have queues, and retries and ways to back things out, undo changes.
To my understanding, Temporal's idea is to factor that part out, the queues and retries, and offsetting actions, so you write the logic and not the workflow orchestration.
For me, I'm pretty sure it's related to having just awful seasonal allergies as a child - I'd quite often get stuffed up and have an unfortunate amount of pressure in my sinuses and ears. So then I'd pinch my nose and try to adjust it, eventually realizing I didn't need to pinch my nose.
I scrolled the whole list looking to see if there was anything about being able to stop hiccups by conscious effort alone. Suspect that there would be some overlap between voluntary tube openers and hiccup stoppers, as the internal sensations are kind of similar.
There was a popular (then, anyway) notion that drinking a large glass of water would kill hiccups, and it sometimes seemed to work, but other times it didn't. Long story short, it works when you swallow at just the right time relative to the hiccup cycle. An ordinary swallow starts as a voluntary action but switches over to an automatic involuntary completion. Try to get that start of the involuntary part to happen in the last quarter of the hiccup cycle, a bit before the hiccup arrives. You don't even need to use water, just time a swallow just right and it cancels out the hiccup cycle from retriggering.
My mum taught me the ear popping thing, while we were traveling by air, to relieve the ear pressure.
I can do it on command but never paid attention to how before.
It's basically "yawning manually". There's a couple different muscles activating at the same time, most irrelevant to the popping, I can't control them individually yet. (Probably learnable, if I practiced I could probably get the pop more efficiently.)
But yeah, just yawn real wide, that should do it.
A lot of stuff is learnable, like I couldn't move my ears but taught myself how to do it because I was bored.
For me I describe it more as trying to open your jaw with the muscles near your ear, while holding the jaw in place with the main jaw muscles. Like an isometric hold.
I actually didn't know I could do it until now, I could always get the click and it's how I clear ear popping sensations but I didn't realize it I push it harder I get the hollow voice thing and apparently it's useful for diving. Cool.
I'm not a diver but I suspect it can be learned. I have allergy issues that cause my ears to not equalize when allergies are hitting and I think I just learned how to do this. its just sort of muscle that can sort of be lightly controlled. Controlling one side vs the other I can't really manage.
That might be a reason, but probably more relevant is the fact that when you want solar to provide all power for a data center, you need a lot of buffering capacity (batteries). And then you also need a back up for when the sun doesn't shine... Which is probably a gas generator.
AI training, not inference, seems like it would actually be a good application for a solar powered DC because you could just checkpoint and pause when it gets cloudy. It might end up worth it for the free electricity.
It doesn’t make sense to do that, hardware deprecates in 5years (it used to be 3) and is currently very expensive. So by using this approach you double the average cost of hardware per unit of compute completed.
If there wasn’t such a hardware squeeze and electricity was a higher percentage of the cost, yes, it would have made sense.
They should definitely use wind power, since you could use the fans to cool the computers at the same time!
I for one would love a nice HTTP status in the form of "due to the weather being nice and calm outside, AI services are down, go outside and enjoy the day"
It's a writer perspective versus a reader perspective maybe?
Sometimes when you're trying to write something, it really seems like the exact words matter a lot. Suggestions made to be more direct or use a more common word here or whatever seem to really impact the thought that you're trying to communicate.
Certainly we've all had times when trying to communicate clearly when the specific words seem very important.
I tried to reproduce those results, at least in terms of compression ratios, not speed.
However I would say that testing on alice29, enwiki8, text8 data is kinda cheating. Alice in Wonderland and Wikipedia are very likely part of the training data of the LLM models used there.
So I tried on HN comments from a few days ago, extracted from the text column of the public HN bigquery dataset.
Using RWKV v7 0.1B instead of RWKV v4, I get 0.962 bits per byte on alice29, and 1.156 bits per bytes on the HN comments. Still a lot better than 2.826 bits per bytes of xz level 9.
Oh wow, so it worked pretty well on data it hasn't seen. That expected but cool to reproduce.
Have you seen this leaderboard of sorts[1], and this proposal to change hutter prize[2]?
I think it's a really clever idea that you could measure an LLM's prediction abilities and language understanding by some sort of held-out compression metric because file sizes are very concrete. They are already beating shannon's numbers using a human prediction for compression, from what i can see.
>I think it's a really clever idea that you could measure an LLM's prediction abilities and language understanding by some sort of held-out compression metric
I think it's limited to using base models (non post-trained), because the post-training would skew the logit distribution. There are ways to "coax" post-trained models back into behaving "like" a base model, I wonder if the benchmark could be unofficially updated with those somehow.
Small world. I just did a podcast on this same topic, but coming at it from a different direction, ie. me and my neighbor trying to beat the hutter prize for compression.
Hutter Prize being where you are paid if you can compress wikipedia small enough. LLMs do very well at that, if, big if, you ignore the cost of initial weights.
A cool Claude Shannon story:
Shannon wanted to measure how much information is actually contained in ordinary
English text. His 1948 theory said such a number must exist, but he had no way to
calculate it, because the patterns in English reach across dozens of letters and no
equation or frequency table captures all of them at once.
So instead of calculating it, he ran an experiment on a person.
He took a passage from a novel that the subject had not read, and covered it with a
card so only the text already guessed was visible. He asked the subject to name
the first letter. If the guess was wrong, he asked again, and kept asking until the
subject named the correct letter. He wrote down how many guesses it had taken,
revealed the letter, and moved the card one position to the right. Then he repeated
the process for the next letter, and the next, through the whole passage.
What this produced was not a sequence of letters but a sequence of numbers — one
number per letter, recording how many guesses that letter required. Most of the
numbers were 1, because someone fluent in English, seeing the preceding text,
usually names the next letter correctly on the first attempt.
Shannon then argued that this sequence of numbers contains exactly as much
information as the original passage.
> Hutter Prize being where you are paid if you can compress wikipedia small enough. LLMs do very well at that, if, big if, you ignore the cost of initial weights.
then.
If all you care about is compressing Wikipedia, but ignore the size of the actual data, what is it that you are actually trying to do?
You need to reserve some inventory, charge a card, eventually email a customer and steps can go wrong. So you have queues, and retries and ways to back things out, undo changes.
To my understanding, Temporal's idea is to factor that part out, the queues and retries, and offsetting actions, so you write the logic and not the workflow orchestration.
reply