Worth underlining that designers work very hard to understand the needs of the particular situation.
Usually type designers consider the legibility of 3, 6, 8, 9, 0 (particularly 8 and 0) to be more important than between O and 0. But for coders, the ambiguity between O and 0 is a big problem, so a designer would consider that.
An example for pilots: you are heading 180 and radio it as "one zero eight". Even if you immediately correct yourself, it's a problem.
What does it mean to have less active parameters (36B) than the full model size (132B) and what impact does that have on memory and latency? It seems like this is because it is an MoE model?
The mixture of experts is kinda like a team and a manager. So the manager and one or two of the team go to work depending on the input, not the entire team.
So in this analogy, each team member and the manager has a certain number of params. The whole team is 132B. The manager and team members running for the specific input add up to 36B. Those will load into memory.
Means that it’s a mixture of experts model with 132B parameters in total, but a subset of 36B parameters are used / selected in each forward pass, depending on the context. The parameters not used / selected for generating a particular token belong to “experts” that were deemed not very good at predicting the next token in the current context, but could be used / selected e.g. for the next token.
That way, at inference-time you get the speed of 36B params because you are only "using" 36B params at a time, but the next token might (and frequently does) need a different set of experts than the one before it. If that new set of experts is already loaded (ie you preloaded them into GPU VRAM with the full 132B params), there's no overhead, and you just keep running at 36B speed irrespective of the loaded experts.
You could theoretically load in 36B at a time, but you would be severely bottlenecked by having to reload those 36B params, potentially for every new token! Even on top of the line consumer GPUs that would slow you down to ~seconds per token instead of tokens per second :)
I'm about to give up on London and move to San Francisco. Both cities are experiencing 10%+ house price inflation and all the pain that goes with it. New York has experienced much the same since the 2008 housing crash. Cities are in, suburbs are out. So I have trouble buying the hypothesis that London's increase is due to its use as a quasi-currency or foreign demand. The demand to live in London and other world cities is common and organic. I don't doubt the UK's retrogressive tax policies encourage more investment than normal but that's not the centre of the demand, nor are Eric Schmidt or Michael Bloomberg legitimately competing with me for housing.
I work in London but couldn't imagine living there - not because of the cost but because it feels so claustrophobic. I live in the Suffolk countryside, and am grateful every evening when on the train that I can escape. Cambridge is close enough to provide most of the amenities of London but on a human scale.
I'm glad you can afford it. We'd like to move to Cambridge but the house prices are even more than where we are in the South of England. We're having to think of somewhere further north or less sexy...
I'm pretty sure timthorn means he's living (not in Cambridge, but) somewhere in Suffolk, and can go to Cambridge for shopping, cultural events, etc., in the same sort of way as someone nearer London might go to London.
Cambridge itself has really expensive housing, not as bad as London but still very bad; think San Francisco as opposed to New York. But there are plenty of places within reasonable reach of Cambridge that are much more affordable. I live in a nice village about 7 miles from the centre of Cambridge, and prices there are 2-3 times cheaper than in the centre; move further out to somewhere that's not so much "one of the villages around Cambridge" and prices go down by another 1.5-2x.
Look back 30-40 years:
* Places like the Lower East Side of Manhattan were magnets for broke artists because they were dirt cheap places to live.
* San Francisco neighborhoods like SOMA were similarly inexpensive. A few people with meager incomes could go in together and rent a warehouse to live in.
* London had numerous squats: housing stock so unvalued that it wasn't worth stopping people just moving in for free.
Think about property values in those places today.
In the past 60 years we've witnessed a remarkable cycle. First you had the "white flight" to the suburbs starting in the late 1950s or so. By 1980 the general consensus seemed to be that central cities were doomed forever to poverty. Now the trend is reversed and people want to pour back in. However, the regional population is much larger than when this cycle started so central housing has become a scarce resource.
The article is mostly about forces particular to the London market though. It is amazing how high property values have gone there. I live in San Francisco so I think I'm fairly accustomed to high prices, but visiting London and seeing mews houses going for £3m still blew my mind. Another peculiarity of the UK market is that a lot of property trading are just leaseholds, so you don't even really own the house after you "buy" it. (There are "freeholds" for sale as well, but that's even more expensive) Lovely city, but it does make you poor.
Actually it totally cycles. Availability of cars as well as a government policy encouraging home ownership (cheap loans) created a shift of people moving out to suburbs. If you're really familiar with the biggest cities, you can see this shift over time and it happens with subways and commuter rail lines too. Certain areas of New York's outer-boroughs have seen large shifts in class/demographic over the past 200 years. Coney Island used to be a resort/hotel town for the mega rich. With cars and the completion of major highways, the rich moved out to Westchester and Long Island. The subway eventually reached Coney Island and it became a poor neighborhood. It is now starting the swing back the other way because of new construction and the convenience of the Q line. The biggest shift happened right around when the subway lines were first connected at Times Square and the companies merged.
EFF continues awful tradition of confusing personal digital liberty advocacy with lame complaining about not getting things that cost money for "free".
This map and many others include digital/creative agencies. They may be startups in the sense of new companies, but if we accept that definition why not include every new lawyer, restaurant, or retailer? Agencies are contractors and they don't own the value they build in a product. I hate to be bitchy and don't begrudge agencies but I think they're fundamentally different to tech startups.
For all this article's many faults, at least it is long-form and has more than two sources. I hope PandoDaily continues to support long-form journalism and I hope they inspire other tech blogs to do the same.
What exactly is misleading given there are several common algorithms used (cubic by Linux and compound by windows) and the gains are at least an aggregate 2x improvement on key performance metrics.
One thing I do see is that it hasn't yet been tested on real networks, only synthetic networks. It will be interesting to see if it provides real gains across the range of conditions, particularly in hyper optimized data centers.
It's a difference in approach that is tied to time: suburban American houses were mostly built after forced air central heating, and in that kind of heating insulation is "less" important (the air mass is mixed and heated thoroughly, not the structure). Europeans still use radiators which means insulation is more important (a radiator next to an outside wall is radiating half of its heat toward the outside wall). But fundamentally it seems like a US apathy toward waste more than an improvement.
SF has historically shoddy construction due to the gold rush, mail order housing bought sight unseen, and a mild climate.
If I built a house in the US I would have it designed by a European.
Amount of head on lager varies from country to country but it is hard to argue with a Czech given that they invented Pilsen lager. A good head has a different taste and adds a different texture to the beer and makes the carbonation (high in a lager) more smooth and less bitter.
Usually type designers consider the legibility of 3, 6, 8, 9, 0 (particularly 8 and 0) to be more important than between O and 0. But for coders, the ambiguity between O and 0 is a big problem, so a designer would consider that.
An example for pilots: you are heading 180 and radio it as "one zero eight". Even if you immediately correct yourself, it's a problem.