It really does. I think it's one of the reasons the fediverse has remained a pretty positive place, without having to deal with all the ugliness all the time.
Google search results suck and I’ve actually heard regular people mention it. They just either don’t know how to switch search engines or they think Bing search sucks.
Bing and Bing based searches have also gotten worse. The study in question actually says they preform worse than google. Its all Goodhart's law in action.
Or they still think they are entitled to a “free” search engine and don’t see the amount of resources needed for that and that it’s actually a service worth paying for, either through a subscription or through a donation-based service.
Switching one private company for another is definitely not the way to go…
Or they’re working class or buried in medical bills and can’t afford to be spending money on things like search engines that have a free alternative, even if it is worse.
I’m not actually convinced the alternatives are any better here, anyway.
Careful about how you throw around the word "entitlement". The top competition is free and search engines are very low value for the average person. It's very reasonable to expect search engines to be free and for anything paid to be a niche product. Google search results may be terrible, but not so terrible that I'm going to pay $5/month to escape it.
Subscription services still get worse. The arrogance Cable TV must have to show us ads—cable was the ad-free service back in its day. The same is happening with Netflix. The same will happen with Spotify. This thing is a snake eating it’s own fucking tail.
I want something without perverse incentives. Donations, maybe. Taxes, possibly. I get free roads, why not a free search index.
Amazingly Google is still the best unpaid search engine, as bad as it has become. Terrible websites have completely taken over the web. To find actual information you need access ProQuest or EBESCOhost or something like that, though their indexes are much smaller.
The GitHub copilot example seems to indicate it’s a pricing problem. In fact this situation might indicate that users are finding it so useful that they are using it more than MS expected when they set up their monthly subscriptions. Over time, models are going to be optimized and costs will reduce.
Expecting AI to take over all human intensive tasks is not realistic but eventually it’s going to become part of a lot of repetitive tasks. Though I hope that we see more open source base models instead of the current situation with 3-4 major companies providing the base models behind most of the AI applications.
GitHub Copilot is extremely useful. It also runs pretty much on every key stroke and programmers make a lot of keystrokes throughout the day…
It’s a useful enough tool that people would be willing to pay more, but at the same time it’s not using an advanced AI model. It uses an older (and now deprecated) model that I’m pretty sure a high end computer (even some laptop) could provide similar output without any cloud service, using open source / freely available models.
My feeling is Copilot needs to either lower their price or improve the quality of the product if it’s going to survive. And I suspect they’re going to do the latter one.
The other factor not discussed here is the hardware we use today for this task isn’t really designed for it. The GPUs most datacentres run AI models on are designed for graphics, not AI, and the algorithms mostly just need huge amounts of fast memory. I’m sure soon there will be dedicated hardware specifically designed for large language models with less compute cores and more memory. They’ll likely also run at lower clock speeds and use less power/generate less heat/etc.
Just because companies are losing money right now doesn’t mean they will be in five years time.
Even with the costs implied here, Copilot would be useful as hell.
Think about it, an average (western) developer costs easily 100k/year, sometimes even 2 or 3 times that. Spending something like 1000€ per year makes sense, even if productivity is increased by just one percent.
Okay… let’s call wine “wine” and bread “bread”: the acronym “AI” is mostly an advertisement stunt.
This is not artificial intelligence; and even if it was, “AI” is used for a bag of a thousand cats - game mob pathfinding, chess engines, swarm heuristic methods, so goes on.
What the article is talking about is far more specific, it’s what the industry calls "machine learning"¹.
So the text is saying that machine learning is costly. No surprise - it’s a relatively new tech, and even the state of art is still damn coarse². Over time those technologies will get further refined, under different models; cost of operation is bound to reduce over time. Microsoft and the likes aren’t playing the short game, they’re looking for long-term return of investment.
I’d go a step further and claim here that “model-based generation” is more accurate. But that’s me.
LLMs are a good example of that; GPT-4 has ~210¹² parameters. If you equate each to a neuron (kind of a sloppy comparison, but whatever), it’s more than an order of magnitude larger than the ~110¹¹ neurons in a human brain. It’s basically brute force.
The comparison of GPT parameters to neurons really is kinda sloppy, since they’re not at all comparable. To start with, “parameters” encompasses both weights (ie. the “importance” of a connection between any two neurons) and biases (sort of the starting value of an individual neuron, which then biases the activation function) so it doesn’t tell you anything about the number of neurons, and secondly biological neurons have way more dynamic behavior than what current “static” NNs like GPT use so it wouldn’t really be surprising if you needed much more of them to mimic the behavior of meatbag neurons. Also, LLM architecture is incredibly weird so the whole concept of neurons isn’t as relevant as it is in more traditional networks (although they do have neurons in their layers)
Another sloppiness that I didn’t mention is that a lot of human neurons are there for things that have nothing to do with either reasoning or language; making your heart beat, transmitting pain, so goes on. However I think that the comparison is still useful in this context - it shows how big those LLMs are, even in comparison with a system created out of messy natural selection. The process behind the LLMs seems inefficient.
I wouldn’t discount natural selection as messy. The reason why LLMs are as inefficient as they are in comparison to their complexity is exactly because they were designed by us meatbags; evolutionary processes can result in some astonishingly efficient solutions, although by no means “perfect”. I’ve done research in evolutionary computation and while it does have its problems – results can be unpredictable, it’s ridiculously hard to design a good fitness function, designing a “digital DNA” that mimics the best parts of actual DNA is nontrivial to say the least etc etc – I think it might be at least part of the solution to building, or rather growing, better neural networks / AI architectures.
It’s less about “discounting” it and more about acknowledging that the human brain is not so efficient as people might think. As such, LLMs using an order of magnitude more parameters than the number of cells in a brain hints that LLMs are far less efficient than language models could be.
I’m aware that evolutionary algorithms can yield useful results.
But the point is that not only is the human brain actually remarkably efficient for what it is, and that you’re still confusing parameter count and neuron count. The parameter count is essentially the number of connections between neurons plus the count of neurons in a network.
If I recall correctly the average human brain has something like 80 billion neurons, and each neuron can have anywhere from 1 000 to 10 000 connections. This means that in neural net technology terms, we meatbags have brains with trillions of parameters. I just meant that it wouldn’t be surprising if an “artifial brain” needed more neurons to do (a part of) the same thing as our brains do since they’re vastly simpler
But ML is being used in the industry in tons of places, and it’s definitely cost effective. There’s simple models that take the input of machinery sensors and detect when something is faulty or needs repairing, not just malfunctioning parts but worn out parts too. It’s used heavily in image processing, tiktok is used by a lot of people and the silly AR thingies use image recognition and tracking in real time through your phone. I’m not saying that this features created revenue directly, but they do get viral and attract users, so yeah. Image processing is also used in almost any supermarket to control the amount of people in the store, at least since covid I see a dynamically updated counter in every supermarket I visit.
It is also used in time estimations, how much traffic influences a trip is not manually set at all, it gets updated dynamically with real time data, through trained models.
It is also used in language models, for real usages like translation, recommendation engines (search engines, store product recommendation…).
The article is talking about generative models, more specifically, text prediction engines (ChatGPT, Copilot). ChatGPT is a chatbot, and I don’t see a good way to monetise it while keeping it free to use, and Copilot is a silly idea to me as a programmer since it feels very dangerous and not very practical. And again, not something I would pay for, so meh.
But (+> contradiction) ML is being used in the industry in tons of places […] store product recommendation…).
By context it’s rather clear which type of machine learning I’m talking about, given the OP. I’m not talking about the simple models that you’re talking about and that, as you said, already found economically viable applications.
Past that “it’s generative models, not machine learning” is on the same level as “it’s a cat, not a mammal”. One is a subset of the other, and by calling it “machine learning” my point was to highlight that it is not a “toothed and furry chicken” as the term AI implies.
By context it’s rather clear which type of machine learning I’m talking about
Eh, it was to you and me, but we are not in a specialised community. This is a general one about technology, and since people tend to misunderstand stuff I prefer to specify. I get that you then wrote footnote #1, but why write statements like this one:
So the text is saying that machine learning is costly. No surprise - it’s a relatively new tech, and even the state of art is still damn coarse²
I know which branch of ML you are talking about, but in written form on a public forum that people might use as a reference, I’d prefer to be more specific. Yeah you then mention LLMs as an example, but the new ones are basically those, there’s several branches with plenty maturity.
“it’s generative models, not machine learning”
IDK why you are quoting me on that, I never said that. I’d just want people to specify more. I only mentioned several branches of machine learning, and generative models are one of them.
Also, what’s that about contradiction? In the first paragraph I was mentioning the machinery industry, since I talk about machines. Then I talked about language models and some of their applications, I don’t get why that contradicts anything. Store product recommendations are done with supervised ML models that track your clicks, views, and past purchases to generate an interest model about you, and it’s combined with the purchases people with similar likes that you do do to generate a recommendation list. This is ML too.
Dunno, you read as quite angry, misquoting me and all.
Isn’t Snapchat terrible at security? Heard of someone who made a plane bombing joke in a private message and the airport wifi intercepted it and he got arrested
As someone who has always been skeptical of “AI,” I definitely hope corporations dial back their enthusiasm on it; but I think its value has never been commercial, but industrial.
“AI” was not designed so consumers could see what it would look like to have Abraham Lincoln fighting a T-Rex without having to pay artists for their time. “AI” was designed so that could happen on a much larger enterprise scale (though it would probably be stock images of technology or happy people using technology instead).
With this in mind, I think “AI” being a money pit won’t dissuade corporations since they want the technology to be effective for themselves, they just want consumers to offset costs.
“AI” was not designed so consumers could see what it would look like to have Abraham Lincoln fighting a T-Rex without having to pay artists for their time.
Sure… AI can do that… but it can also be used for “here’s photo of my head with trees in the background, remove the trees”.
I could also do that as a human, but it’d take me hours to do a good job blending my hair into a transparent png without a green tinge. AI can do it in seconds.
Just because a tool can be used to do useless things, doesn’t mean the tool is useless.
Brennan is incredible, and that is an entertaining cast. Going to have to watch this with my wife. Kind of surprised Heidi N Closet isn't joining up. I know she did a one shot with critical role.
Oh that's right she did! I've seen clips of Brennan before and have always wanted to watch, but never wanted another subscribtion... this might make me change my mind.
This is the reason I slept on D20 lol, It's totally worth it plus you get all of the other dropout content. The service is very aware that they're yet another monthly subscription and imo make an effort to continually produce excellent content.
Dropout (via Sam) also explicitly stated around the time Netflix was rumbling about cracking down on account sharing that they're cool with it. So if you can afford to pay for the subscription, that's great -- it helps support some fantastic content and the people who make it. If you can't afford to pay, but have a friend or family member who has it and is willing to share(or are the person in the position to share), that's also good.
It’s a bit ironic. If the Olympics weren’t corrupt, the Russians would have been kicked out wholesale instead of being allowed to compete as independent athletes or whatever.
gizmodo.com
Top