Good, we as a society aren’t ready for these kind of tools. AI would further increase the divide between people. One of the reasons is that it costs too much to run it.
Everyone who can’t afford the hardware would be dependent on AI owned by corporations. And most people can’t even afford those fees. Since we build our society around (materialistic) “productivity”, I am sure AI would escalate how we treat people and whole countries who fall off the capitalism train. I hope the hype dies.
Ok, but AI isn’t going away. So if these companies stop serving open access, the ONLY people that will use them will be the people who can afford the server/processing time.
This article isn’t about usefulness of the models to normal people. It’s about profitability of the models to the corporations that serve them.
Okay… let’s call wine “wine” and bread “bread”: the acronym “AI” is mostly an advertisement stunt.
This is not artificial intelligence; and even if it was, “AI” is used for a bag of a thousand cats - game mob pathfinding, chess engines, swarm heuristic methods, so goes on.
What the article is talking about is far more specific, it’s what the industry calls "machine learning"¹.
So the text is saying that machine learning is costly. No surprise - it’s a relatively new tech, and even the state of art is still damn coarse². Over time those technologies will get further refined, under different models; cost of operation is bound to reduce over time. Microsoft and the likes aren’t playing the short game, they’re looking for long-term return of investment.
I’d go a step further and claim here that “model-based generation” is more accurate. But that’s me.
LLMs are a good example of that; GPT-4 has ~210¹² parameters. If you equate each to a neuron (kind of a sloppy comparison, but whatever), it’s more than an order of magnitude larger than the ~110¹¹ neurons in a human brain. It’s basically brute force.
The comparison of GPT parameters to neurons really is kinda sloppy, since they’re not at all comparable. To start with, “parameters” encompasses both weights (ie. the “importance” of a connection between any two neurons) and biases (sort of the starting value of an individual neuron, which then biases the activation function) so it doesn’t tell you anything about the number of neurons, and secondly biological neurons have way more dynamic behavior than what current “static” NNs like GPT use so it wouldn’t really be surprising if you needed much more of them to mimic the behavior of meatbag neurons. Also, LLM architecture is incredibly weird so the whole concept of neurons isn’t as relevant as it is in more traditional networks (although they do have neurons in their layers)
Another sloppiness that I didn’t mention is that a lot of human neurons are there for things that have nothing to do with either reasoning or language; making your heart beat, transmitting pain, so goes on. However I think that the comparison is still useful in this context - it shows how big those LLMs are, even in comparison with a system created out of messy natural selection. The process behind the LLMs seems inefficient.
I wouldn’t discount natural selection as messy. The reason why LLMs are as inefficient as they are in comparison to their complexity is exactly because they were designed by us meatbags; evolutionary processes can result in some astonishingly efficient solutions, although by no means “perfect”. I’ve done research in evolutionary computation and while it does have its problems – results can be unpredictable, it’s ridiculously hard to design a good fitness function, designing a “digital DNA” that mimics the best parts of actual DNA is nontrivial to say the least etc etc – I think it might be at least part of the solution to building, or rather growing, better neural networks / AI architectures.
It’s less about “discounting” it and more about acknowledging that the human brain is not so efficient as people might think. As such, LLMs using an order of magnitude more parameters than the number of cells in a brain hints that LLMs are far less efficient than language models could be.
I’m aware that evolutionary algorithms can yield useful results.
But the point is that not only is the human brain actually remarkably efficient for what it is, and that you’re still confusing parameter count and neuron count. The parameter count is essentially the number of connections between neurons plus the count of neurons in a network.
If I recall correctly the average human brain has something like 80 billion neurons, and each neuron can have anywhere from 1 000 to 10 000 connections. This means that in neural net technology terms, we meatbags have brains with trillions of parameters. I just meant that it wouldn’t be surprising if an “artifial brain” needed more neurons to do (a part of) the same thing as our brains do since they’re vastly simpler
But ML is being used in the industry in tons of places, and it’s definitely cost effective. There’s simple models that take the input of machinery sensors and detect when something is faulty or needs repairing, not just malfunctioning parts but worn out parts too. It’s used heavily in image processing, tiktok is used by a lot of people and the silly AR thingies use image recognition and tracking in real time through your phone. I’m not saying that this features created revenue directly, but they do get viral and attract users, so yeah. Image processing is also used in almost any supermarket to control the amount of people in the store, at least since covid I see a dynamically updated counter in every supermarket I visit.
It is also used in time estimations, how much traffic influences a trip is not manually set at all, it gets updated dynamically with real time data, through trained models.
It is also used in language models, for real usages like translation, recommendation engines (search engines, store product recommendation…).
The article is talking about generative models, more specifically, text prediction engines (ChatGPT, Copilot). ChatGPT is a chatbot, and I don’t see a good way to monetise it while keeping it free to use, and Copilot is a silly idea to me as a programmer since it feels very dangerous and not very practical. And again, not something I would pay for, so meh.
But (+> contradiction) ML is being used in the industry in tons of places […] store product recommendation…).
By context it’s rather clear which type of machine learning I’m talking about, given the OP. I’m not talking about the simple models that you’re talking about and that, as you said, already found economically viable applications.
Past that “it’s generative models, not machine learning” is on the same level as “it’s a cat, not a mammal”. One is a subset of the other, and by calling it “machine learning” my point was to highlight that it is not a “toothed and furry chicken” as the term AI implies.
By context it’s rather clear which type of machine learning I’m talking about
Eh, it was to you and me, but we are not in a specialised community. This is a general one about technology, and since people tend to misunderstand stuff I prefer to specify. I get that you then wrote footnote #1, but why write statements like this one:
So the text is saying that machine learning is costly. No surprise - it’s a relatively new tech, and even the state of art is still damn coarse²
I know which branch of ML you are talking about, but in written form on a public forum that people might use as a reference, I’d prefer to be more specific. Yeah you then mention LLMs as an example, but the new ones are basically those, there’s several branches with plenty maturity.
“it’s generative models, not machine learning”
IDK why you are quoting me on that, I never said that. I’d just want people to specify more. I only mentioned several branches of machine learning, and generative models are one of them.
Also, what’s that about contradiction? In the first paragraph I was mentioning the machinery industry, since I talk about machines. Then I talked about language models and some of their applications, I don’t get why that contradicts anything. Store product recommendations are done with supervised ML models that track your clicks, views, and past purchases to generate an interest model about you, and it’s combined with the purchases people with similar likes that you do do to generate a recommendation list. This is ML too.
Dunno, you read as quite angry, misquoting me and all.
Rather than debate every new technology for energy worthiness on a case-by-case basis up front, it would be more productive to direct our efforts towards better policy that internalizes the cost of pollution.
“These new-fangled ‘lasers’ don’t even do anything useful for the energy they consume!”
TBH, 26.3 billion dollars are just a drop in the bucket for Google. That bucket of course filled with the money they got with industrial scale spying, cross-site tracking, denial of control, forced ads, destruction of competition, among countless dirty tricks they play on regular netizens.
Yet, in a redacted copy of an internal email chain released on Friday, Jim Kolotouros, the vice president of Android Platform Partnerships, wrote: “Chrome exists to serve Google search, and if it cannot do that because it is regulated to be set by the user, the value of users using Chrome goes to almost zero (for me).”
So Chrome’s whole point is bringing users to Google Search… and Google Search’s whole point is Google Ads. I’m Glad I use Firefox.
Dunno about “the last update” or the current state in each region but as far as I know the default search engine in FireFox has varied over the years and has always depended what country you’re in.
Baidu, Yandex and Yahoo are / have been the default in some countries. They made Bing the default for “1%” of users in a bunch of major countries recently to test the waters (and didn’t take it further than that).
Google blocks traffic from Chinese IP addresses as a protest against censorship there, so nobody has Google as the default in that country.
Well, you really can’t in a traditional sense. This isnt a victory screen, it’s a kill screen. He got so far into the game it crashed and you can’t continue. There are still more goals that can potentially be reached higher by avoiding the crash.
I just learned about some of the manual techniques pro players have come up with to play Tetris at a high level. It’s not my thing, but along with speedrunning, the level of community-driven ingenuity is inspiring.
I’m using Qwant and it gives me better results than Google. Even Startpage does and it’s using Google behind the scenes.
Google managed to fuck up their personalization so much it makes the results worse (it’s almost like they only really care about tailoring the ads /s). And I’m suspecting it’s by design, if the results suck the users are more likely to either press the ads or go through more result pages, therefore seeing more ads.
I agree it does suck in general One thing I tried is using a metasearch engine and for the least part I find the results better and way more customizable (for reference I am self hosting searxNG)
same, just being able to blacklist content farms I dislike is worth the price (standing up a container) of admission, but there’s plenty more good things.
I am actually kinda ok with DDG, but the results are.. not always very great and the second page is filled with weird websites related to my location..
Maybe i should try both Kagi and Searx
TL;DR: They became partners with Brave, when people pointed out Brave’s objectionable politics they responded… poorly (still read the thread, it has much more info)
Quick edit: Was pointed at this, looks like it’s even worse and the Kagi owner himself might be very objectionable too
Yikes, thanks so much for these links. I’ve avoided kagi for awhile now for lack of need but it had been top of list of ‘things I should try’. Guess I can strike that one out now.
eh, ddg is equally bad with it insisting it knows better than me what my query is and “fixing” it, leaving me to have to either fix it or click a link telling it “yes I really did want to search for that and not what you assumed”
gizmodo.com
Oldest