No.2534
Does anyone here actually believe improvements in artificial intelligence will lead to a runaway process ending in artificial super-intelligence (aka the singularity), and if so, can you explain why the feedback effect of an AI improving itself has to be runaway rather than simply lead to diminishing returns? Is it about AI's speed advantage and innate breadth of knowledge?
You could, I suppose, argue that if smart humans can do something, then a smart AGI could do it too, and faster. Therefore if smart humans could eventually create super-intelligence, and smart AGI is successfully created, the rest of the staircase gets climbed much quicker in something resembling a runaway process. But I think people who argue this assume artificial super-intelligence is possible in the first place. What if it's simply not possible, or at least not possible with techniques derived from current advances? How can people be so sure there isn't a hard wall not so far away?
Despite being intelligent, and despite arguably harboring low-level super-intelligence among us in the form of the rare genius, we haven't been able to improve our own brains at all… at best we have some blunt hammers in the form of psychiatric drugs, and we don't even know how some of them work.
No.2536
>>2534I believe in the approaching singularity, and I never stopped to question whether or not there was simply a wall. It's an interesting thought. I guess if there is a wall, we'll find out soon enough.
I hope somebody gives you a better answer. Good luck!
No.2539
>>2538What about distributing itself over a botnet? No one computer has to host it all.
Maybe I lack imagination but I don't think AI escape is going to be much of an issue in practice, even if we reach some level of AGI. We already have malware and the reason it doesn't overtake everything is because there are people working on both sides with the same tools. The same model that escapes and scans for vulnerabilities is going to be used in the very same way to plug the holes.
No.2541
>>2539>AI escape isn't an issueUntil 5 computers in the botnet holding key resources are unreachable for whatever reason. Botnets are specifically used for DDOS and other less sophisticated attacks because even if some are turned off, the mass of them is still dangerous. An AI is not an automated script, it needs all it's resources to act together, not to mention the Vram and high storage needs. whereas a botnet is designed to open a webpage and hold resources, which is possible on most computers. Very different design capabilities. Have you looked at the full size on open LLMs? They're gigantic. A real AGI would be probably a lot larger.
Also WAN transfer speeds are not fast. You better hope this proto-indo skynet doesn't get turned off before transfers happen lol
No.2578
>actually believe improvements in artificial intelligence will lead to a runaway process
I don't.
You can't quantify intelligence in terms that allow for "runaway superintelligence" because information processing would then be a form of work that requires power. You could theoretically make the energy conversion more efficient but this can still only lead to diminishing returns as you reach maximal efficiency.
No.2579
>>2539The bigger the concept of the system gets the less concepts like "access" make sense. Once your system-concept becomes too permissive you've essentially just wrapped around to claiming society (or some aspect of it, like technocapital) is already a superintelligence and that theory has been dated nonsense since the dot com crash.
Singularitarianism in all its forms is a postmodern ideology for a world that has already moved on. The regressiveness of the tech industry is the perfect example. These ideas originate in science fiction that was only ever meant as a speculative illustration of simpler theo-philosophical concepts, like simulation theory being equivalent to descartes' hypothetical demon that can completely manipulate reality to deceive you. This isn't even a modern idea at all.
No.2608
>>2534tl;dr no because Large Language Models (LLMs - Please use this term in the future) have already improved as much as they can through using all human-produced data that has ever been made publicly available on the internet and using synthetic data has an effect similar to photocopying a photocopy.
Ask this question again in 100 years or so when science finally figures out synthetic consciousness and/or philosophy has figured out The Hard Problem.
No.2620
>>2608>using synthetic data has an effect similar to photocopying a photocopyWhat about RLVR? Sure, it only works in verifiable domains, but math and code are pretty verifiable and you don't need anything else to get to RSI
and doom.
And it's not even obvious that we're out of pretraining data already, and there might be further architectural improvements invented that can squeeze more out of existing datasets.
No.2621
>>2534There are physical limits right now, like memory and storage and power etc. The solution for creating smarter models has generally been "make them bigger" and "train them longer" at a high level, and while there are a lot of changes at the model architecture level, the fundamental approach is still the same. So these practical boundaries will probably exist.
>>2608This isn't true, people said this a year ago and models currently are a lot better than they were then too. More expensive to run too, but there have certainly been improvements. They are particularly useful in cases like mathematics or programming where it is (relatively) easy to check if what you have written is correct, I don't know how well AI works in domains outside of these but time and time again we say, "AI will never be able to do this" and then it does it. But at least for now, it seems like these are the cases where AI does the best, it makes sense to me that it doesn't help as much outside of the sciences yet.
No.2625
>>2608>The Hard Problemwhether or not the chatbot experiences qualia is kind of irrelevant to the quality of its output, innit? The point of the Hard Problem is that you can't distinguish them anyway.
No.2626
Do the main AI companies have strategies for improving their LLMs that aren't "make it bigger"? I am skeptical we'll even see more than incremental improvement at this point. I could be wrong
