Live data from Hacker News

Viewing profile — hollerith

hollerith

HN member
Joined
Sun, Nov 04, 2007, 5:11 AM UTC
HN karma
8,093
Public activity
6,980 items

About hollerith

Richard Hollerith, San Francisco Bay Area. hruvulum@gmail.com

No one has any plausible plan or halfway-decent plan for how to maintain control over an AI that has become super-humanly capable. Essentially all of the major AI labs are trying to create a super-humanly capable AI, and eventually one of them will probably succeed—which would be very bad (i.e., probably fatal) for humanity. So, clearly the AI labs must be shut down and in fact would have been shut down already if humanity were sufficiently competent. The AI labs must stay shut down until someone comes up with a good plan for controlling (or aligning) AIs, which will probably take at least 3 or 4 decades. We know that because people have been conducting the intellectual search for a plan for more than 2 decades as their full-time job, and those people report that the search is very difficult.

A satisfactory alternative might be to develop a method for determining whether a novel AI design can acquire a dangerous level of capabilities along with some way of ensuring that no lab or group goes ahead with an AI that can. This might be satisfactory if the determination can be made before giving the AI access to people or the internet. But I know of no competent researcher that has ever worked on or made any progress on this problem whereas at least the control problem has received a decent amount of attention from researchers and funding institutions.

The AIs that might prove fatal to humanity will be significantly different in design from the AIs that have been already widely deployed: for one thing, they will constantly learn (like a person does) as opposed to already-deployed AIs in which the vast majority of the AI's learning happens during a training phase that ends before any widespread deployment of the AI. Also, they will be much better than current AIs at working towards a long-term goal. I say this because I don't want to be misunderstood as believing that Google Gemini 2.5 or ChatGPT 5.0 might take over the world: I understand that those AIs are incapable of a such a thing. The worry is the AIs that are still on the drawing board or that will appear on a drawing board 5 or 10 years from now. There is no need to ban Gemini 2.5 and ChatGPT 5. Since some AI researchers pursue AI "progress" for ideological reasons and will tend to persist stubbornly even after AI research is banned, the best time to ban AI frontier research is now, so that these stubborn ideologues whose research will have been driven underground (because of the ban) but are capable of making a little more "progress" on AI will be unlikely to be able to make enough "progress" to end the world.

Again, as soon as anyone comes up with a solid plan for controlling (or aligning) an AI even if the AI turns out to be more capable than us, the ban on frontier AI research can be lifted as long as the majority of AI experts and AI researchers are in agreement that the plan is solid. No one can say how long this search for a solid plan will take, but IMHO it will probably take at least 3 or 4 decades.

More at

https://intelligence.org/the-problem/

Recent public activity

  1. comment
    Comment #49211755

    You wouldn't be worried about constant noise from the chillers?

  2. comment
    Comment #49177649

    Someone conjectured that every organization that doesn't have opposing Leftism as its mission will eventually be run by Leftists.

  3. comment
    Comment #49177462

    Good point, but China's generals were trained by generals with experience in a proper war, which is not nothing. Specifically, Beijing sent 3 million Chinese soldiers to assist Pyo…

  4. comment
    Comment #49177372

    The atmosphere of coastal California (i.e., where most Californians live) has about one tenth the capacity to dilute pollutants as the atmosphere of most of the rest of the US (bec…

  5. comment
    Comment #49177355

    I understand why war with Iran would raise gas prices, but why would war with China?

  6. comment
    Comment #49160742

    >China can at least so far be relied upon to not shoot itself in the foot It can? Have you heard of the Great Leap Forward? The Cultural Revolution? Either of those damaged the met…

  7. comment
    Comment #49150975

    Here's a list of 20 researchers and managers that would need to be fired according to you. My guess is that there are hundreds, possibly thousands, of researchers and managers empl…

  8. comment
    Comment #49146610

    The "paperclips" were never actually paperclips (at least not to the originator of the word-picture, Yudkowsky) but rather tiny molecular squiggles which are a physical manifestati…

  9. comment
    Comment #49139815

    >you can easily improve acoustic performance of drywall in many ways. Double layering, acoustic sealing caulk, rockwall in walls, etc. There's a principle-agent problem here. It is…

  10. comment
    Comment #49129529

    If drywall is better than lath and plaster, then why is lath and plaster still used today -- in high-end construction?

  11. comment
    Comment #49109493

    Maybe this is part of some complicated plan to increase his net worth or maybe he's worried that the AI juggernaut will get him killed (along with everyone else) if it keeps going …

  12. comment
  13. comment
  14. comment
    Comment #49097027

    The person you're quoting is a lawyer.

  15. comment
    Comment #49096545

    For admirers of the Enlightenment, it's more than just a convenient technicality to want to avoid including in the Enlightenment a period a political chaos during which about a mil…

  16. comment
    Comment #49096267

    Today I learned that some historians include the French Revolution in the Enlightenment. Here is the intro to the Wikipedia article: >European historians traditionally dated its be…

  17. comment
    Comment #49070489

    AI hasn't "toppled" (or found any mistakes in) any theorems, at least not any enjoying any consensus among mathematicians.

  18. comment
    Comment #49068601

    I'm not so worried about this part as I am the part where a lab loses control of an AI, and it goes on to pursue whatever meaningless objectives (meaningless to us , that is) its d…

  19. comment
    Comment #49029874

    I don't see any mention of shareholder value or financial performance upstream (unless we count your latest comment). I do see, "their managers are not capable of leading innovatio…

  20. comment
    Comment #49029553

    Apple tried to create a new OS and failed (during Jobs's absence from Apple), bought NeXT, then used NeXT's work as the basis for the very successful OS X. And despite this, in you…

  21. comment
    Comment #49015788

    Most working-class and many middle-class American women made their own clothes until the 1960s. It was too expensive to buy ready-made clothes. A company named Simplicity prospered…

  22. comment
    Comment #48994452

    There was certainly a vast gold rush around the internet with massive attention from almost everybody in journalism or in business and almost every retail investor -- at least in t…

  23. comment
    Comment #48994030

    Sounds like the government (which owned the entire system during Beeching) should have kept at least a right-of-way (if they have that concept in the UK) instead of selling everyth…

  24. comment
    Comment #48987224

    [flagged]

  25. comment
    Comment #48986432

    >appears to actually make people more susceptible to nonsense I think it is more that teenagers and very young adults and people high in the trait of "openness to new experiences" …