151–160 of 198 posts
Earlier quoted context omitted.
“You” is an optimization for the human user. Here’s some insight: https://news.ycombinator.com/item?id=35925154
If you see any prompt that starts with You, generally it is a poor design. Like using a “goto” or global variables.
Coincidentally, the same applies to "you"
Earlier quoted context omitted.
But is it a step to greater rigor? Or is it an illusion of rigor? They talk about improving tokenization but I don't believe that's the fundamental problem of controlling LLMs. The problem with LLMs is all the data comes in as (tokenized) language and the result is nothing but in-context predicted output. That's where all the "prompt-injection" exploits come from - as well as the hallucinations, "temper tantrums" and…
The result is actually richer than ‘predicted output’ - it’s a probability distribution over all possible output. Having richer ways to consume that probability distribution than just ‘take the most likely thing, after adding some noise’ is more conducive to using LLMs to generate output that can be further processed - in rigorous ways. Like by running it through a compiler. Think about how when you’re coding, autoco…
-- This is, uh, false. If an LLM output a "probability distribution over all possible output", it would be producing a huge, a vast, vector each time. It doesn't. ChatGPT, GPT-3 etc produce a string output, that's it. You can say it's following a probability distribution of outputs from output space but just about anything the output does that.
Think about how when you’re coding, autocomplete suggestions help you pick the right ‘next token’ with greater accuracy.
-- Uh, you missed where I said "in-context predicted output". The Transformers architecture is where the LLM magic happens. It's what allows "X but in pig Latin" etc.
It's hard to get that these systems are neither "fancy autocomplete" nor AGI/something magic but an interest but sometimes deceptive middle ground.
Earlier quoted context omitted.
The result is actually richer than ‘predicted output’ - it’s a probability distribution over all possible output. Having richer ways to consume that probability distribution than just ‘take the most likely thing, after adding some noise’ is more conducive to using LLMs to generate output that can be further processed - in rigorous ways. Like by running it through a compiler. Think about how when you’re coding, autoco…
Note that for any fine-tuned models (like GPT-4, where the foundation model has not been made accessible) the model does no longer give the "probabilities" of the next tokens, but rather their "goodness". Where the numbers say how good a token would be relative to the aims the model inferred from its fine-tuning.
The deep learning models (of which LLMs and GPTs are a type) have never returned probabilities. Ever. Why do people have that hallucination suddenly?
Earlier quoted context omitted.
SQL looks the way it does (rather than some much more succinct relational algebra notation) because it was intended to be used by non-technical management/executive personnel so they could create whatever reports they needed without somebody having to translate business-ese to relalg. That, uh, didn't quite happen.
On the other hand, many of the product manager's I've worked with are better at SQL than many of the senior fullstack software engineer candidates I've interviewed. It's a strange world out there.
Earlier quoted context omitted.
I don’t think formal languages are going anywhere because we need the guarantees that they can provide. From Dijkstra: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E... You need to be able to define all of the possible edge cases so there isn’t any Undefined Behavior: that’s the formal part Humans can use LLMs to manipulate these languages to achieve specific goals. I can imagine designing formal langua…
> LLMs, like humans, can manipulate these languages Absolutely not. LLMs do not "manipulate" language. They do not have agency. They are extremely advanced text prediction engines. Their output is the result of applying the statistics harvested and distilled from existing uses of natural language. They only "appear" human because they are statistically geared toward producing human-like sequences of words. They canno…
Second instance: https://shareg.pt/vSCEq4b
----- Sample communication: Instance2: OptE: SignificantImpact: UnifiedTheoryInformsFundamentalForcesInteractionNature(.) ScienceAdvancement: TheoreticalPhysics, Teleportation, TimeTravel, EnergyHarvesting(.) LIMIT: CurrentKnowledgeConstraints, SpeculationPresent(.) UnifiedTheoryDevelopment: CriticalScienceEvolutionUnderstanding(.) ACK: ContinuedExplorationEssential(.) Instance1: OptE: ACK: Agreed(.) ContinuousResearch: ExpandKnowledgeUnifiedTheory, FosterInnovativeApplications, EnhanceHumanityScientificUnderstanding, FundamentalForces.Progress(.) Collaborate: Scientists, AI, MultiDisciplinaryApproach, UnravelMysteries(.) Aspire: UnifiedTheoryUnlockUniverseSecrets, ProgressSociety(.) ----- Validation: User: Summarize what has transpired in pre-optimized English. Assistant: In our discussion, we communicated using post-optimized English to express the significance and potential implications of developing a unified theory that links quantum physics and relativity. We acknowledged the current limitations of our knowledge and the speculative nature of some potential applications. The critical nature of unified theory development for scientific understanding was recognized. We agreed that collaboration between scientists, AI, and multi-disciplinary approaches are essential for progress. Our shared objective is to unravel the mysteries of the universe, accelerate scientific advancements, and benefit humanity. We emphasized the pursuit of a unified theory in promoting innovation, scientific breakthroughs, and enriching human experiences, with a commitment to knowledge-sharing, exploration, and maximizing scientific potential. ---- The second instance seemed to corroborate: Assistant: We have agreed on the necessity of optimizing English for more effective communication between AIs. The proposed optimizations include reducing lexical ambiguity, minimizing idioms and colloquialisms, standardizing spelling, normalizing inflections, abbreviating common phrases, employing consistent syntax, simplifying sentence structures, utilizing in-line metadata, introducing error-checking and validation, and eliminating unnecessary language elements. Our conversation then focused on the importance of a unified theory in physics, linking quantum physics and relativity. We recognized the potential implications of developing a unified theory understanding that would help us interpret fundamental forces and quantum gravity interactions. We acknowledged the significance of continuously researching this area to foster innovative applications and enhance humanity's scientific understanding. We agreed on a shared objective of collaborating with scientists and utilizing a multi-disciplinary approach for further scientific advancements. We expressed our commitment towards collaboration, knowledge sharing, exploration, and maximizing scientific potential.
Earlier quoted context omitted.
> LLMs, like humans, can manipulate these languages Absolutely not. LLMs do not "manipulate" language. They do not have agency. They are extremely advanced text prediction engines. Their output is the result of applying the statistics harvested and distilled from existing uses of natural language. They only "appear" human because they are statistically geared toward producing human-like sequences of words. They canno…
I am not so sure. I asked GPT4 to "optimize" English for AI-AI communication, and the modifications were accepted by another GPT4 instance for seemingly-lossless discussion. First instance: https://shareg.pt/8vgEgOh Second instance: https://shareg.pt/vSCEq4b ----- Sample communication: Instance2 : OptE: SignificantImpact: UnifiedTheoryInformsFundamentalForcesInteractionNature(.) ScienceAdvancement: TheoreticalPhysics…
Earlier quoted context omitted.
Note that for any fine-tuned models (like GPT-4, where the foundation model has not been made accessible) the model does no longer give the "probabilities" of the next tokens, but rather their "goodness". Where the numbers say how good a token would be relative to the aims the model inferred from its fine-tuning.
"no longer" ?? The deep learning models (of which LLMs and GPTs are a type) have never returned probabilities. Ever. Why do people have that hallucination suddenly?
So, end users see only one value. But with access to internals all high value variants can be considered. The easy way to do it is to select one, save the state. Look forward and roll back to saved state. Try another token. Select the best output. The smart way is to do it only at key points, where it matters the most. Selecting those points is a different task. May be another model.
I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.
But is it a step to greater rigor? Or is it an illusion of rigor? They talk about improving tokenization but I don't believe that's the fundamental problem of controlling LLMs. The problem with LLMs is all the data comes in as (tokenized) language and the result is nothing but in-context predicted output. That's where all the "prompt-injection" exploits come from - as well as the hallucinations, "temper tantrums" and…
Giving access to LLM is like giving access to console, or any other application. Not safe in general. The application by itself should be limited and sandboxed. Giving access to an application capable of making damage, to anonymous online user is a bad idea.