wow this is such an excellent framing of something that i have not managed to think about or come across in the vast ocean of discourse about LLMs. i had not really thought about how explicitly they are designed to not have histories that are user-interpretable. it is really quite not unlike how Apple locks down iOS and MacOS-- there are UI/UX reasons for the choices the frontier models make, but that really is only part of the story, and these choices undeniably do conspire to make the resource more locked-down than it absolutely has to be. and the analogy to "people don't switch OSes every day, but the ability to switch OSes changes your relationship with the provider" is extremely apt.
The session you cannot take with you
141–150 of 238 posts
Re: The session you cannot take with you
#142Will https://buzz.xyz/ not help with this a bit?
Not really, to my understanding. You own your buzz instance but it still interfaces with existing models, open or otherwise. The linked issues apply to the model outputs themselves and is orthogonal to what buzz is ostensibly doing.
Re: The session you cannot take with you
#143Earlier quoted context omitted.
It's not even that it's hard to build a modern account system. It's that if you put your site behind a private / custom account system, and you post a link to it on HN (for example), everyone and their mother complains that it's requiring an email address and password. But if you put the same site behind a Google auth, most people wouldn't think twice to click the button.
I worked on the Google account system for a few years. I'd say it's a lot of work to beat it. Not necessarily "hard" for a team with the right skills, but certainly a lot of sweat, blood and tears. A modern account system is expected to have, in rough implementation order: email confirmations, password strength checks, password reset emails, forgot password flows (=advanced ID verification as otherwise this becomes a…
Thank you for this. The whole attitude of "it's just [complex thing we take for granted], how hard could it be?" is exhausting.
Re: The session you cannot take with you
#144BTW the web page uses massive iGPU if you don't have a fast machine.
I came here to complain about something similar, the excessive use of box shadows on both the top and the sides. It made the text horrible to read on mobile, if it not for Firefox's Readability mode.
Re: The session you cannot take with you
#145I don’t see this as a big deal in practice. Conversations contain a bunch of junk anyway, so removing it from the context is usually good. In my repo, I have a notes directory. I ask the AI to write a markdown file with what it learned, what work has been done, and what remains. In the next conversation, I can ask another model to pick it up from there. Sometimes I edit the note first.
Oh, it very much is a big deal. In a setting where users can switch models easily and with no downsides, market forces will give us better and cheaper AI over time. If switching models is painful (as in, losing part of your context), the providers can create vendor lock-in, enshittify the user experience, and drive up cost. Even if there are workarounds right now, those AI providers have every incentive to make freed…
The coding agent I use (Shelley on exe.dev) has a pulldown menu where you choose the LLM when starting a new conversation. There is no lock-in. Conversations are not precious.
Although, I do use ChatGPT more nowadays because their subscriptions work with Shelley. Sometimes I try other models after I hit the weekly cap on the subscription.
Re: The session you cannot take with you
#146I don’t see this as a big deal in practice. Conversations contain a bunch of junk anyway, so removing it from the context is usually good. In my repo, I have a notes directory. I ask the AI to write a markdown file with what it learned, what work has been done, and what remains. In the next conversation, I can ask another model to pick it up from there. Sometimes I edit the note first.
> Conversations contain a bunch of junk anyway This is what brought me around to doing more agentic coding. I set the task, require tests and the strict linting must pass and then leave it to blow smoke up its own ass about what's going on. I see glimpses scrolling past of all the conversational language that used to frustrate me so much when using a chat interface and I can just let it flow on past. I come back when…
Maybe I am still a novice. When I insist on tests, I get plenty of them, but on close review I'll ask "did you handle [obvious and important thing]" and suddenly it's tearing the code apart and building more stuff. It just loves corner cutting.
Re: The session you cannot take with you
#147Will https://buzz.xyz/ not help with this a bit?
Not really, to my understanding. You own your buzz instance but it still interfaces with existing models, open or otherwise. The linked issues apply to the model outputs themselves and is orthogonal to what buzz is ostensibly doing.
Fair point. The core issues will remain but I am hoping buzz will provide a small layer of persistence that will make it easier for me to keep track of many different interactions more easily.
I have only just started with it in the last few hours so I could be massively disappointed but I live in hope.
Re: The session you cannot take with you
#148This is exactly what I'm working on. Very rough yet but it's progressing nicely. https://github.com/pantoniou/fyai The idea is that your session data are what's important, and what you need to keep yourself, using a model similar to git.
This doesn't (and cannot) solve what the post is talking about
Admittedly the documentation is woefully out of date, but I think the premise of the article is how the session data are increasingly move away from the control of the user and more to the control of the model provider.
The questions that the article poses are, and this is the way fyai addresses them:
Inspection: Can the user see what the model saw, what tools did, and what agents told each other?
fyai> Full session log, tool calls, agent invocation, along with their durable state is stored locally in the durable arena and are available for inspection.
Export: Is the session self-contained, apart from ordinary artifacts that can also be downloaded?
fyai> The session is completely self-contained, no artifacts are stored anywhere besides the arena. Not only that sessions are export-able, import-able and transferable to other fyai sessions.
Replay: Can another implementation reconstruct a semantically equivalent context?
fyai> Full canonical and provider dumps can be generated. There is a standard schema, and the format is just YAML. Other tools are free to ingest it, and replay it.
Audit: Can a human explain why the system took an action after the fact?
fyai> Complete conversation logs, mcp session logs, authentication logs + full wire logs are available via fyai log.
Deletion: Can the user identify and remove every server-side copy on which the session depends?
fyai> Obviously fyai cannot help there. But as a business model, FYAI can work with a completely stateless, inference provider where absolutely no logs are kept, beside the ephemeral KV caches of the active sessions.
The rest of the article deals with the woes of the encrypted session content (reasoning and compaction). fyai fully supports their use, but, it is completely capable of operating without their use - you merely pay the cost of not hitting the cache of the provider.
The article then offers action points for portable inference.
This is how fyai answers:
1. The local event log is canonical. Server storage may mirror or accelerate it, but the client can reconstruct the session without dereferencing server IDs.
fyai> This is exactly the model that fyai provides. No server IDs are needed besides reconstructing the provider session that is hitting the cache.
2. Storage is explicit. store: false should be easy, documented, and preferably the default. Features that require retention should say so at the point of use.
fyai> fyai provides a --transient option, where no content is committed to durable storage; the operation only works on a transient RAM arena that is overlaid and then discard at the end of the operation.
3. No opaque item is the sole carrier of meaning. Encrypted reasoning, compaction, and tool signatures may be included for same-provider quality, but each has a readable, provider-neutral handoff representation.
fyai> Opaque items are the providence of the provider meta stream and is not required for operation; merely for cache optimization.
4. Hosted tools have full-fidelity logs. Record exact inputs, outputs, evidence, filtering, provenance, timestamps, and content hashes — not only a polished answer and citations.
fyai> The arena records all operations of the main conversation, tool calls _and_ their agent invocations. There is not context loss what-so-ever, by design.
5. Subagent communication is auditable. Persist the exact readable task, messages, results, lineage, model, and tool permissions for every agent.
fyai> All agent communication, and agent tool calls are inspectable because it is stored, again by design.
6. Compaction is inspectable. Return a readable summary, the instructions used to create it, and enough lineage to understand what was discarded.
fyai> Compaction is configurable; you can configure the agent to only use model summary compaction which is generating a text answer, which is stored durably.
7. Artifacts are exportable. Files, container outputs, search snapshots, and generated media can be downloaded into a content-addressed local archive.
fyai> All artifacts are local; Nothing is executed on a hidden provider server.
In conclusion, I think fyai address the points of the article quite well, don't you think?
Re: The session you cannot take with you
#149[flagged]
Re: The session you cannot take with you
#150Earlier quoted context omitted.
I’ve actually started ignoring services where that’s the only login possibility. Sorry even if your platform is the greatest thing ever, but I’ll find a different tool. I’ve read one too many stories about Google (or Apple!) closing the entire account over some bullshit unnecessary reason like “fraudulent” gift card issues or whatever. I’m certain the affected people would’ve preferred to just pay back the amount in…
It's the only reason I want to replace Tailscale with something else or look into self-host when have a bit of time during my vacation. They only allow login through a third party, which is a big no for me.
While that isn't convenient is you don't want to use the public identity providers, it should make you think about what makes your identity on the Internet, and consider to have your own identity provider on a DNS domain you control.