Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The session you cannot take with you

earendil.com

Post reply on HN
Print view View original

91–100 of 238 posts

Re: The session you cannot take with you

#91

Post by lukebuehler »

the_mitsuhiko wrote: post #88
lukebuehler wrote: post #87

I think there are two things going on more generally: 1) providers increasingly adding hidden state that the user or developer cannot inspect, port, or do anything with. This is clearly bad. 2) providers increasingly diverging how they implement certain features and the APIs becoming quite complicated and provider specific. The former makes porting sessions or transcripts impossible, the latter just more difficult. I…

> providers increasingly diverging how they implement certain features and the APIs becoming quite complicated and provider specific This post only talks about the former.

Fair. But in the background of the article is clearly the goal to make sessions portable, not just being able to inspect it. It's partially implied even in the title.

There is just a whole discussion to be had about the second issue and the two are connected. For example session caching--and similar features in the future--will introduce session incompatibilities that break portability too.

Re: The session you cannot take with you

#92

Post by the_mitsuhiko »

Recursing wrote: post #60

Another comment got downvoted for raising this, but I found it interesting that pangram flags large parts of this blog post as AI generated ( https://www.pangram.com/history/b82cdbf4-bdbc-46ea-8315-bd43... ), while earendil denies it: https://news.ycombinator.com/item?id=49119957

The post for the most part is mine, so if something AI generated slipped into it it's on me. I can share with you what I did to write the article which I do for most of my writing.

I start out with what I want to talk about, and then make an initial draft. In this case I also had Sol research all current APIs so I don't have blind spots. I tend to also talk with an AI to see if the structure structure of the post makes sense. I then wrote all sections by hand but used Sol to fix up typos, grammar and punctuation. I also had Sol apply notes and patches that were provided via Discord.

If I find the earlier drafts I can check if it did some more outrageous edits, but I kinda doubt it did. I do know that my workflow of having an LLM to fixes up later is increasingly breaking on SOTA models and I have complained about this before. Normally I now carefully apply fixes that it proposes, but in this case I didn't due to time constraints.

I am curious though why it claims that this is AI generated.

Re: The session you cannot take with you

#93

Post by confusus »

try-working wrote: post #42

OpenAI is doing this so that you can't preserve cache when routing between for example GPT 5.6 and DeepSeek V4, or other providers. I'm building a router that easily reaches >99% cache hit rate in Pi when routing between GPT 5.4, Kimi, DeepSeek etc. Working on a big update that's coming out in a week or so.

Awesome! Any place I can follow the project?

Also, how to achieve these levels when switching mid conv? Don’t you effectively need to read the tokens per model switch?

Re: The session you cannot take with you

#95

Post by the_mitsuhiko »

lukebuehler wrote: post #91
the_mitsuhiko wrote: post #88

Earlier quoted context omitted.

> providers increasingly diverging how they implement certain features and the APIs becoming quite complicated and provider specific This post only talks about the former.

Fair. But in the background of the article is clearly the goal to make sessions portable, not just being able to inspect it. It's partially implied even in the title. There is just a whole discussion to be had about the second issue and the two are connected. For example session caching--and similar features in the future--will introduce session incompatibilities that break portability too.

> But in the background of the article is clearly the goal to make sessions portable, not just being able to inspect it. It's partially implied even in the title.

But not on the level that providers need to agree to a common format. Within Pi we are able to model portable sessions between providers even if they have provider specific APIs. For instance most models do not agree on how to do deferred tool loading. But they now have some support for that even if it diverges between models and APIs. So we can model deferred tool loading specifically for different providers without having to align on a common set of APIs or similar.

What cannot be made portable, is when information is sealed in the transcript or the information is entirely locked away on the provider's servers (store = true).

Re: The session you cannot take with you

#97

Post by carterschonwald »

im building my own harness and inference tool chain for much of these reasons. theres so much to do that makes a big difference for users. hoping to get things into shape for early alpha as a saas in the next two months.

heres the easiest biggy: compactions should include all user turns albeit with pastes and attached files not inlined. omg does it make a huge difference.

Re: The session you cannot take with you

#98

Post by eru »

mystifyingpoi wrote: post #36
eru wrote: post #16

Earlier quoted context omitted.

I'm actually more annoyed by having to pick passwords than confirming my email.

I'm opposite. If my email goes down, or I do not have access to it, I cannot login. With passwords I still can.

My email is with Google. They are less likely to go down than my password manager.

Re: The session you cannot take with you

#99

Post by eru »

hotelsacher wrote: post #21
eru wrote: post #16

Earlier quoted context omitted.

I'm actually more annoyed by having to pick passwords than confirming my email.

Picking passwords is a job best left to password managers. You don't use a hand crank when riding an elevator, do you?

I am using a password manager.

Re: The session you cannot take with you

#100

Post by skeledrew »

Part of the solution here I think is to move things out of band as much as possible. Make subagent invocations into tool calls to that agent. Externalize the tool calls themselves to CLI utils, maybe block native tools completely (unrelated but relevant, I just added the AskUserQuestion tool to the global deny list couple days ago because Claude occasionally forgot to honor my standing order to prefer plain text, and that tool is extremely annoying as it creates a gap in the dialogue) and use 3p alternatives. The /compact degradation is also very annoying, so make an alternative that does the same and saves the summary to a regular file. Maybe it's also worth prompting the LLM to save it's reasoning process to file, even if it takes a few extra tokens and it isn't the actual reasoning tokens.

With that said, I may have something that can already help with at least the subagents/tooling bit. Didn't really have a timeline (or solid intent) on releasing it, but with these shenanigans increasing there's no time like the present.

Post reply on HN