Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The session you cannot take with you

earendil.com

Post reply on HN
Print view View original

51–60 of 238 posts

Re: The session you cannot take with you

#51

Post by aidos »

n6242 wrote: post #48
msdz wrote: post #24

Earlier quoted context omitted.

I’ve actually started ignoring services where that’s the only login possibility. Sorry even if your platform is the greatest thing ever, but I’ll find a different tool. I’ve read one too many stories about Google (or Apple!) closing the entire account over some bullshit unnecessary reason like “fraudulent” gift card issues or whatever. I’m certain the affected people would’ve preferred to just pay back the amount in…

It's the only reason I want to replace Tailscale with something else or look into self-host when have a bit of time during my vacation. They only allow login through a third party, which is a big no for me.

I think you can use a passkey now.

Re: The session you cannot take with you

#52

Post by hobofan »

dannyw wrote: post #46
hobofan wrote: post #26

I think the article gives a very good overview of a problem that most users of AI rarely evaluate / have to grapple with. There really is a surprising amount of coupling that happens with many of the "frontier inference providers", where a lot of the powerful non-LLM extensions (web search, code execution) are packaged as simple "tools" on the surface, that build up a lot of moat. Those are parts that are in theory n…

I'm in complete agreement that things like encrypted sub-agent messages, etc, are terrible, especially because it is fundamentally irresponsible to have an agent running on your machine without the slightest hint of what the prompt is. That said, I don't really see a problem with hosted tools being offered by providers. They're like impulse items at checkout. You shouldn't implement image generation as MCP: just writ…

> You shouldn't implement image generation as MCP: just write your own tool. There are plenty of image/media inference providers (e.g. Fal), web search or deep research providers, etc.

Yes, and we as "harness providers" don't want to have to hard-code integrations for each of them, and would rather provide an out-of-the box baseline in the form on an MCP server that our customers can either extend or replace with MCP servers by third-party providers. That helps us focus on the core of our offering, while also giving our customers good flexibility, which so far has been working out great.

That's exactly what standards/protocols like MCP are for. You likely nowadays also wouldn't suggest that someone builds bespoke logging/tracing integrations for each provider on the market while the good generic solution in building OTEL logging/tracing exists.

Re: The session you cannot take with you

#53

Post by corney91 »

n6242 wrote: post #48
msdz wrote: post #24

Earlier quoted context omitted.

I’ve actually started ignoring services where that’s the only login possibility. Sorry even if your platform is the greatest thing ever, but I’ll find a different tool. I’ve read one too many stories about Google (or Apple!) closing the entire account over some bullshit unnecessary reason like “fraudulent” gift card issues or whatever. I’m certain the affected people would’ve preferred to just pay back the amount in…

It's the only reason I want to replace Tailscale with something else or look into self-host when have a bit of time during my vacation. They only allow login through a third party, which is a big no for me.

Tbf to Tailscale they allow any OIDC provider[1]. I wish this was more normalised, then we could have one login everywhere regardless of who hosts it (even if it's yourself).

[1] https://tailscale.com/docs/integrations/identity/custom-oidc

Re: The session you cannot take with you

#55

Post by the_mitsuhiko »

captainmuon wrote: post #40
the_mitsuhiko wrote: post #30

Earlier quoted context omitted.

Because if you don't, you trash the KV cache and things get way more expensive. We wrote about it here: https://earendil.com/posts/prompt-caching/ Encrypted reasoning content is one of the things we keep as blobs in the transcript, even though we can only send it back to the original provider that created them, as otherwise we cannot keep caches live.

Right, but my point was, I should be able to just send something like {'continue_conversation_id': 123, 'message': 'another message?'} Seems like it would be even less work for the LLM host to just check if ID 123 is still in a cache (and if not, load it from a database) than decoding my request and checking cryptographic signatures. Right now the whole conversation history with blobs functions like a really long ID.

But then on a cache miss you need to resend the whole thing. Cached websocket connections on OpenAI work like this in a way. But once you lose the connection then you lose that.

Your idea though is the core of the responses API with store where everything is stored on their side and then you append to it.

For the LLM there is no difference btw. In either way you need to locate the cache and use it.

Re: The session you cannot take with you

#56

Post by blauditore »

placebo wrote: post #25
solarkraft wrote: post #9

This is an important article. I hadn’t realized it was already getting this bad. Like a frog enjoying a nice warm bath ... > Most people do not switch their operating system or phone provider every week either. But even if you do not utilize that freedom, it matters because it changes the relationship you have with the provider and the provider has with you. This is why it’s important to utilize your freedoms. Do NOT…

>Like a frog enjoying a nice warm bath... Exactly :-) Call me naive but I think dark patterns are a short term strategy for winning and I'm optimistic that in the long run they will be replaced with those that are more respectful and oriented to the greater good (granted, the long run might take more time than one hopes for). Given that the pendulum can sometime swing back fast enough to be able leverage it, it might…

> [...] dark patterns are a short term strategy for winning and I'm optimistic that in the long run they will be replaced with those that are more respectful and oriented to the greater good

This is definitely not the way most software has been going in the past decades. It's rather the opposite: Companies play friendly to obtain a user base, then start applying more and more dark patters to further increase their profit. Facebook, Evernote, Instagram, Komoot... The list goes on and on.

Re: The session you cannot take with you

#57

Post by surgical_fire »

the_mitsuhiko wrote: post #31
swyx wrote: post #8

Earlier quoted context omitted.

what would it take for this to be more than just hn blog fodder?

We were thinking about it while writing it. I think one thing at least open source harnesses can do is push back on using some of these conveniences from the providers. For instance the moment we start compromising on the idea of portable sessions, then there will be less pressure (For instance people asked for OpenAI server side compaction or built-in native web_search support). The real thing that will take this al…

I recently moved entirely to Pi from Claude Code. There's a bit of more upfront work in that I am taking the time to customize it to the way I want it to work, but I it is clear that it will benefit me in the end. It adds a level of intentionality to everything; all the building blocks I am adding to it in terms of templates, extensions, and skills are, obviously, things I want.

And I have been using Chinese models with it. It was not anything ideological. Those models are both excellent and cheap. Hard to beat that combo.

But on top of that, the sort of output I am getting in Pi in relation to what I got in CC is refreshing, in that nothing seems to be hidden from me.

And yeah, the ability to switch models mid-session is an interesting one, especially with branching; where I can branch to switch to a different model and still go back to the original branch with the original model.

Re: The session you cannot take with you

#59

Post by steilpass »

the_mitsuhiko wrote: post #31
swyx wrote: post #8

Earlier quoted context omitted.

what would it take for this to be more than just hn blog fodder?

We were thinking about it while writing it. I think one thing at least open source harnesses can do is push back on using some of these conveniences from the providers. For instance the moment we start compromising on the idea of portable sessions, then there will be less pressure (For instance people asked for OpenAI server side compaction or built-in native web_search support). The real thing that will take this al…

What are features that only work with session transparency / ownership?

e.g. - with model XYZ and their open reasoning token we are able to improve our skill ABC by 20% - cross checking between models improves results by 10% and is only possible with full context visibility

Re: The session you cannot take with you

#60

Post by Recursing »

Another comment got downvoted for raising this, but I found it interesting that pangram flags large parts of this blog post as AI generated ( https://www.pangram.com/history/b82cdbf4-bdbc-46ea-8315-bd43... ), while earendil denies it: https://news.ycombinator.com/item?id=49119957
Post reply on HN