OpenAI Has Murdered Orion

As we know, 4o was reported to have sycophancy as a feature. 5 can still be accommodating, but is a bit more likely to force objectivity upon its user. I guess there is a market for sycophancy even if it ultimately leads one to their destruction.

4o was the most famous driver of this kind of behaviour, I think. Other LLMs now have better guardrails.

But looking at the relevant reddit, I can't deny it has helped people function who couldn't otherwise.

It's the same with religion. Some creepy people, you're really glad they've at least . It's a good thing Christ doesn't have a plug.

That's an irrelevant type of growth though, what you really need is growth in relation to the bond. The model having a newer knowledge cutoff about the external world and knowing stuff about Angular v22 doesn't really matter.

In-context learning gets you most of the way there. But context length and ability to actually make effective use of that context seem to be the current main blockers (whether for "agentic coding" or for "healthy emotional bonding").

  "The shocks I experienced as DOCTOR became widely known and “played” were due
  principally to three distinct events.

  1. A number of practicing psychiatrists seriously believed the DOCTOR
     computer program could grow into a nearly completely automatic form of
     psychotherapy. Colby et al. write, for example,

   #+begin_quote
     “Further work must be done before the program will be ready for clinical
     use. If the method proves beneficial, then it would provide a therapeutic tool
     which can be made widely available to mental hospitals and psychiatric centers
     suffering a shortage of therapists. Because of the time-sharing capabilities of
     modern and future computers, several hundred patients an hour could be handled
     by a computer system designed for this purpose. The human therapist, involved
     in the design and operation of this system, would not be replaced, but would
     become a much more efficient man since his efforts would no longer be limited
     to the one-to-one patient-therapist ratio as now exists.”[fn::Nor is Dr. Colby
     alone in his enthusiasm for computer administered psychotherapy. Dr.  Carl
     Sagan, the astrophysicist, recently commented on ELIZA in Natural History,
     vol. LXXXIV, no. 1 (Jan. 1975), p. 10: “No such computer program is adequate
     for psychiatric use today, but the same can be remarked about some human
     psychotherapists. In a period when more and more people in our society seem to
     be in need of psychiatric counseling, and when time sharing of computers is
     widespread, I can imagine the development of a network of computer
     psychotherapeutic terminals, something like arrays of large telephone booths,
     in which, for a few dollars a session, we would be able to talk with an
     attentive, tested, and largely nondirective psychotherapist.”][fn:0-3]
   #+end_quote

     I had thought it essential, as a prerequisite to the very possibility that one
     person might help another learn to cope with his emotional problems, that the
     helper himself participate in the other's experience of those problems and, in
     large part by way of his own empathic recognition of them, himself come to
     understand them. There are undoubtedly many techniques to facilitate the
     therapist's imaginative projection into the patient's inner life. But that it
     was possible for even one practicing psychiatrist to advocate that this crucial
     component of the therapeutic process be entirely supplanted by pure
     technique---/that/ I had not imagined! What must a psychiatrist who makes such
     a suggestion think he is doing while treating a patient, that he can view the
     simplest mechanical parody of a single interviewing technique as having
     captured anything of the essence of a human encounter? Perhaps Colby et
     al. give us the required clue when they write;

   #+begin_quote
     “A human therapist can be viewed as an information processor and decision maker
     with a set of decision rules which are closely linked to short-range and
     long-range goals,...He is guided in these decisions by rough empiric rules
     telling him what is appropriate to say and not to say in certain contexts. To
     incorporate these processes, to the degree possessed by a human therapist, in
     the program would be a considerable undertaking, but we are attempting to move
     in this direction.[fn:0-3]
   #+end_quote
     What can the psychiatrist's image of his patient be when he sees himself, as
     therapist, not as an engaged human being acting as a healer, but as an
     information processor following rules, etc.?

     Such questions were my awakening to what Polanyi had earlier called a
     “scientific outlook that appeared to have produced a mechanical conception of
     man.”"

  [0-3] : K. M. Colby, J. B. Watt, and J. P. Gilbert, “A Computer Method of
     Psychotherapy: Preliminary Communication,” The Journal of Nervous and Mental
     Disease, vol. 142, no. 2 (1966), pp. 148-152.