Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Retrieval in LangChain

blog.langchain.dev

Post reply on HN
Print view View original

41–50 of 71 posts

Re: Retrieval in LangChain

#41

Post by NicoJuicy »

hellcow wrote: post #30
ArenaSource wrote: post #27

Earlier quoted context omitted.

If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...

My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5. As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong). In my case, understanding/fixing the code would…

It can't learn zig without plenty of samples

Re: Retrieval in LangChain

#43

Post by hombre_fatal »

jcims wrote: post #29
ArenaSource wrote: post #27

Earlier quoted context omitted.

If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...

Yeah I can't wait to get API access to gpt-4, it is a stepwise more capable based on the stuff I've done with chatgpt on gpt-4. That said, even gpt-3.5 will try multiple routes to get to the same endpoint. It seems to get distracted pretty easily though.

One demo of gpt-4’s superiority over gpt-3 is to come up with a prompt that determines the language of some given text.

I couldn’t figure out a gpt-3 prompt that could handle “This text is written in French” correctly (it thinks it’s written in French), but with gpt-4 you can include in the prompt to disregard what the text says and focus on the words and grammar that it uses.

Re: Retrieval in LangChain

#45

Post by roflyear »

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

And you only used the most popular example of getting started and one of if not the most popular programming languages. Amazing.

Re: Retrieval in LangChain

#46

Post by roflyear »

antibasilisk wrote: post #20
jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

That pastebin is mindboggling.

How so?

Re: Retrieval in LangChain

#47

Post by roflyear »

hellcow wrote: post #30
ArenaSource wrote: post #27

Earlier quoted context omitted.

If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...

My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5. As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong). In my case, understanding/fixing the code would…

It's a step up by an order of magnitude for certain things. Like chess. It is really good at chess actually. But not programming. Seems maybe marginally better on average. Worse in some ways.

Re: Retrieval in LangChain

#49

Post by SCUSKU »

jcims wrote: post #24
px43 wrote: post #23

Earlier quoted context omitted.

Can you share the code for llm.py?

#!/home/ubuntu/venv/bin/python3.10 from langchain.agents import load_tools from langchain.agents import initialize_agent from langchain.chat_models import ChatOpenAI llm = ChatOpenAI(model='gpt-3.5-turbo',temperature=0) tools = load_tools(['python_repl', 'requests', 'terminal', 'wolfram-alpha', 'serpapi', 'wikipedia', 'human', 'pal-math', 'pal-colored-objects'], llm=llm) agent = initialize_agent(tools, llm, agent="ch…

People are joking that they can't wait for the next big thing in AI to come out in a few weeks, but this seems pretty big. After fiddling with it for a while, its not perfect, but this isn't that far from being able to replace (or at least dramatically change) my job as a software engineer.

For example, I asked it to write conway's game of life, and it took about 4-5 attempts but it wrote fully functional code that popped up a matplotlib window with a fully functional simulation. This would've taken me a day at least.

I asked it to write a FastAPI backend that uses SQLite for storing blog Posts, and it struggled with that one a lot and couldn't quite get it right, although I think that's largely a limitation of the python REPL from langchain as opposed to GPT.

On the one hand I'm excited to build all sorts of new things and projects with this, but on the other hand I'm worried my standard of living will decline because my skills will become super commodified :/

Re: Retrieval in LangChain

#50

Post by VectorLock »

hellcow wrote: post #30
ArenaSource wrote: post #27

Earlier quoted context omitted.

If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...

My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5. As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong). In my case, understanding/fixing the code would…

GPT-4 is just regurgitating what its "learned" from previously scraped content on the Internet. If somebody didn't answer it on StackOverflow before 2021, it doesn't know it. It can't reason able anything, it doesn't "understand" stacks or pointers.

That said its really good at regurgitating stuff from StackOverflow. But once you step beyond anything that someone has previously done and posted to the Internet, it quickly gets out of its depth.

Post reply on HN