Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

Post by MMMercy2 » Fri, Mar 31, 2023, 1:27 AM UTC

Jackson__ wrote: post #51

>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them. This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the man…

Post by ghc » Fri, Mar 31, 2023, 1:35 AM UTC

Post by istjohn » Fri, Mar 31, 2023, 1:43 AM UTC

pazimzadeh wrote: post #39

huseyinkeles wrote: post #23

Earlier quoted context omitted.

Interesting. GPT3.5 almost completely fails at this task. Just spouts some random names, some of which has the letter Z. Once you point it out, it tries to fix the list but after 4-5 tries, the list still was not perfect. GPT4 on the other hand gave a correct list on the first try. ---- GPT3.5 Zlatan Ibrahimovic - Swedish professional footballer who plays as a striker for AC Milan Sergio Ramos - Spanish professional…

What was your prompt? I just asked GPT4 and it is way off the mark: Zinedine Zidane Cristiano Ronaldo Mesut Özil Gianluigi Buffon Zlatan Ibrahimović David Alaba Nemanja Vidić Miroslav Klose Franck Ribéry Patrice Evra Gonzalo Higuaín Marek Hamšík Miralem Pjanić Diego Godín Sadio Mané Dani Alves Kalidou Koulibaly Koke Hakim Ziyech Piotr Zieliński Wilfried Zaha Arturo Vidal Raphaël Varane Steven N'Zonzi Blaise Matuidi M…

Post by evanchisholm » Fri, Mar 31, 2023, 2:13 AM UTC

nico wrote: post #36

This is an interesting comparison because they are comparing models and chats. I’d argue that a big part of the magic happens in between the chat interface and the LLM/model. We had GPT3 for a long time and most people didn’t really care too much. But then when they added Chat on top and whatever they are doing to provide previous context in each prompt… that was magic. So then I wonder how much better could we make…

Post by mdaniel » Fri, Mar 31, 2023, 2:24 AM UTC

ajdude wrote: post #74

m3kw9 wrote: post #72

Can it write code or suggest ffmpeg command line? If not you can’t seriously say this is nearly as good as ChatGPT.

I just asked to create an HTTP server in python that returned the code I am a tea pot, this is what it said. Not exactly what I wanted it to do lol (Using a paste bin link because of HN formatting) https://pastebin.com/hcBubZUn

    import http.server
    import socketserver
     
    class TeapotHTTPServer(http.server.SimpleHTTPRequestHandler):
        def end_headers(self):
            self.send_response(200)
            self.send_header('Content-type', 'text/plain')
            self.end_headers()
            self.wfile.write(b'I am a teapot')
     
    Handler = TeapotHTTPServer
     
    with socketserver.TCPServer(("localhost", 8000), Handler) as httpd:
        print("serving at port", httpd.address())
        httpd.serve_forever()