91–100 of 217 posts
Earlier quoted context omitted.
> Have links gone out of fashion? Yes. You wrote far more words than needed to answer the comment, I did it for you instead.
One of the reasons I write so much stuff is so I can provide links to things I've written to answer relevant questions.
Earlier quoted context omitted.
The very definition of what constitutes open source is being called into question in these kinds of discussions about AI. Without the training details and the weights being made fully open it’s hard to really call something truly open, even if it happens to meet some arbitrary definition of “open source”. A good definition of “truly open” is whether the exact same results can be reproduced by someone with no extra in…
Notice you are creating your own arbitrary definition of 'truly open', which IMHO corresponds more with 'reproducible'. We already have a definition of open source. I don't see any reason to change it.
Earlier quoted context omitted.
I imagine it's more useful for finding related articles and clustering things than for semantic search, which will work much better against smaller chunks - especially if you're implementing Retrieval Augmented Generation.
I think the point is: if you compress 25 pages of text into 1024 floats, you will lose a ton of information, regardless of what the use case is, so you're probably still better of with chunking.
Earlier quoted context omitted.
It definitely is open source even if they don’t disclose all details behind the training
The very definition of what constitutes open source is being called into question in these kinds of discussions about AI. Without the training details and the weights being made fully open it’s hard to really call something truly open, even if it happens to meet some arbitrary definition of “open source”. A good definition of “truly open” is whether the exact same results can be reproduced by someone with no extra in…
Earlier quoted context omitted.
> You could just sum it up for us all rather than do a divert to your blog? Why? Have links gone out of fashion? I even linked directly to the relevant section rather than linking to the top of the page. The paper that coined the term used the hyphen, though I think I prefer it without: https://arxiv.org/abs/2005.11401
> Have links gone out of fashion? Yes. You wrote far more words than needed to answer the comment, I did it for you instead.
Earlier quoted context omitted.
> Have links gone out of fashion? Yes. You wrote far more words than needed to answer the comment, I did it for you instead.
One of the reasons I write so much stuff is so I can provide links to things I've written to answer relevant questions.
How well do LLMS like this work with a non-English language? Or are these open source models limited to English?
Earlier quoted context omitted.
Did you compare with simple baselines like bag-of-words and word vectors?
My previous implementation used TF-IDF - I basically took all the words in the post and turned them into a giant "word OR word OR word OR word" search query and piped that through SQLite full-text search. https://til.simonwillison.net/sqlite/related-content I jumped straight from that to OpenAI embeddings. The results were good enough that I didn't spend time investigating other approaches.
Earlier quoted context omitted.
The very definition of what constitutes open source is being called into question in these kinds of discussions about AI. Without the training details and the weights being made fully open it’s hard to really call something truly open, even if it happens to meet some arbitrary definition of “open source”. A good definition of “truly open” is whether the exact same results can be reproduced by someone with no extra in…
Notice you are creating your own arbitrary definition of 'truly open', which IMHO corresponds more with 'reproducible'. We already have a definition of open source. I don't see any reason to change it.
It's basically like giving people a binary program and calling it open source because the compiler and runtime used are open source.
Earlier quoted context omitted.
My previous implementation used TF-IDF - I basically took all the words in the post and turned them into a giant "word OR word OR word OR word" search query and piped that through SQLite full-text search. https://til.simonwillison.net/sqlite/related-content I jumped straight from that to OpenAI embeddings. The results were good enough that I didn't spend time investigating other approaches.
That's not quite tfidf though. I agree you can get better results than that with Ada embeddings, but I would argue you can get even better results with embeddings from smaller chunks.