Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jina AI launches open-source 8k text embedding

jina.ai

Post reply on HN
Print view View original

201–210 of 217 posts

Re: Jina AI launches open-source 8k text embedding

#201

Post by EGreg »

simonw wrote: post #181
EGreg wrote: post #151

Earlier quoted context omitted.

People need to realize something… The model weights in eg TensorFlow are the source code . It is not a von-Neumann architecture but a gigabyte of model weights is the executable part, no less than a gigabyte of imperative code. Now, the training of the model is akin to the process of writing the code. In classical imperative languages that code may be such spaghetti code that each part would be intertwined with 40 ot…

That doesn't work for me. The model weights aren't source code. They are the binary result of compiling that source code. The source code is the combination of the training data and configuration of model architecture that runs against it. The model architecture could be considered the compiler. If you give me gcc and your C code I can compile the binary myself. If you give me your training data and code that impleme…

No, you would need to spend “eye watering amounts of compute” to do it, similar to hiring a lot of developers to produce the code. The compiling of the code to an executable format is a tiny part of that cost.

Re: Jina AI launches open-source 8k text embedding

#202

Post by simonw »

EGreg wrote: post #201
simonw wrote: post #181

Earlier quoted context omitted.

That doesn't work for me. The model weights aren't source code. They are the binary result of compiling that source code. The source code is the combination of the training data and configuration of model architecture that runs against it. The model architecture could be considered the compiler. If you give me gcc and your C code I can compile the binary myself. If you give me your training data and code that impleme…

No, you would need to spend “eye watering amounts of compute” to do it, similar to hiring a lot of developers to produce the code. The compiling of the code to an executable format is a tiny part of that cost.

I still think of millions of dollars of GPU spend crunching away for a month as a compiler.

A very slow, very expensive compiler - but it's still taking the source code (the training material and model architecture) and compiling that into a binary executable (the model).

Maybe it helps to think about this at a much smaller scale. There are plenty of interesting machine learning models which can be trained on a laptop in a few seconds (or a few minutes). That process feels very much like a compiler - takes less time to compile than a lot of large C++ projects.

Running on a GPU cluster for a month is the exact same process, just scaled up.

Huge projects like Microsoft Windows take hours to compile and that process often runs on expensive clusters, but it's still considered compilation.

Re: Jina AI launches open-source 8k text embedding

#203

Post by Der_Einzige »

simonw wrote: post #189
theptip wrote: post #186

Earlier quoted context omitted.

> The 8k context window is new Hasn’t Claude had this for many months (before they bumped to 100k)? Edit: ah, you mean new for OSS maybe?

Claude is a large language model, which is a different thing from an embedding model.

Any large language model generates embedding representations at every layer of the model, and these can be trivially extracted. So, large language models are indeed embedding models.

This leaderboard doesn't compare these custom tailored embedding models vs the obvious thing of average pooling layered with any traditional LLM, which is easily implemented using sentence transformers.

Re: Jina AI launches open-source 8k text embedding

#204

Post by EGreg »

simonw wrote: post #202
EGreg wrote: post #201

Earlier quoted context omitted.

No, you would need to spend “eye watering amounts of compute” to do it, similar to hiring a lot of developers to produce the code. The compiling of the code to an executable format is a tiny part of that cost.

I still think of millions of dollars of GPU spend crunching away for a month as a compiler. A very slow, very expensive compiler - but it's still taking the source code (the training material and model architecture) and compiling that into a binary executable (the model). Maybe it helps to think about this at a much smaller scale. There are plenty of interesting machine learning models which can be trained on a lapto…

Actually, the dirty secret is that a lot of human work (at below minimum wage) went into training and refining the AI models:

https://time.com/6247678/openai-chatgpt-kenya-workers/

And billion-dollar companies made their money off it:

https://www.forbes.com/sites/kenrickcai/2023/04/11/how-alexa...

That’s the dirty secret of why ChatGPT 4 is better. But they’ll tell you it has to do with chaining ChatGPT 3’s together, more fine tuning etc. They go to these poor countries and recruit people to work on training the AI.

Not to mention all the uncompensated work of humans around the world who put their content up on the Web.

Re: Jina AI launches open-source 8k text embedding

#205

Post by magicalhippo »

torginus wrote: post #157
magicalhippo wrote: post #130

Earlier quoted context omitted.

The quake engine is still open source even though it doesn't come with the quake game assets, no? It seems unreasonable to require the training data just to be called open source, given it has similar copyright challenges as game assets. Of course, this wouldn't make the model reproducible. But that's different from open source.

It's a bit different - here most of the value lies in the weights. A better analogy would be some graphics card drivers which ship a massive proprietary GPU firmware blob, and a small(ish) kernel shim to talk with said blob.

Well perhaps we can consider this a kind of short-sightedness of Stallmann. His point with GPL and the free software movement, as I understand it, was to ensure the user could continue to use the software regardless of what the software author decided to do.

Sometimes though the software alone can be near useless without additional assets that aren't necessarily covered by the code license.

Like Quake, having the engine without the assets is useless if what you wanted was to play Quake the game. Neural nets are another prime example, as you mention. Simulators that rely on measured material property databases for usable results also fall into this category, and so on.

So perhaps what we need is new open source licenses that includes the assets needed for the user to be able to reasonably use the program as a whole.

Re: Jina AI launches open-source 8k text embedding

#206

Post by Havoc »

do-me wrote: post #164

Just quantized the models for onnx usage in e.g. transformers.js and got 4x reduced file size: - 𝟐𝟖.𝟓 𝐌𝐁 jina-embeddings-v2-small-en ( https://huggingface.co/do-me/jina-embeddings-v2-small-en ) - 𝟏𝟎𝟗 𝐌𝐁 jina-embeddings-v2-base-en ( https://huggingface.co/do-me/jina-embeddings-v2-base-en ) However, I noted, that the base model is performing quite poorly on small text chunks (a few words) while the small vers…

Why quantize something that is already very small (270mb)?

Re: Jina AI launches open-source 8k text embedding

#207

Post by anigbrowl »

m3at wrote: post #131
anigbrowl wrote: post #97

Earlier quoted context omitted.

Stability has a Japanese port which is getting lots of work https://twitter.com/StabilityAI_JP/status/171699857824440759...

This is not an embedding model though. Yes you can always extract some embeddings from somewhere, but for most LLMs those won't perform well for retrieval (which makes sense as it's not what the models are optimizing for)

This isn't an embedding model, but it is a group of people working in this general area in a language other than English. Maybe they'll get to an embedding model next?

Re: Jina AI launches open-source 8k text embedding

#208

Post by osigurdson »

TeMPOraL wrote: post #127
minimaxir wrote: post #65

Earlier quoted context omitted.

That's a property of Word2Vec specifically due to how it's trained (a shallow network where most of the "logic" would be contained within the embeddings themselves). Using it for embeddings generated from LLMs or Embedding layers will not give as fun results; in practice the only thing you can do is average or cluster them.

> That's a property of Word2Vec specifically due to how it's trained (a shallow network where most of the "logic" would be contained within the embeddings themselves). Is it though? I thought the LLM-based embeddings are even more fun for this, as you have many more interesting directions to move in. I.e. not just: emb("king") - emb("man") + emb("woman") = emb("queen") But also e.g.: emb( ) + a v(sad) + b v(short) -…

Interesting video. When he says "we decode the embedding", does he essentially mean that he is searching a vector database or something else?

Re: Jina AI launches open-source 8k text embedding

#209

Post by pietz »

Havoc wrote: post #206
do-me wrote: post #164

Just quantized the models for onnx usage in e.g. transformers.js and got 4x reduced file size: - 𝟐𝟖.𝟓 𝐌𝐁 jina-embeddings-v2-small-en ( https://huggingface.co/do-me/jina-embeddings-v2-small-en ) - 𝟏𝟎𝟗 𝐌𝐁 jina-embeddings-v2-base-en ( https://huggingface.co/do-me/jina-embeddings-v2-base-en ) However, I noted, that the base model is performing quite poorly on small text chunks (a few words) while the small vers…

Why quantize something that is already very small (270mb)?

Just making up stuff here, but smaller models are great for serverless compute like functions, which would also benefit from lighter computation. Don't forget, some people are dealing with hundreds of millions of documents. Accelerating this by 4x may be worth a small performance hit.

Re: Jina AI launches open-source 8k text embedding

#210

Post by jimmySixDOF »

egorfine wrote: post #192

One thing that is missing in comparison: OpenAI's model is multilingual. And not only it supports and embeds a variety of languages, it also computes the same coordinates for the same semantics in different languages. I.e. if you embed "russia is a terrorist state" and "россия - страна-террорист", both of these embeddings will have almost the same coordinates.

I heard one of the developers on a regular Open Source DIY AI X/twitter space [1] & they are targeting two new models German/English and French/English for the next release

https://x.com/thursdai_pod

Post reply on HN