Live data from Hacker News

Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

github.com

111–120 of 261 posts

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#111
post #45

Earlier quoted context omitted.

Unlike the big companies who do this, which often are merely impure scams on tax payer money a little more downstream.

Companies that generate loads of corporation tax, income tax, and VAT revenue are the exact opposite of wastes of public money.

Yes, when they do so proportional to what they take, especially as compared to individuals and their tax liabilities.

You'll have to let me know when that finally happens, because that ain't now.

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#112

> Every weight tensor in Rio is, to thousands of standard deviations, the same 0.6/0.4 blend of Nex and Qwen — across all 60 layers and every component of the network. Other finetunes cannot be explained as interpolations. I find it amazing how robust the current deep learning models are. A simple linear combination of every weight did not degrade the performance of the model, but enhanced it.

It shows that LLMs are an extremely wasteful approach to intelligence.

or that intelligence is merely the composition of many redundant, lossy, ~random components

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#113
post #69
post #47

“Well, Steve (Jobs), I think it’s more like we both had this rich neighbor named Xerox, and I broke into his house to steal the TV set, but I found out that you had already stolen it.” -- Bill Gates

What’s more funny to me is the set up to that quote: > Bill Gates had somehow manifested, alone, surrounded by ten Apple employees. … Steve started yelling at Bill, asking him why he violated their agreement. And what’s more interesting is the conclusion: > Apple filed a monumental copyright lawsuit against Microsoft in 1988, but they eventually lost on a technicality (the judge ruled that Apple inadvertently gave Mi…

That isn’t fully true is it?

Microsoft claimed that its software’s use of various visualizations related to window state was covered by the 1985 agreement, and Apple claimed that this was not true; those window states were produced by Macintosh while Microsoft’s software was being rendered in the Mac environment.

> In his March 20, 1989 Order, Judge Schwarzer declined to consider whether the visual displays in issue were generated by the Microsoft application programs or by the Macintosh system software. The point arose in connection with Microsoft's argument that the 1985 Agreement licensed to Microsoft all visual displays that could possibly be called up by running the five Microsoft application programs on the Macintosh system software then or in the future. 709 F. Supp. at 929. Judge Schwarzer concluded that Microsoft's contention would "defy common sense." Id.

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#114
post #77

Earlier quoted context omitted.

why not?

It is a recurrent Brazilian meme: Rio is known in Brazil as "terra de bandido" (gangster's land). The majority of their politicians have ties to organized crime. There is a virtual revolving door between police and crime, where people migrate from one to the other. It is like Chicago in the 20s, Naples and Medelin in the 80s or Moscow and Culiacan (Sinaloa, Mexico) today.

Somehow I doubt that political affiliations with crime syndicates are affecting heavily the dispositions of LLM developers. The industry itself though is one of incest.

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#115

> Every weight tensor in Rio is, to thousands of standard deviations, the same 0.6/0.4 blend of Nex and Qwen — across all 60 layers and every component of the network. Other finetunes cannot be explained as interpolations. I find it amazing how robust the current deep learning models are. A simple linear combination of every weight did not degrade the performance of the model, but enhanced it.

> A simple linear combination of every weight did not degrade the performance of the model, but enhanced it.

Which could be a signal that your "performance" was so abysmal in the first place that even randomly applied training methods can't make it _worse_.

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#116

Oh no, someone is profiting off of their work without proper attribution!?!?

Attribution isn't the relevant part. Lying about your lab's capabilities is.

It seems to me like the lies are both for the same reason. To capture attention and profits that are not deserved.

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#117
post #69
post #47

“Well, Steve (Jobs), I think it’s more like we both had this rich neighbor named Xerox, and I broke into his house to steal the TV set, but I found out that you had already stolen it.” -- Bill Gates

What’s more funny to me is the set up to that quote: > Bill Gates had somehow manifested, alone, surrounded by ten Apple employees. … Steve started yelling at Bill, asking him why he violated their agreement. And what’s more interesting is the conclusion: > Apple filed a monumental copyright lawsuit against Microsoft in 1988, but they eventually lost on a technicality (the judge ruled that Apple inadvertently gave Mi…

Two spoiled rich kids arguing over who's morality is the least worst.

That this moment is held up as some great exchange in business is annoying. That our regulatory agencies are perennially sleep at the switch and allow this nonsense to keep happening is extremely frustrating.

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#119
post #69

Earlier quoted context omitted.

What’s more funny to me is the set up to that quote: > Bill Gates had somehow manifested, alone, surrounded by ten Apple employees. … Steve started yelling at Bill, asking him why he violated their agreement. And what’s more interesting is the conclusion: > Apple filed a monumental copyright lawsuit against Microsoft in 1988, but they eventually lost on a technicality (the judge ruled that Apple inadvertently gave Mi…

Two spoiled rich kids arguing over who's morality is the least worst. That this moment is held up as some great exchange in business is annoying. That our regulatory agencies are perennially sleep at the switch and allow this nonsense to keep happening is extremely frustrating.

[flagged]

Re: Rio de Janeiro's "homegrown" LLM appears to be a merge of an existing model

#120
post #46

One funny thing about incompetence is that they don't have the competence to know that their incompetence is straightforward to verify by a competent person.

I wouldn’t describe what happened here as incompetence. As a “carioca”, I am pleasantly surprised to know that the government’s IT department is involved in AI work — even without the budget to create its own models from scratch.

It is a testament to the bloat and overreach of the Brazilian state in the economy. Such endeavors should be left to the private sector
Post reply on HN