Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta AI Unleashes Megabyte, a Scalable Model Architecture

artisana.ai

Post reply on HN
Print view View original

71–80 of 213 posts

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#71

Post by dumpsterdiver »

1024core wrote: post #7
crakenzak wrote: post #2

Paper: https://arxiv.org/abs/2305.07185 Wow, seems like Meta AI is so ahead of the curve compared to even Google and OpenAI recently especially with their open sourcing pushes. Great for the research community as a whole!

Meta has no horse in the race (i.e. they don't have a search engine). So, they don't mind throwing random things out. Withholding it won't really make much of a difference for them, as they don't have a way to productionize the tech.

While I disagree that having a search engine is the only way to have a "horse in the race", I must agree that at this point Meta does not appear to have a horse in the race.

Other companies are providing services that are so useful that it makes us think twice about how secure our jobs are. Then there is Meta, who seems to think that the world at large will forget about the terrible motion sickness that their VR products have wrought upon us. I for one will not forget. I'm actually traumatized, and even thinking about putting on VR goggles at this point makes me feel queasy.

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#72

Post by qas123 »

abwizz wrote: post #67
1024core wrote: post #7

Earlier quoted context omitted.

Meta has no horse in the race (i.e. they don't have a search engine). So, they don't mind throwing random things out. Withholding it won't really make much of a difference for them, as they don't have a way to productionize the tech.

would not be surprised if they integrated a "suggested conversations" feature based on your chat history and behaviour, where the user just picks sentences from a list and both parties can enjoy a effortless "organic" conversation.

My question then would be how it impacts advertising. we will essentially have bots talking to bots

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#73

Post by machdiamonds »

zxexz wrote: post #10

Great paper. But wow, I really wish everyone would use more easily searchable names for their projects. In 6 months, there’s a high probability I’ll end up googling/ddging “megabyte model” trying to find this paper again.

I've been using myReach as a bookmark manager lately. I've found it to be pretty useful for resurfacing information from links I have saved. I think they're using embeddings to help you track down information across different documents, articles, posts, etc. The big sell is that it's a personalized AI assistant that answers your specific queries based on what data you feed it (e.g.,"what was my electricity bill last month"). However, I'm hesitant about uploading personal data, so I am just using it as a bookmark manager. Their LLM although a little slow, works pretty well. I had this HN link saved about an open-source TTS model. A commenter said they were going to release a model later that week that could be comparable to Elevenlabs. I asked the chatbot on myReach: "What's that TTS model that's looking to rival ElevenLabs?" It surfaced the article and used the specific comment for a response. I'm not sure about the future of these kinds of services from startups. Given their ability to integrate browsers, emails, cloud storage, photos, and more, Google and Microsoft are likely to develop a similar service. With the resources at their disposal, they could probably design something superior and streamline the process of joining. But for now, I think I'll continue using myReach.

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#74

Post by wolfd »

1024core wrote: post #7
crakenzak wrote: post #2

Paper: https://arxiv.org/abs/2305.07185 Wow, seems like Meta AI is so ahead of the curve compared to even Google and OpenAI recently especially with their open sourcing pushes. Great for the research community as a whole!

Meta has no horse in the race (i.e. they don't have a search engine). So, they don't mind throwing random things out. Withholding it won't really make much of a difference for them, as they don't have a way to productionize the tech.

I would be _shocked_ if Zuck wasn't thinking 24/7 about how to capitalize on LLMs. I'm sure there are a thousand ideas (maybe even a few good ones?) being thrown around Meta at how to use LLMs to beat Google/Microsoft+OpenAI at the "search buddy" game.

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#75

Post by comex »

qup wrote: post #43
p-e-w wrote: post #37

Earlier quoted context omitted.

> If AGI is a computer passing a test, then AGI was achieved a long time ago. AIs couldn't even pass a third-grade reading comprehension exam until about 5 years ago. Computers being able to pass tests designed for humans is a very new thing. > it's clear LLMs are not AGIs And the main argument for that is that "it's clear". They're beating lawyers, doctors, and software engineers, but obviously, that's not real inte…

> Computers being able to pass tests designed for humans is a very new thing. Yes! Captchas so effective!

Yes, it would be more accurate to say “computers being able to pass tests designed for humans by humans”, as opposed to CAPTCHAs which are automatically generated tests.

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#76

Post by baq »

mirekrusin wrote: post #60
wudangmonk wrote: post #39

Earlier quoted context omitted.

You mean the average brain with about 100 billion neurons with about 1000 connections each bringing it to around 100 trillion connections. With an estimated 1000 "AI" neurons required per biologial neuron. I don't think you are givin these "below average" intelligence individuals enough credit. What we consider a genius is the equivalent of a dog show obstacle course. We measure intelligence/genius as whatever is har…

Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Majority of human brain computation is spent on things that are simply not necessary for computer models (how to wiggle limbs, mouth, eyes etc). Current LLM are impressive, but we know they can be much more efficient - we're using very low quality training data, we don…

> Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc.

Wheels are trivial but useless without bearings.

Bearings most certainly aren't trivial. All the things you listed further are dependent on bearings somewhere.

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#77

Post by shreyshnaccount »

flangola7 wrote: post #51
shreyshnaccount wrote: post #47

Earlier quoted context omitted.

You're assuming that everything needed to write code or make arguments is purely intelligence based and has nothing to do with patterns, structural repetition and things glorified autocomplete could do, and that's not true.

Who said anything about everything? AI could not write code AT ALL until very recently. We could invent a drug that kills 99% of cancers and the next day there would be people bemoaning that it isn't a "true" cure.

Hey you're the one claiming intelligence and all, burden of proving it in squarely on you

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#78

Post by mirekrusin »

baq wrote: post #76
mirekrusin wrote: post #60

Earlier quoted context omitted.

Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Majority of human brain computation is spent on things that are simply not necessary for computer models (how to wiggle limbs, mouth, eyes etc). Current LLM are impressive, but we know they can be much more efficient - we're using very low quality training data, we don…

> Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Wheels are trivial but useless without bearings. Bearings most certainly aren't trivial. All the things you listed further are dependent on bearings somewhere.

I think 5yo kid with lego would have problems agreeing with this otherwise moot statement.

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#79

Post by baq »

mirekrusin wrote: post #78
baq wrote: post #76

Earlier quoted context omitted.

> Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Wheels are trivial but useless without bearings. Bearings most certainly aren't trivial. All the things you listed further are dependent on bearings somewhere.

I think 5yo kid with lego would have problems agreeing with this otherwise moot statement.

Don't want to sound mean but I don't think you've dealt with bearings if you say so.

Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture

#80

Post by pedrosorio »

shreyshnaccount wrote: post #48
p-e-w wrote: post #22

Earlier quoted context omitted.

> when an average human that knows programming should be able to make one that works. Very few people who "know programming" are actually capable of creating a complete application that serves a specific purpose. Many professional programmers struggle to implement basic algorithms like prime number testing. Coding AIs absolutely outperform the average programming professional, because the average programming professi…

Who is this average programmer you're talking about? We did primality testing in literally the first sem at uni

Take a look at this old, commonly referenced article, might be before your time:

https://blog.codinghorror.com/why-cant-programmers-program/

Post reply on HN