Viewing profile — necroforest

yeah, and as far as I'm aware there isn't even a definition of "civilization" or "intelligence" that doesn't boil down to "sufficiently like me".

I don't entirely disagree with you, but "what products do people want" is overly conservative. Pre-ChatGPT, very few people wanted a (more or less) general purpose chatbot.

It's branding (see: TensorFlow); also, pretty much anything (linear) you would do with an arbitrarily ranked tensor can be expressed in terms of vector ops and matmuls

cosine similarity is (isomorphic to) "distances to nearby objects". and not all embeddings are word embeddings.

If you need to relearn your iPhone/Mac interfaces every year, you might want to get screened for Alzheimers.

> Who doesn’t think China has a ten year AI algorithm to takeover Taiwan? anybody who works in either AI or natsec

> We arent going to see more progress until we have a way to generalize the compute graph as a learnable parameter That's a bold statement since a ton of progress has been made wit…

which laws are broken exactly? it's not remotely settled law that "training an NN = copyright infringement"

What's an order of magnitude between friends?

Maybe "best word processor for mac" is a better search for you?

LLMs are trained in parallel. The model weights and optimizer state are split over a number (possibly thousands) of accelerators. The main bottleneck to doing distributed training …

which jargon here is "just for show"?

>it's actually impossible to have a non-Hermitian matrix C such that x†Cx > 0 Nice catch, it's been a few years since I had to think about these details.

no, at least not for anything non-contrived.

I used Bayesian analysis for an industry problem where we had a lot of data (~1M samples), but the relationship between the observed data and the latent variables of interest was c…

JS divergence isn't just the average. it's 0.5(KL(P||M) + KL(Q||M)) where M is the mixture of P and Q: M = 0.5(P + Q)

kind of. you can decompose an arbitrary matrix into symmetric and antisymmetric components: R = S + A. Since A = -A^H (anti-symmetric), for any vector x, = - => = 0. So for any mat…

Does anyone use Ada in 2023? Back in college (~2008), I had a prof who was obsessed with it but AFAIK almost nobody used it even back them.

Nobody (to good approximation) plays with weights during the peer review process.

Having a project page with extra results is very common in computer vision, even in academia.

To answer (2): You are token i. In order to see how much of a token j's value v_j you update yourself with, you compare your query q_i with token j's key k_j. This gives you the as…

you can't store a function as a tensor. the tensors are the inputs/outputs that flow along the edges of the graph. TF stores things directly as a graph: https://github.com/tensorfl…

> AI research has put hardly any effort into building goal-directed agents The entire (enormous) field of reinforcement learning begs to differ.

no

not in collaboration with openai, one of the authors joined openai before the paper was written and arxiv'd