Tensor Is the Might • Hacker News Board

Re: Tensor Is the Might

#11

Post by bssrdf » Tue, Jul 14, 2026, 2:49 PM UTC

zserge wrote: post #6

srean wrote: post #5

If one wants to add the capability to reason about shape and shape compatibility, Barry Jay's FiSh would be an interesting detour. https://web.archive.org/web/20111015133833/http://www-staff.... This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops. [Programming in…

I wish we used something else but Python as the default ML language, so that some advanced type system would work for us to ensure tensor shapes and compatibility.

I found [ggml] is really easy, educational and fun to use. It is written in C and the backbone for llama.cpp.

[ggml]https://github.com/ggml-org/ggml

Re: Tensor Is the Might

#12

Post by khalic » Tue, Jul 14, 2026, 2:59 PM UTC

Why does does the diagram say tensors are 3D?

Re: Tensor Is the Might

#13

Post by zserge » Tue, Jul 14, 2026, 3:07 PM UTC

khalic wrote: post #12

Why does does the diagram say tensors are 3D?

I guess I miscommunicated it. I meant I'd call everything a tensor but for some smaller dimensions we have specific names. Of course, 1D can still be a tensor and so is 6D or nD.

Re: Tensor Is the Might

#14

Post by reerdna » Tue, Jul 14, 2026, 3:28 PM UTC

Cool, but I find rather than just shapes and indexes, tensors with labels are much easier to use and reason about. E.g.:

  {
      {user:bob, movie:"Heat"}:0.1,
      {user:alice, movie:"Frozen"}:0.9,
      {user:carol, movie:"Top Gun"}:0.3,
  }

https://docs.vespa.ai/en/ranking/tensor-user-guide.html

Re: Tensor Is the Might

#15

Post by ux266478 » Tue, Jul 14, 2026, 3:32 PM UTC

zserge wrote: post #6

srean wrote: post #5

If one wants to add the capability to reason about shape and shape compatibility, Barry Jay's FiSh would be an interesting detour. https://web.archive.org/web/20111015133833/http://www-staff.... This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops. [Programming in…

I wish we used something else but Python as the default ML language, so that some advanced type system would work for us to ensure tensor shapes and compatibility.

Haskell is actually really good for that. Hasktorch[1] is high quality, batteries included, and leverages Haskell's GADTs for some cute stuff like gradual tensor shape checking. There's also just a grain to Haskell that feels really good for the machine learning domain. It doesn't just alleviate whole classes of bugs and a lot of annoying background noise reasoning, but it's also a much more natural-feeling expression.

The "gotcha" is that Haskell is heavy duty machinery, and getting up to speed with it if your background lacks solid type-theory can be really daunting. For that reason alone, it could never be the default. Sometimes I like to think about how much of a disservice academia has done to itself by training mathematicians without giving them the foundational knowledge they need to utilize the nuclear-grade tooling they themselves have the most potential to benefit from. For a number theorist? Sure makes sense. But the fact that machine learning courses don't have rigorous undergraduate prerequisites in learning the foundation of computation is pure absurdity.

[1] - http://hasktorch.org/

Re: Tensor Is the Might

#16

Post by hasteg » Tue, Jul 14, 2026, 3:47 PM UTC

I just recently watched some (not all) of this video "coding a machine learning library in c from scratch" and seems like he's going through a similar process in this blog as this video. I would recommend watching the video to get an idea of what the fundamentals of a ML library look like. From someone who has recently been getting interested in actually writing ML code and trying to make sense of it myself (from the perspective of just a typical backend engineer) it was very interesting to see. Previously my experience with ML libs (PyTorch specific) was writing my own Mini-GPT and training it on a small dataset using my own GPU (5090). Cool to see the behind the scenes and took away some o the handwaveyness... https://www.youtube.com/watch?v=hL_n_GljC0I

Re: Tensor Is the Might

#17

Post by khalic » Tue, Jul 14, 2026, 4:09 PM UTC

zserge wrote: post #13

khalic wrote: post #12

Why does does the diagram say tensors are 3D?

I guess I miscommunicated it. I meant I'd call everything a tensor but for some smaller dimensions we have specific names. Of course, 1D can still be a tensor and so is 6D or nD.

Got it, thx!

Re: Tensor Is the Might

#18

Post by zserge » Tue, Jul 14, 2026, 4:42 PM UTC

khalic wrote: post #17

zserge wrote: post #13

Earlier quoted context omitted.

I guess I miscommunicated it. I meant I'd call everything a tensor but for some smaller dimensions we have specific names. Of course, 1D can still be a tensor and so is 6D or nD.

Got it, thx!

I suppose one of a few examples when a picture is totally not worth a thousand words, if a picture is made by me

Re: Tensor Is the Might

#19

Post by terminalbraid » Tue, Jul 14, 2026, 4:48 PM UTC

> A tensor is nothing but a flat array of numbers

I'm so very, very tired of tech coopting rigorous mathematical terms.

Re: Tensor Is the Might

#20

Post by mamcx » Tue, Jul 14, 2026, 4:51 PM UTC

terminalbraid wrote: post #19

> A tensor is nothing but a flat array of numbers I'm so very, very tired of tech coopting rigorous mathematical terms.

I read it as "tensors in programming*.

This also happens to "numbers", "integers" and such, because in the computer universe don't exist out math, only a subset(?) variation(?) of it