Rust SIMD on the GPU

What is vectorware's business model? Are you planning to sell support/consulting to companies using your stack? Or are you looking to sell licenses to your tool? Or something else?

Welcome to the lucky 10,000! SIMD is actually a pretty integral part of how GPUs are able to work efficiently, it's part of why there's such a strong focus on branchless programming in the field.

[flagged]

Didn't want to go into crazy detail in the post.

Each family of operations is a trait parameterized by the operation itself:

  pub trait EvaluateReduction: LaneEvaluator {
      /// Reduce one distributed definition to an ordinary uniform scalar.
      fn evaluate_reduction(&self, value: LaneValue) -> T;
  }

Call sites name the operation:

  let one   = evaluator.splat::(1_u32);
  let two   = evaluator.splat::(2_u32);
  let three = evaluator.binary::(one, two);

  let total   = evaluator.reduce::(three);   // a uniform u32
  let running = , u32>>::scan(&evaluator, three);

Operations like Sum, Max, ReduceXor, Inclusive, and Exclusive are all distinct types.

As mentioned in the post, execution shape is typed too. A static shuffle takes its control as a type-level constant, and the shuffle mode constrains which controls are expressible:

  // Shift down one lane, keeping our own value where the source is inactive.
  let down  = , DownOrSelf, u32>>::shuffle(&ev, v);
  // Broadcast from lane zero.
  let bcast = , WarpLane, u32>>::shuffle(&ev, down);
  // Butterfly exchange with the neighbor one bit away.
  let bfly  = , Butterfly, u32>>::shuffle(&ev, bcast);

For an example of errors caught, a warp-scoped executor for a device-scoped barrier is a compile error:

   as EvaluateBarrier>>::barrier(evaluator)
  // error[E0277]: the trait bound `Device: NvptxBarrierScope` is not satisfied
  //               help: the trait `NvptxBarrierScope` is implemented for `Warp`

Strip mining is typed on the amount of work and the lane capacity, and it hands back one chunk at a time along with the predicate saying which lanes live in that chunk:

  // Six work items across four active lanes: two chunks, based at 0 and 4.
  >), i32>>::
      for_each_strip_mined(
          &evaluator,
          (WorkItems::new(6)?, ActiveLanes::new(4)?),
          |index, active| {
           // ...
          },
      );

Hopefully that gives the flavor of it.

Given the massive demand for GPUs for LLMs, what sorts of work do you expect to economically benefit from utilizing GPUs more?