Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ExecuTorch: Run PyTorch programs on mobile and edge devices

pytorch.org

Post reply on HN
Print view View original

1–10 of 27 posts

Re: ExecuTorch: Run PyTorch programs on mobile and edge devices

#5

Post by dang »

ge96 wrote: post #3

interference or inference?

We've changed the title now to use language from the article itself.

(submitted title was "ExecuTorch: Enabling On-Device interference for embedded devices")

Edit: I switched "anywhere" to "mobile and edge devices" per https://pytorch.org/blog/pytorch-edge/. One does not get away with saying things like "anywhere" in HN titles...

Re: ExecuTorch: Run PyTorch programs on mobile and edge devices

#6

Post by flakiness »

There is also PyTorch mobile [1]. I'm curious how these two relate.

[Edit] And here is the answer [2]

> PyTorch Mobile uses TorchScript to allow PyTorch models to run on devices with limited resources. ExecuTorch has a significantly smaller memory size and a dynamic memory footprint resulting in superior performance and portability compared to PyTorch Mobile. Also, ExecuTorch does not rely on TorchScript, and instead leverages PyTorch 2 compiler and export functionality for on-device execution of PyTorch models.

[1] https://pytorch.org/mobile/home/ [2] https://pytorch.org/executorch/stable/intro-overview.html#ho...

Re: ExecuTorch: Run PyTorch programs on mobile and edge devices

#7

Post by liuliu »

Very cool. A few notes:

1. MPS backend uses MPSGraph exclusively, might hit some performance ceilings limited by MPSGraph. s4nnc moved more and more ops from MPSGraph to Metal directly to have better control on both allocation and some erratic behaviors from MPSGraph.

2. CoreML backend uses coremltools, thus it carries all the baggage of that: requiring to generate CoreML model AOT because Python dependency, have no control over memory planning, weight quantization scheme, or where to put the weights. Dynamic shape might further making memory planning worse as static shape is the main use-case of CoreML so far and far better tested.

I would love to see an updated port of coremltools either in Swift or C++ (ONNX's coremltools implementation is in v3 I believe, and coremltools moved to v7 spec?).

Re: ExecuTorch: Run PyTorch programs on mobile and edge devices

#9

Post by iseeyuan »

ai-joe wrote: post #8

ONNX Runtime has supported this for years - https://onnxruntime.ai/blogs/pytorch-on-the-edge

One difference here is that there is no conversion for ExecuTorch. In general conversions introduce inconsistency and are difficult to maintain, so ExecuTorch eliminates that step.
Post reply on HN