Live data from Hacker News

Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism

Post reply on HN

Print view View original

1–3 of 3 posts

Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism

#1

Post by yzh » Wed, Jan 17, 2024, 3:48 AM UTC

Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism
github.com

Re: Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism

#2

Post by ufotalent » Fri, Jan 19, 2024, 8:31 AM UTC

An interesting playground to understand the scheduler is here https://huggingface.co/spaces/sail/zero-bubble-pipeline-para...

Re: Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism

#3

Post by QPHutu » Fri, Jan 19, 2024, 8:39 AM UTC

ufotalent wrote: post #2

An interesting playground to understand the scheduler is here https://huggingface.co/spaces/sail/zero-bubble-pipeline-para...

Cool!

Post reply on HN