Regardless of how long it took to derive, the JPEG standard is still pretty straight forward and obvious. Perhaps more so in hindsight?

1) Convert image to YUV color space so that brightness (Y) component can be compressed separately from color (UV) components to which humans are less sensitive.

2) Transform each component (YUV) to frequency domain, then throw away image fine detail, i.e. high frequency components, according to desired level of compression. This obviously is the key idea.

3) Encode remaining frequency components to as small a size a possible

Once the general idea for this approach was conceived, it seems a rather minor step to try DCT vs the more obvious FFT for the frequency encoding. DCT gives slightly better compression. The approach isn't dependent on Huffman encoding for step 3) either - any encoding scheme would work, so experimentation would also have worked there to see what gives best compression on some set of test images.