Reducing the amount of requests and roundtrips required to render the page, and reducing the latency on those requests is key.
For example, you use Google Fonts. For that you need to load CSS. So that's three roundtrips for rendering your page: 1) your page, 2) the Google Fonts CSS, 3) the font files.
We've build a WordPress plugin called PhastPress (fast press) that helps you reduce the request count and those roundtrips without building your own theme. https://wordpress.org/plugins/phastpress/
Among other things, it inlines the critical CSS needed for rendering the page, defers all script loads, and optimizes images. Cuts page size by ½ on the stock theme, and the amount of additional roundtrips to 0.