Seasonal storage and thermal batteries are pretty easy and cheap to construct. The materials tend to be cheap commodities. All you need is space to construct them and a way to produce the heat you use to charge them.
For example, I know of an experimental setup in the Netherlands called CESAR (https://materialdistrict.com/article/battery-natural-stone/) that uses basalt rock. The basalt is heated up using solar energy in the summer. It is stored in an a metal box insulated with wool that apparently retains heat for months/years. Long enough to last the winter. You 'discharge' by pumping cold water through it. Warm water comes out. Very simple. It has been running for a few years and it seems to be cheap and work as advertised. Relative to burning gas, this looks like a nice idea.
Converting heat to electricity is more tricky and the efficiencies tend to be not great. Of course if the storage is cheap enough that might not be that big of a deal. Efficiencies are also a challenge with geothermal energy. But it's a great solution for heating buildings. This particular solution seems to work around this using molten salt which allows for a relatively large temperature gradient. But without having to drill kilometers into the earth's crust.
Probably over time, cost will decide which solutions end up getting used where and how. A good insight here is that different storage solutions serve very different purposes. You don't use lithium ion for seasonal storage. It's way too expensive for that. But it's great for short term grid balancing needs and seems to be popular for that. Vice versa, thermal mass would be probably bad for balancing the grid it's great for storing lots of energy that you use up slowly during the winter.
It's a tradeoff between how much energy you need stored, how much space you have for storing it, and how quickly you want to charge/discharge. It's telling a lot that a relatively expensive solution like lithium ion batteries are actually cost competitive with things like gas peaker plants. We can do probably do better long term. There is no shortage of good & already validated ideas in this space. All we need is good old engineering and manufacturing to prove products in the market at scale.