When I approach a new creative coding project, it is invariably because I want to learn how to use a specific technology that I have heard of.
So, for example, I wanted to build a "real time" stack using RabbitMQ and websockets with Tornado. I also wanted to build a front-end with Backbone. These were all kind of new and cool at the time. So what kind of app would use this tech? How about an app where you can submit a "currently playing" where you are, e.g. in a nightclub or at a concert, and have people within a certain radius get live updates as they are pushed? This was a really fun project and it also gave me an excuse to dive a bit more into PostGIS and nginx than I had in the past.
Years later, I wanted to have a go with some of the cool machine vision tools I had been hearing about. I also wanted to have a go at building a front-end with Web Components. So I built an app that would let you build a specification with a nice UI and send it to a worker that would create a video using six ML models working together. This was a really fun project and also gave me an excuse to have a go building a lightweight Python JSON API backend I would later reuse quite a few times.
In both cases, the driving force was a perceived need that had arisen from my day job. Specifically, I was aware that there was tech out there that was rumoured to be able to solve problems I had or thought I would soon have, and I wanted to get a handle on what the "unknown unknowns" and "gotchas" were involved.
This was also what got me working with VueJS for the first time, a technology that does extraordinarily well at what it says it does, better even than I expected. (I think I built a vanity site for my music work at the time?). Vue became my go-to at my day job for the purposes, so much so that I found myself instigating a rewrite of an existing AngularJS front-end soon before I left the company. I feel very confident their AngularJS code has all been burnt since then.
I'm a Web developer so I don't know if this approach translates to other fields of Software Engineering. Web is one of those things where new tools are cropping up and being hyped all the time, and staying abreast of developments in the tech is kind of part of the job.