Earlier quoted context omitted.
It still matters because in your example it will affect how smoothly the computer responds once it gets the user input.
But how much does that matter? If you're scaling to 1000s of users then yes. If you have a GUI for a monthly task that two administrators use, then no. The less something gets used the longer the payback time on the initial development.
Fine, but be honest with yourself and admit that you are contributing a lot to making the lives of those two admins miserable.
It doesn't matter if I'm using your software once a month, or once a day. If it's anything like typical modern software, it will make me hate the task I'm doing, and hate you for making it painful. In fact, shitty performance may be the very reason I'm using it monthly instead of daily - because I reorganized my whole workflow around minimizing the frequency and amount of time I have to spend using your software.