Virtue signaling goes both ways. Yes, there's plenty of middle class progressive virtue signaling in having a little free library, but there's also plenty in bashing them as a threat to the public library system. Progressiver-than-thou is a seriously annoying thing.
My own gripe against them isn't the virtue signaling, but rather the KonMari problem. People who read tend to treat books as sacred. Throwing books out is a sin, even if the books are, frankly, junk. I live in one of those neighborhoods they describe, with a little free library or two on every block, and they are often stuffed with garbage books that someone couldn't bear to throw out - old Danielle Steele and Steven King novels, obsolete computer books, etc. It's okay to throw books out, it really is. The magic isn't in the book, it's in the reading.
That said, there are other ways to do Good Work with books besides little free libraries and the public library system. My spouse works with a nonprofit that distributes books for underprivileged children. Every time she buys something on Amazon, she also buys a children's book - exclusively books that feature non-white characters. Her goal isn't just to get books in the hands of poor children, but to get minority kids images that look like themselves, and get the white kids images that look like someone else.