I'm doing a big Apollo/GraphQL piece for the BBC for a lot of the same reasons OP lists, and have run into pretty much all the same issues with Apollo that OP has. Its rough because Apollo does so much for you, its wonderful, but there are so many sharp edges and pitfalls - from the buggy devtools to the lack of/incorrect docs, major unanswered github issues & constant API breaking changes even on patch/minor version…
In the past I had to write or figure out my own Redux solutions to store data from queries in the state and now that comes out of the box with Apollo. If I want to manipulate this state locally, I can just do a client.writeQuery in a standard JS function (no resolver bullshit). In the rare event I want purely client-side state, I can use the same method with the @client directive. I've never really needed resolvers as I can contain the logic in my `update` handlers or state manipulation (action) functions that look and behave like standard JS.
Caching is great out of the box. You get all variations you might want (cache-first, cache-only, cache-and-network) and it's very easy to configure. Not sure what problem you ran into. Maybe server-side?
State persistence is a whole different story. I've found it's hard to do well due to Apollo's ROOT_QUERY combining all pieces of the state. It requires almost an all-or-nothing approach or a lot of hacking to get it to work. I've spent a lot of time searching for proper solutions and even built my own, but they've all been insufficient. Considering local storage persistence brings XSS vulnerabilities as well, I've decided to focus my efforts on performance for now which helps both returning and new users.
Personally I love Apollo for its state management and will continue to integrate it in more projects. My biggest gripe at the moment is writing maintainable client-side queries. The code gets quite verbose, especially when you try to do proper (TypeScript) typing without splitting up your code into tiny fragments all over your codebase. And keeping the types in sync with the queries and server-side schema takes a lot of time and effort. I've mostly been thinking about solutions to auto-generate client-side query types, this would increase type-safety as well.