Security and usability are always going to be on a spectrum.
Even in the San Bernadino case, if the FBI hadn't changed the user's iCloud password, just bringing the phone to a trusted WiFi network would have caused it to automatically back itself up to the cloud. Even assuming the phone were completely 100% locked down perfectly, as long as you're sending backups to Apple that aren't encrypted, you're putting the data one subpoena away from the FBI (or from a hacker who breaks into Apple).
So say you don't use iCloud, or Apple starts a service where the backup is encrypted by your password. Your password needs to be really strong for this to work, and Apple needs to use something like bcrypt or better to hash it, but say they do that as well, and you have an long random password.
Then you can put a chip on the phone that refuses to ever be updated, and that implements the password lockout logic (try 10 bad PINs and it wipes the key). What they're asking Apple to do now would simple be impossible.
But you're allowing iOS itself to be updated, right? So if, for instance, you're discovered to be a terrorist, Apple could push a patch to the main OS that simply grabs the key after you've unlocked the phone and sends it to them. No more need for the secure chip; just decrypt the flash directly.
Or easier, the patched OS could just slowly upload all the data on the phone to a backup server whether or not the user opted in to such a backup plan.
If there's a lot of data (tons of photographs, for instance), they could even have the phone wait until it detects a known wireless access point to trigger the backup. The FBI could then arrange for that access point to be active near you and your device, and it could connect and upload to a server sitting in the FBI van nearby.
Also keep in mind that just about every release of iOS was designed to be impossible to jailbreak. And it seems that iOS 9 can still be jailbroken. This is typically done using an OS vulnerability that's exploited; the FBI/NSA could easily use those approaches to hack your phone (though all would require an already unlocked device, so they'd probably need to get you to run something via social engineering, but some of the hacks only require you to click on a specially crafted link on a web site...).
Finally, remember what I said about security and usability? Yeah, now if you forget your password and need it to be reset, you no only lose the data on your phone, but all of your backups. Oops. All of that and you're still not protected from a coordinated attack.
It's great for Apple to do whatever they can to resist giving the government the keys to everyone's privacy, because that can be abused. But it's safest to consider anything you typed into a computer or phone to be something that might get posted publicly.