Just so you know, there isn't always a tax break for being married, unless one of the partners is making significantly less than the other or is a stay at home parent.
In many cases, you can actually save more money by filing separately, and instead of getting legally married, just become "domestic partners". Health insurance treats domestic partners just like spouses (so you can still add each other to the primary person's insurance if you want).
Many of the other "benefits" of marriage can be solved by intentionally creating a living will where you state how you want medical issues to be handled, and where you have named the other person as your benefactor if you die. You can even list your domestic partner as the benefactor on your life insurance policy.
As for kids, you don't lose your parental rights just because you aren't married. It's still your kid, and you still owe child support regardless. There are plenty of people who are divorced and have kids. Being in a domestic partnership with kids works just fine.
Honestly, I don't even know why anyone gets married these days. The laws regarding marriage in many states are so antiquated and nonsensical. A lot of the laws and protections in marriage are designed around the antiquated idea of stay-at-home moms who can't work or provide for themselves. That's not the modern world we live in anymore.
My significant other and I have been in a relationship for over 5 years now, and we have a 3 year old kid. We keep almost all of our finances completely separate (with the exception of a shared savings account just for household and childcare expenses). She makes her money, and I make mine. We track all of our shared household and childcare expenses using Splitwise, and we file our taxes separately. When we do our taxes, we figure out who gets the bigger refund that year (when filing for head of household and adding the childcare expenses), and then we split the refund difference.
Everything is totally fair, and everyone pulls their own weight in the relationship. We split all of the household duties, and we split things like childcare duties, too. For example, she watches the kiddo all day on Saturday (giving me a day off), and I watch him all day on Sunday (giving her a day off). We also play World of Warcraft together in the evenings after the kiddo has gone to bed. Life is great!