This is what comes to mind for me, though I have no "advice" as to whether or not you should accept the position:
I lived in the same house from the age of 3 until adulthood. I graduated high school with some of the same folks I went to kindergarten with. I got married at a young age to a man who only wanted to join the army. He did and we moved around for a couple of decades until we divorced.
The transition at the first duty station was quite hard for me. My then husband had grown up as a military brat and hit the ground running. I had no clue how to do that and he was zero help to me with this major adjustment. For that and other reasons (like a pregnancy where I threw up for 8 months out of the 18 months we were there), I was really miserable at that first duty station and it took me years to realize it wasn't the fault of Texas and to "forgive" Texas. I irrationally hated Texas for quite some time.
I wanted out of my home town and I have no regrets, though I resented my then husband's lack of support at the time (and never did feel he did the right thing in that regard). I returned to my home town during the divorce and I have been here about 5 years or so. I intend to leave again someday (basically as soon as I can arrange it, logistically) and I don't think I could be making such plans had I not left to begin with. For me, it was very much a growth experience and I think my life is vastly better for all of it.
But if you do this, I would suggest you be prepared to help the spouse and kids adjust. Maybe they will be fine. Maybe your wife is more resourceful than I was at, oh, age 21. Maybe she will have no problem finding hobbies, friends, and whatever else floats her boat. Maybe your kids will also jump in with both feet, gleefully. But if they do not, odds are good that they will blame you for their misery. Being sensitive to that and willing to assist will most likely go over a lot better than my ex husband's choice to let me just sink or swim on my own.
Good luck with this decision.