That's because you haven't trained.
When I first started, despite swimming on a fairly regular basis, the longest I could hold for was 90 seconds and to hold that long was agony. But in doing the exercises, learning to relax during a hold and learning that the feeling in the diaphragm isn't a lack of oxygen but, rather, a build up of CO2 which doesn't need to be obeyed, I improved. Over the course of the next 6 months doing 15-25 minutes 3-4 mornings a week, I got my max time up to just over 7 minutes. That requires a full breathe-up, lying prone and I can only achieve that if I haven't eaten in a while since digestion seems to require oxygen. I do the 3-4 minute holds because they're about my limit when sitting upright and not going through my full preparation.
With training, I'm fairly confident I can get anyone under 60 who's reasonably healthy to the point where they can hold for 4 minutes. Really fit people in their 20s can usually train up to 8-10 min holds. What truly gifted apnea specialists can do is nothing short of astonishing [1].
To start, the first thing that most people can learn pretty quickly is to resist the urge to breathe. When CO2 builds up, it causes involuntary contractions of the diaphragm. These usually start to occur 1/3 to 1/2 of the way through a person's maximum hold time, but don't signify an actual need to breathe. So a good rule of thumb is to shoot for double the time when you notice your first contraction. But to ignore what feels like a reflex to breathe takes practice, and that's where the exercises fit in.
My favorite exercise is a form of what's known as a CO2 table...an exercise where you're never short of oxygen, but you don't give your body adequate opportunity to expel CO2 and it builds up. The exercise I do follows the form inhale 1x, hold 4x, exhale 2x where x is whatever level you're currently at. In your case, if you can hold for 30 seconds, you'd start at around a 4 second level, which means you'd inhale for 4 seconds, hold for 16 seconds and then exhale for 8 seconds. Repeat that cycle for, to start, about 15 minutes. I like to lie down when I do my exercise, but some people like to sit, just make sure to relax and breathe normally for about a minute before starting. As you improve, you can up the level while maintaining the 1x-4x-2x ratio and also increase the overall time. My current timing is level 17 seconds and do the cycle for 25-30 min.
Eventually, after building CO2 tolerance, you work on what's known as O2 tables, which accustom the body to lower levels of oxygen, but those are more advanced and should probably be done with supervision. Two other notes to anyone wanting to experiment with this kind of thing...never hyperventilate before attempting a breathe hold, which purges CO2 from the body and can result in actually running out of O2 before the CO2 build up causes an urge to breathe, and never do wet training (in the water) without a qualified training partner. The last one should go without saying, but some people still push their limits and drown.
[1] https://www.youtube.com/watch?v=L89kY5ewgqU