An anecdote on what you might be missing. There is nothing concrete that not using org-mode would cause you to miss out on, it is rather the possibility of being able to access one of countless workflows that have been built on top of it.
When I started to use org-mode it was because I needed to generate a gant chart and I wanted a text format. I haven't used that functionality every again. Since then I have used org-mode for a variety of things, but never all at once, always a little piece at a time. Typeset a thesis in latex? Execute analysis code in a reproducible way? Write DRY bash scripts and tangle (compile) them? Run and log the output of one-off commands on remote servers so that if you ever have to do it again you have a record of it? The list goes on and on.
Last week I was told that I needed to start keeping track of the time I spend working on different tasks. Despite keeping todo lists I have never used the org clocking functions, ye there they were there, just a C-c C-x C-i and C-c C-x C-o away. Integrated into my workflow when I needed it with all of 30 seconds of effort and two key combinations of overhead.
Emacs is useful because countless others have encountered problems the likes of which a single person only rarely dreams. Not only that but they have probably already implemented a solution that is fairly easy to find once you encounter the problem and in 80% of the cases can be used unmodified.
Yes, you and all the rest of us are missing something. It is impossible NOT to miss something and no tool can solve that for you. The question to me is whether you want to spend your time learning a tool and integrating with an ecosystem where you never have the possibility of benefiting from the things that others did not miss.