I'll first ask the question of do you remember if you had any ear infections as a child - and were they painful at all?
It sounds like that audiologist still wasn't specialized enough - otherwise I feel like your story would have extended differently; high demand mainstream audiologists, including the mainstream audiologist profession, don't seem to have a certain lineage of knowledge that I dramatically benefit from first when I was 23 (now 41), when I was in essentially a high-functioning Asperger's state - to where my hearing had devolved into a hyperacusis state - a severe hypersensitivity to sound - but where prior to that I had the same symptoms of difficulty with conversation, and busy rooms with lots of noise was extremely mentally draining-fatiguing, not realizing it was putting my mind on overdrive drying to actively focus and hone in on sounds vs. it autonomously happening.
Have you ever heard of Berard AIT (Auditory Integration Training/Therapy) or the Tomatis method? There's a book on the sound therapies called "Hearing Equals Behavior: Updated and Expanded."
There's a non-standard audiogram to check for imbalances in the hearing. The standard audiogram is they just pick say 30 Dbs volume and check various frequencies in each ear at that frequency, and if you can hear that - then "great!" The non-standard audiogram checking for imbalances checks for HOW LOW-QUIET of a sound you can hear at different frequencies, and interestingly, with 100% accuracy you can predict a set of behaviours that a person will have if they have an imbalance at certain frequencies like 1000 Hz; not all frequencies have associated behaviours with them.
An example of an imbalance would be if at 1000 Hz in your left ear the lowest sound you could hear was 15 Dbs, but in right ear you could hear at 10 Dbs - an imbalance of 5 Dbs, but where the idea is that the body-brain-mind is a system of finding homeostasis and equilibrium, so it should be able to have it so sounds are heard at the same level - evenly, save for actual physical damage. This is just a simple example and there can be high peaks and valleys that show up in the non-standard audiogram.
I have a similar story as yours. My issues with sound were almost identified in Grade 2 when going from a kindergarten setting, where there were no real performance or attention expectations, to Grade 2. The teacher noted I was having trouble paying attention. They thought maybe I had hearing issues, it was a private school and so they brought in an audiologist. My hearing was fantastic! So indeed, unfortunately, 30+ years ago especially they didn't consider that my hearing could be "too good" - where sound was overwhelming me; so I was hypersensitive to sound, arguably hyposensitive to touch and other senses, and it was medications in my late teens and early 20s that caused my hearing to get super hypersensitive - to the point where I was in what I consider a torture state for 8 months - where even the sound of blinking was painful, or at least that was the sensory I was associating the pain with - until I was forced to do my own research and eventually stumbled into Berard AIT.
There's also pre-care questionnaires that some practitioners offer - a checklist for behaviour of a child, and also of an adult, where I had ~80% of the behaviours both as a child and as an adult, e.g. preferred to sit in the back or corner of a classroom, essentially as there'd be less directions noise would be coming from, had trouble relaying a story or following instructions, etc.
Adult checklist: https://www.aithelps.com/auditory_care_adults.php
Child checklist: https://www.aithelps.com/auditory_care.php
NOTE: these places started offering "home programs" to send some audio equipment and the specially modified music, however I won't personally trust them until I've had the chance to do what I'd consider to be thoroughly done research to compare the original high-quality sound equipment that would produce the absolute highest quality of frequencies vs. what the rentable-shippable equipment produces.
Someday I need to write a chapter of a book or perhaps a whole book on my experiences of it all - the before and after state, the blocked development process (e.g. autistic state) that got unblocked and the development process that began to unlock - essentially I was blocked from processing emotions properly, so I arguably had a lifelong backlog of unprocessed memories with emotions associated with them needing to be labeled to be organized, as well as PTSD from many very intense traumatic years.