I'm not even sure it was ahead in its UX; it had a three button mouse, and daily operations needed all three mouse buttons. I had to use them for years at school and I feel you could never quite be sure what the third button would do. In some cases, it was like what shift-click does today, in other cases it selected menu items without closing the menu, in other cases it moved windows without bringing them to front and in yet still different cases it opened a _different_ menu to the one the middle-click did.
For menus, I feel it was the worst of all systems. The Mac and Amiga had menus consistently at the top of the screen, and the Mac was good for discoverability in that it showed you the menus were there without you having to click a button. Windows also did that, but menus were attached to windows (bleh). RISC OS was worst of all, _every_ menu is a context menu, including app-level menus - and you got different menus depending on whether you middle-clicked on the icon bar icon, or you right-clicked on the icon bar icon.
There was no standard file requester, _everything_ was drag and drop; to load a file, you had to drag it onto the application (although yes, default file associations allow you to double-click it). To save, first make sure you've got a filer window of the directory you want to save to open and visible on screen, then middle-click in the window of the file you're working on, navigate to File -> Save -> a tiny box with a file icon appears, you get to type the filename, then you have to _drag_ the file icon to the folder to save. And if you accidentally mouse-out of that box while typing the name, you lose the name.
The OS was also ridiculous in some of its APIs, particularly that there were a million and one things under the calls OS_Byte and OS_Word - yes, really, API calls all clustered together because they return a byte or return a word. It's a design holdover from the original BBC Micro's OSBYTE and OSWORD calls. There's also a pile of crap multiplexed behind "VDU" calls, and much like terminal emulators, there's a lot of behaviour you can invoke by printing specific control sequences to the text screen, including mode-switching.
It had a weird system where _all_ OS calls were either "I'll handle errors" (e.g. SWI XOS_WriteC) _or_ "let the system handle errors" (SWI OS_WriteC), which in most cases means that if the OS call ever had an error, it stopped and exited your entire program. The problem with this approach is lots of programmers chose to write software that falls over at the slightest provocation, rather than think to deal with every error and decide how much to deal with recovering it. So, for example, let's say you've been working in a paint package on your masterpiece, you save to disk, and there's a read/write error. Goodbye masterpiece.
You can get a flavour of its programming environment from http://www.riscos.com/support/developers/prm/
It also had its own filesystem metadata craziness; there were no file extensions, but rather file type metadata saved separately (the Mac also had this madness), and it also saved the "load address" along with the file.
Nonetheless, what I did like about it was:
1. That the whole OS was ROM-resident and you can boot a device with no media needed at all, within 3 seconds of turning it on. AmigaOS was _nearly_ all ROM-resident, but nonetheless required a boot disk to get to Workbench (all you need on that boot disk to get to Workbench is a ~200 byte program that launches it; it was clearly a deliberate choice to insist on a bootdisk, and I think it would've been better if it didn't)
2. That it pioneered "an app is a special kind of directory", so you can keep all your app's assets inside a folder. Mac OS at this time was using the awful resource-fork system to do this, but by Mac OS X it had seen the light and create apps and resource bundles
3. That it had a built-in BASIC interpreter, and this was a very fine BASIC because it had a full assembler built into it and it had fantastic BASIC-to-machine-code interoperability. You could write all the bits that needed to be fast in assembler, while writing the rest in BASIC. There were even a few commercial games released written in BASIC+Assembler.
Overall, AmigaOS was a much better OS than RISC OS, but I do still have space in my heart for the plucky British operating system.