Traditionally, VTxxx terminals only supported 8 colours and a 'bold' variant, while PC hardware supported 16 colours and no bold, so it became traditional to map the 'bold' variants of the basic 8 colours to the brighter half of the PC 16-colour palette. Unfortunately, VTxxx doesn't have a way to set the
background colour to bold (why would it, boldface is a thing for text, not backgrounds) so, people had to resort to hacks like setting the 'bold' flag to get a bright foreground colour, then setting the 'inverse' flag to switch it to the background.
Some terminals actually properly support all 16 colours directly, without bold/inverse hacks, but the standard "xterm" and "vt100" terminal definitions don't, for compatibility reasons. If you put something like this in a shell startup script:
TERM=xterm-16color
export TERM
...then you'll be advertising to any programs you run that your terminal supports all 16 colours.
The 256-colour mode is a far more recent invention, and I'm not sure exactly why it came about; I suspect Thomas Dickey had a spare byte in the xterm framebuffer data structure or something and a 256-colour palette was the obvious choice. It is a 256-colour palette, by the way; by default you have the original 16 terminal colours, a 216-colour RGB cube, and a bunch of greyscale values to pad it out to 256 colours, but there's an escape sequence to set the RGB value for any of the 256 palette entries to any value.
If you want to specfically know how to pick colours from the 256-colour palette and you don't mind hard-coding control sequences, the official reference is the xterm documentation: http://invisible-island.net/xterm/ctlseqs/ctlseqs.pdf
Specifically, you want the section named "Functions using CSI, ordered by the final character(s)", then scroll down to the function ending in 'm', named "Character Attributes". To summarize, to set the foreground colour to palette entry N (0
ESC[38;5;Nm
...and to set the foreground colour to the palette entry closest to the given RGB tuple (where R, G and B are between 0 and 255), send:
ESC[38;2;R;G;Bm
To set the background colour, it's the same as above except with "48" instead of "38".
If you'd rather use a library to talk escape codes rather than hard-coding them in your program, you'll need to set the TERM variable to a setting that describes a 256-colour terminal (such as xterm-256color), and then apps (like Vim) and libraries (like terminfo) that respect that environment variable will report the full 256 colours as available.
Modern versions of ncurses come with command-line tools that look up the current terminal definition in the terminfo database and will extract information from it; in my terminal I can find out how many colours the active terminal definition reports with:
$ tput colors
256
...and I can print a prettily-coloured message like this:
$ tput setaf 2; echo "hello, world"; tput sgr0
(where "setaf" means "Set ANSI Foreground", 2 is a palette entry number, and "sgr0" is a historical phrase meaning "reset all graphics attributes to defaults"). For a list of names you can give to tput, see terminfo(5).