Earlier quoted context omitted.
Signal having the contact list means that they'd be subject to legal (and extralegal) process to obtain the entire contact list for everybody using the service, which is untenable for them. Again: Signal is not Discord or WhatsApp; these are different services with different primary objectives.
Because the app constantly prompts for contact-list access, Signal's software-on-device definitely has the contact list. And, that software regularly re-sends that encrypted list to Signal's servers' SGX enclaves for their contact-discovery protocol. So whether or not Signal, or some entity near/around it, "has" the contact list is a matter of how much users trust Intel™ SGX® (as well as the chain of processes that d…
What they're moving towards is a design that looks like what Apple did with their HSM quorum system. The contact information we're talking about is encrypted clientside, but with (usually) a memorable pin. Without countermeasures, memorable PINs are very easy to attack; SGX allows them to artificially limit guesses. As a user, you retain a security dial on this: you can use a more complicated passcode than a 4-digit pin if you don't trust SGX.
Obtaining the whole database Signal maintains gives you ciphertext that you need to mount attacks on user-by-user (and to make those attacks, you'd have to break SGX). It doesn't simply give you the plaintext SQL database other messaging systems collect.