Their homebrew crypto seems very shaky:
https://cs.au.dk/~jakjak/master-thesis.pdf
https://eprint.iacr.org/2015/1177.pdf
tl;dr: they tried to use SHA-1 as a MAC. This is something of a crypto 101 mistake. Had they even used HMAC they'd be in much better shape. Worse, even after this was pointed out to them and people started writing papers about potential attacks, they have stood by their shaky design, refusing to update it or even admit mistakes were made in the initial design.
But even worse than that, end-to-end encryption is off-by-default, and users must opt into it. Why?
https://telegram.org/faq#q-why-not-just-make-all-chats-secre...
"This allows Telegram to be widely adopted in broad circles, not just by activists and dissidents, so that the simple fact of using Telegram does not mark users as targets for heightened surveillance in certain countries. We are convinced that the separation of conversations into Cloud and Secret chats represents the most secure solution currently possible for a massively popular messaging application."
Putting aside the fact that if Telegram's cryptography were properly implemented, an outside observer shouldn't be able to tell whether or not end-to-end encryption is being used or not (i.e. Telegram does not provide proper separation of data-at-rest versus data-in-motion), this seems to be the "I don't need encryption because I have nothing to hide" argument, but perpetrated by what's allegedly supposed to be a secure messenger.
The real reason why Telegram doesn't enable end-to-end encryption is pretty clear: they don't have the features to provide a good end-to-end encryption experience: They don't support end-to-end encrypted group chats. They don't have encrypted backups like Signal and WhatsApp.