My team develops our company's own homegrown internal CRM. It manages basically everything about the company, and contacts themselves is a very tiny part: there're marketing campaigns, followups, accounting, incoming/outcoming calls management, our products' entire licensing/subscription system (neatly integrated into everything else), tech support, inbox/outbox, activities, contacts, leads/lead events, analytics, the backend of the shopping cart on the site (products, pricing, discounts), the reseller portal, integrations with various vendors including payment systems, A/B testing etc. -- to name a few, off top of my head.
Before me, there was an attempt to migrate to Dynamics CRM but our current CRM we've being developing for 15 years has so many custom rules/quirks/etc. eventually they abandoned the attempt because of the huge scope and the difficulty to reimplement millions of lines of custom logic in a proprietary CRM.
Before I joined the team, the CRM's development didn't follow best practices and now it's very hard to maintain and update, due to architectural problems aka "big ball of mud". Another issue is that it was not designed to be extendable, every small rule the business comes up with is hardcoded in the code, and so there's no way to configure things somewhere in the UI with least amount code possible.
So if I was to migrate our CRM to an open source engine/core, the following is a list of things I would want to see:
1) good modular internal design/API so that it was possible to easily add custom complex logic/flows
2) design/API which is extendable and doesn't have hardcoded logic/doesn't assume your company's flows
3) allows to configure as much as possible from the UI
4) an ecosystem of plugins/some sort of marketplace, with, again, solid, standardized API