The biggest risk of sortition is moving the balance of power from representatives to the appointed/hired bureaucracy. I suspect that non-politicians are much easier to "manage up" against - I fear having it wind up as political lottery winners getting "advised" into going a certain direction.
That's a risk, but I don't think its the biggest even within the "balance of power between officeholders and X" category of risks; I think the biggest in that category is the shift in the balance of power between office holders and professional interest group influencers (lobbyists et al.) who aren't going to be randomly selected short-timers that get swapped out every few years, they are going to be long-term career professionals representing entities with lots of resources, durable interests, and a commitment to achieving them.
OTOH, sortition doesn't have to be a wholesale replacement for elections, either; it can complement them (as it does now, you just expand the scale.)
Say, have an additional house of the (state or federal, though some of these powers would be more familiar and perhaps appropriate at the state level) legislature (the "House of Citizens", say) selected by sortition, and given it some or all of the following powers: (1) To take any bill passed by the other house(s) and delay it by putting a hold of up to 90-days on it, after which the other houses must pass it again before it can move forward for executive signature or veto (if there are any changes to the version passed later, its treated as a new bill, and goes back to the House of Citizens again),
(2) To, instead of #1, invoke the power of referendum against any bill passed by the other house(s), such that if it is approved by the executive (or passed over executive veto), it is referred to a vote of the electorate at large who must approve it before it goes into effect,
(3) To directly initiate an initiative measure, that would go directly to a vote of the electorate at large, and become law with their support, without the need for approval of the other house(s) or the legislature,
(4) To (for a federal "House of Citizens"), on a supermajority vote, propose to the State legislatures amendments to the Constitution, without the other houses (the normal thresholds for ratification would still be required.)
Note that #2 and #3 are powers of the people in many states, but usually require an expensive process of gathering signatures on petitions to invoke, making them more accessible to well-funded groups; a sortition-selected body as an additional avenue could provide an alternate filter that might be less cost-driven.