The Bosch BMS is a pretty clever little device, it governs not only the charge discharge cycle but also keeps track of the state-of-charge, and reports the remaining capacity in % to the controller via a CANBUS link, which - annoyingly - also features as auhtorization to release the motor, using some challenge-response system that to date nobody has managed to crack (though some people spent a lot of time trying to do so). So you can output voltage from the pack all you want, without the proper authentication your motor will not turn.
So that's why you really need that Bosch BMS. Then the bad news: the Bosch BMS is on the edge of what's safe, the FETs are underpowered (just a hair under rated maximum current and voltage), not cooled, there is a 40V Schottky diode there which regularly operates at or even slightly above it's ratings, there is a single chip DC-DC convertor which also isn't cooled properly and which tends to blow. So I don't particularly trust the Bosch BMS from a safety perspective, I've seen a whole bunch of these now (about 15) and a good number of them had blown up. And that's with the standard battery. The one that I'll be hooking up to it is four times more powerful. Any short circuit protection in the old BMS is going to be terribly undersized so I need something in series with the Bosch BMS to ensure that if there ever is a short (say in the motor) that it's the battery that gets disconnected well before the wiring goes and I simply do not trust the Bosch BMS not to fail 'closed' in which case the whole thing will turn into a giant fuse. That's the kind of scenario that I can really do without.
On more than one pack that I've opened I've found evidence of small fires, typically BMS balancing wires that shorted out (there are no fuses there, obviously) and that could have ended quite badly. The wires from the pack to the motor are substantially thicker so then the question is what cuts out first: the BMS or the wires. I simply want to play that as safe as possible and aim for the disconnect to happen fast enough that the wiring doesn't light up.
Another option is a very fast fuse in line with the battery terminals, that's definitely a possibility as well (and easier to mount and less chance of it in turn breaking than yet another BMS in series). Anyway, work in progress and to be continued, nice to find someone on HN who has a lot of knowledge about this!
Waiting for a while to see if there is a self discharge is a very good method to spot flawed cells before you assemble a pack but such defects can also develop at a later date, so please do keep a good eye on your pack, especially if - as I suspect - you are using reclaimed batteries, you can't be 100% sure that they have never been deep discharged.
Safety is one of the main reasons why I decided to go with brand new high capacity brand name cells, yes, it's a lot of money, but even so, the whole thing will cost about as much as a 600 Wh commercially available pack for this bike and have more than 3 times the capacity (and a better form factor to boot, and will last a lot longer).