Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debugging bare-metal STM32 from the seventh level of hell

jpieper.com

Post reply on HN
Print view View original

31–40 of 70 posts

Re: Debugging bare-metal STM32 from the seventh level of hell

#31

Post by jpieper »

khoozer wrote: post #28
matrix_overload wrote: post #6

>It seems that the ADCs on the STM32G4 do not like to be turned on in rapid succession, and if they do, bad things can happen like having the prescaler flipped to a different value without it showing in the corresponding register. This sounds very, VERY much like an incorrectly configured clock where some of the peripherals would end up with a clock frequency slightly above what they were designed for. Will work 99%…

Or it was just faulty firmware and he did hit the (almost) correct conclusion: ADEN bit on g4: "Note: The software is allowed to set ADEN only when all bits of ADC_CR registers are 0 (ADCAL = 0, JADSTART = 0, ADSTART = 0, ADSTP = 0, ADDIS = 0 and ADEN = 0) except for bit ADVREGEN which must be 1 (and the software must have wait for the startup time of the voltage regulator" No errata needed, it's clearly stated that…

Actually, if you look at the firmware at the time, the proper procedure was followed as far as I can tell. All of the necessary bits were set and checked with the appropriate delays where required.

https://github.com/mjbots/moteus/blob/dcb900c92ffd5d5c8f5405...

Notably, the datasheet is largely silent on interactions between different ADCs during initialization.

Re: Debugging bare-metal STM32 from the seventh level of hell

#32

Post by varispeed »

kragen wrote: post #24
varispeed wrote: post #20

How did they manage to get STM32s? My friend's small business had to close, because key MCU is not available anywhere and Chinese sell it for the price of the end product. Makes business no longer viable. He spent over a month designing board for another MCU that was available at the time, but as soon as he was ready to produce more the alternative was gone too. It's like chasing own tail. Big corporations can post h…

Probably the beginning of that month of designing would have been a better time to order the microcontrollers than the end of it. The answer to "how did jpieper get STM32s" is "he ordered them from Mouser in early 02021": https://news.ycombinator.com/item?id=32383312 What kind of state intervention are you thinking of? I spent some time trying to think of a state policy that would reduce the risk that small businesse…

The simplest intervention would be to offer state backed loans that could be used for back orders and to be repaid 3-6 months after the delivery.

It's not ideal, probably it will create further delays, but it would put small business on the level playing field with the big corporations when it comes to orders.

The criteria for eligibility may be difficult to define too, so that fraud is minimised.

Maybe such surge of demand could incentivise MCU manufacturers to secure more fab capacity.

State could also make scalping illegal. For instance buying MCUs with a sole purpose to hold on them and then resell at inflated price would be illegal. This could be done by setting maximum margin someone could ask, that should be no higher than Mouser or Digikey have. So anyone selling MCUs at 10x the price would have them confiscated and get fined. Foreign online stores that facilitate such sellers like Aliexpress would risk being sanctioned and banned.

Re: Debugging bare-metal STM32 from the seventh level of hell

#33

Post by kragen »

varispeed wrote: post #32
kragen wrote: post #24

Earlier quoted context omitted.

Probably the beginning of that month of designing would have been a better time to order the microcontrollers than the end of it. The answer to "how did jpieper get STM32s" is "he ordered them from Mouser in early 02021": https://news.ycombinator.com/item?id=32383312 What kind of state intervention are you thinking of? I spent some time trying to think of a state policy that would reduce the risk that small businesse…

The simplest intervention would be to offer state backed loans that could be used for back orders and to be repaid 3-6 months after the delivery. It's not ideal, probably it will create further delays, but it would put small business on the level playing field with the big corporations when it comes to orders. The criteria for eligibility may be difficult to define too, so that fraud is minimised. Maybe such surge of…

The loan idea might work, but it might not. Generally big corporations are better at fulfilling criteria for eligibility than small businesses are (they can afford larger legal departments), and it might just drive up prices without increasing supply. That seems to be what happened with state-backed loans for higher education in the US, for example.

The price-controls idea would create shortages, at least locally; Mouser and Digi-Key, prohibited from allocating scarce parts to those with more willingness to pay, would allocate them according to other criteria, such as customers with the largest established volume. Companies in other countries would have a major edge because they'd have access to distributors who weren't at risk of having their stock confiscated. The upshot would be that companies in whatever country instituted those price controls would be unable to compete on the global market because they couldn't buy on Aliexpress. (I've seen this kind of thing a lot up close and personal because I live in Argentina.)

Re: Debugging bare-metal STM32 from the seventh level of hell

#35

Post by varispeed »

kragen wrote: post #33
varispeed wrote: post #32

Earlier quoted context omitted.

The simplest intervention would be to offer state backed loans that could be used for back orders and to be repaid 3-6 months after the delivery. It's not ideal, probably it will create further delays, but it would put small business on the level playing field with the big corporations when it comes to orders. The criteria for eligibility may be difficult to define too, so that fraud is minimised. Maybe such surge of…

The loan idea might work, but it might not. Generally big corporations are better at fulfilling criteria for eligibility than small businesses are (they can afford larger legal departments), and it might just drive up prices without increasing supply. That seems to be what happened with state-backed loans for higher education in the US, for example. The price-controls idea would create shortages, at least locally; Mo…

Fair points!

Re: Debugging bare-metal STM32 from the seventh level of hell

#36

Post by nottorp »

> The initialization sequence for the ADC is documented as requiring a wait until the ADRDY flag is set, so the fix is just to wait for that for each ADC in turn before enabling the next one.

Hmm? Ok, it may be the clock or something else subtle but if they tell you to wait...

Re: Debugging bare-metal STM32 from the seventh level of hell

#37

Post by Taniwha »

matrix_overload wrote: post #6

>It seems that the ADCs on the STM32G4 do not like to be turned on in rapid succession, and if they do, bad things can happen like having the prescaler flipped to a different value without it showing in the corresponding register. This sounds very, VERY much like an incorrectly configured clock where some of the peripherals would end up with a clock frequency slightly above what they were designed for. Will work 99%…

I think it's more likely that the data needs to cross a clock boundary and multi-clock synchronisation takes time (especially if metastability is potentially an issue) - they way you make it safe is by holding the data you're synchronising stable for multiple clocks and synchronise a strobe signal across the clock boundary and then back again

My guess is that the circuit they're using gets confused if you start a second write when the first is not yet done

Re: Debugging bare-metal STM32 from the seventh level of hell

#38

Post by jpieper »

nottorp wrote: post #36

> The initialization sequence for the ADC is documented as requiring a wait until the ADRDY flag is set, so the fix is just to wait for that for each ADC in turn before enabling the next one. Hmm? Ok, it may be the clock or something else subtle but if they tell you to wait...

In this case, the firmware did wait for the ADRDY flag. It just waited for all 5 to be set, then moved on to enable all 5 ADCs simultaneously. The easy fix was to just do those serially instead.

Re: Debugging bare-metal STM32 from the seventh level of hell

#39

Post by jpieper »

Taniwha wrote: post #37
matrix_overload wrote: post #6

>It seems that the ADCs on the STM32G4 do not like to be turned on in rapid succession, and if they do, bad things can happen like having the prescaler flipped to a different value without it showing in the corresponding register. This sounds very, VERY much like an incorrectly configured clock where some of the peripherals would end up with a clock frequency slightly above what they were designed for. Will work 99%…

I think it's more likely that the data needs to cross a clock boundary and multi-clock synchronisation takes time (especially if metastability is potentially an issue) - they way you make it safe is by holding the data you're synchronising stable for multiple clocks and synchronise a strobe signal across the clock boundary and then back again My guess is that the circuit they're using gets confused if you start a sec…

That is possible, although here the consecutive writes were to different ADC peripherals. The ADC peripherals do share some common configuration and triggering, but I believe are otherwise largely independent.

Re: Debugging bare-metal STM32 from the seventh level of hell

#40

Post by bradfa »

Tangent: reading this, all I could think was "you're allocating memory in a motor controller?!?" and then "you're using interrupts in a motor controller!?!". Surely the bug write up was good and it is interesting, but all of the actual hard real time systems I've ever seen or talked to engineers who actually worked on them always avoided interrupts and never allocate memory. Both of these activities in hard real time code can produce very unexpected results which may be similar to what the author found with enabling the ADCs in rapid succession.
Post reply on HN