Not sure what particulat problem this could be, but I know that "BOOT0" selects the boot mode. Quite common case is when you have wrong code inside (that configures something bad, e.g. disables JTAG/SWD), you have to go into Bootloader mode and connect then over SWD. This is in case when you use jtag/swd and y default work in flash boot mode. So going into STM BL is a measure to stop chip from booting so pins remain correctly set up, and then you can safely connect over jtag/swd.
Is this a known issue with stm32sprog? It soulds like something left from STM BL initialization that breaks you software. Something like pre-configured interrupts, vectors, registers. It is common problem in embedded world when you have this kind of "stages". I mean that obviously the values after cold reset (and boot0 at 0 that skips the STM BL code) are fine for your application. But when STM BL passes before your code it modifies some registers and initial conditions are now different, so your code fails. The obvious solution is to detect what is different, and to take measures inside your code (before enabling interrupts) to fix it.
Where is the hand:
- when your code just starts?
- when interrupts / peripheral modules are configured?
- when interrupts are ENABLED?
I suppose you have some kind of gpio / printf debugging features so you can test and identify the right point.
How about giving SWD a try?. It is supported under linux, and the adapter is quite cheap (look on STM site for ST/LINKv2, I beleive it is something like 25USD). You can use it to program and to have real on-target debugging - isn't it more convient for you? I hope you have those 2 pins free for connection.