Race condition in pioupload.py prevents uploading to Atmega32u4-based boards

[MakerMatrix]

I was not able to get my Atmega32u4 sketches to upload, at all, even though the exact same avrdude/com port settings worked (albeit only about 50% of the time) in the Arduino IDE.

Long story short, there is a race condition in WaitForNewSerialPort(), when these AVRs do the dance of transitioning to a new serial port on reset. It is my current feeling that util.get_serial_ports() is necessary but not sufficient to narrow down precisely when things need to proceed to avrdude. Currently, it moves much too fast on my Win10 system. Windows is still "doing stuff" to the serial port and thus avrdude cannot open it, always failing with:

avrdude: ser_open(): can’t open device “\.\COM5”: The system cannot find the file specified.

Of course you can't wait too long to move ahead, because then the bootloader sketch times out and avrdude dies with a different message. So this is a pretty fussy thing.

I currently have worked around it by inserting sleep(2.5) inside the if p not in before: statement, where the new port is found. Uploading now works without a hitch on both Leonardo, and a custom Atmega32u4 board I've got on hand - far more reliably vs. Arduino IDE. I hold reset, click upload, and let go when I see the "Waiting for the new upload port...." message.

This seems to be somewhat widespread, as there is a discussion on the forum with several victims over the past years. I made additional comments at the bottom there:

https://community.platformio.org/t/uploading-to-micro-leonardo/10935