Most of the time when you lose all speeds at one time it is a loss of ground to the blower, usually at the mode switch or the 3 wire to it. If it is a bad connection sometimes it will be intermittent. Check these links for testing & photo of a bad connector.
Look at the 2nd photo here of a bad switch.
Here for melted wires, look at yours carefully since they are not always this bad but still losing ground connection.
If the wiring & switch look good you can test the rest of the circuit by removing the connector & putting a jumper between the middle & bottom (2 heavier) wires. If you get all 4 blower speeds with the key on then the mode switch is bad.