does this happen in both manual and automatic modes?
If its only in automatic mode, I would definitely check the cabin air temp sensor. Cabin air temp and coolant temperature are the 2 main inputs to the HVAC control head for running the blower speed in auto mode. It keeps the blower pretty slow till coolant temp comes up then itll ramp it up until the cabin temp gets closer to the set temperature on the control head.
The cabin air temp sensor is in the headliner right above your head in the drivers seat. No, thats not a microphone, its a temp sensor with a fan pulling air over it
If its doing it in manual mode (you're setting a specific fan speed) then there is something else going on. Could be a bad speed module, bad blower motor, or control head.