Well, you've left out one type of racing that is one of the least expensive and safest ways to participate: autocross.
Aside from having an appropriate vehicle that's properly maintained, you only need a helmet, which most organizations have on hand as rentals. Past that, high-performance (or track-only) tires would be the next thing.
By "appropriate," I do not mean "modified for racing," I mean something that's not off-road-centric and/or liable to tip over. Competitors range from karts to Miatas to SSVs, in different capabilities and levels of modification. You're slotted into a class that suits your vehicle, so you won't see a kart against an SSV.