You're wired incorrectly somewhere - your switch does nothing but start the engine, right? Somehow your power is bypassing your switch, look under the dash and follow your wires from the switch perhaps you can find where the power is coming from. If that doesn't work pull fuses one at a time until the engine dies, then follow that circuit.