well, whats left of your ridder (white chunk of plastic) can be removed if youre not going to replace it....i dont think its entirely your problem but i'd remove it.

Brad,  If it was like mine, the pin is molded into the rudder and to remove it you need to get the shorter pin.  Otherwise you still have the pin in the way of the flow.  But it really wouldn't have much effect the way it is.  Just my 2 cents.

Here are a couple of photos of my Place Diverter in the full down and up positions from a lower angle looking upwards.

FULL UP:


FULL DOWN:


You can see how the water from the pump flows back and across the rudder in the lower photo.

i have had only larger boats myself...i prefer a "slow speed rudder".... when your either at slow speeds or up to speed it seems to handle a lot better than not having one...