Hmm, a real puzzle. Essentially you've lost approx 20 mph by adding a droop and PD. It seems to me that (1) the motor is OK--4500rpms @ 3500' elevation in a 21' boat sounds about right. (2) the pump is loading OK if you re getting a 100' roost 20' high. If the pump was cavitating, I think your rpms would be higher, and your speed even lower. I've had junk in my pump and had a very loose pump before, and the problem isn't from 30mph to 50 mph, it's from 0 to 15. (3) I'm not sure your boat could hold enough water to slow its speed by approx 20 mph unless it was more than ankle depth--and then it would be pretty noticable.
It sounds to me like something must be dragging in the water--maybe the droop, the box anchor, or like I read one time, "A guy went to the lake for the first time to run his new boat. He launched the boat , idled out and proceeded to run it around the lake. However, he was very disapponted with the top speed which was much lower than he had expected. Back at the ramp he found out that most people disconnect the boat from the trailer when they go boating." That old trailer was just causing too much drag.
I think I would take off the droop, although I've never seen one cause this much of a loss of speed.