I agree, the set up/ RPM seems wrong. The first thing I'd do is check the tach. MSD and others make a digital tach checker. We check them for free here, you can't tune unless you know the truth. Most of the boat tachs we see are off from 200 to as much as 500 RPM at around 5000 RPM, mostly they read high, just like most non-GPS speedos. About 3 or 4 times a year we have guys come in with similar readings and when we check the tach, we find that they are set to the 6 cylinder position! I can understand that because most car guys with big blocks can't believe that their engine only turns 4200 RPM wide open.
In your case, I don't believe that is a correct RPM reading unless your pump is worn out or assembled incorrectly. With your engine, I would expect more along the lines of 4600.
I also believe that a loader will slow your boat by a few MPH too. We have plenty of 70 and 80 MPH boats out there without loaders. If the boat doesn't unload going down the lake, it will probably slow it. The river guys use them to keep loaded in the rough water of the river(Parker/Havasu) After doing the jetting/ timing thing, we take top and bottom pump pressures to see if the shoe/loader setup is correct. For most loaders, we end up going back to a rock grate to see where we are, checking the speed and pressures as we change stuff. Also, just because an impeller is stamped "A" doesn't mean it still is, it may have been modified and not correctly, or where it should be, and not restamped.
Some of the things that affect RPM in the pump are in no particular order: brand of impeller(legends and Aggressors are "tight"), shouldered or old type wear ring, wide or narrow wear ring, side and end wear ring clearances, age of impeller (beat up / eroded leading and trailing edges, brand of bowl, stuffer or not, clearance from impeller to bowl, nozzle size, age/wear in bowl, blueprinted bowl and/or impeller, loader, shoe, shape of hull in front of suction, angle of suction installation, ride plate angle, droop and what brand/type and length, brand of suction housing, type and fitment of clean out cover(yup, really!), and I'm sure GT will add some more that I haven't thought of!
Back to the engine now. IMHO I would try an RPM Airgap intake or a Dart singleplane intake with an 850 to a HP 970 carb with your setup. Try to borrow one before you buy. Most of our big valved, unshrouded, bowl ported, factory oval port headed, solid roller cammed, 496s run fastest with one of those intakes. They usually run about 75-80 at our altitude and 115 degree heat. Check the tach and bowl pressures, top and bottom before spending any cash. TIMINATOR