Do not file, machine, or otherwise open the pin bores. Likewise, do not alter the bushings.
The issue is that something is holding the Diverter from seating properly in the bowl socket.
Judging by your first picture, you bowl socket has a healthy wear pattern. Please also see Picture 1 of C bowl below. Usually these bowls have so much wear that the O-ring hardly touches at all. The O-ring often has quite a bit of interference, that is why I said to file the O-ring. Never did I say to alter the bores or the bushings.
Sometimes the lands (see Picture 2), on either side of the O-ring, might rub and/or prevent the Place Diverter ball from seating down properly where you can install the pins. If you see indications of contact with the bowl socket, file these areas down. You should have no aluminum to aluminum contact.
The last place where you might run into interference would be in the area shown below in Picture 3. It appears that, in your third picture, you have already ground that surface down.
The pins are all interchangeable between your original equipment and your new Place Diverter pins, the only difference being the length. There shouldn't be any discrepancy in diameter. Place Diverter's machine work is all really nice, but if for some reason, the pin doesn't fit (tight) in the Ball, you might need to gently sand (by hand) out the powder coat.
I would recommend not removing the Teflon O-ring from the Ball. The Teflon will stretch and permanently distort.
You need to look and see where the interference is, and adjust accordingly.