Thanks, but I think I've fixed it for now... I pulled the cable checked it all out, and didn't seem to even have any wear. So I rotated the bracket a little, well quite a bit, and lined everything up as best I could. And it's staying now. Wasn't to hard, I thought it was going to be a lot worse. Thanks to everyone that had ideas, I pretty much tried them all.
And it felt weird with the lever switched, from flipping the bracket, so I didn't go that route.
Sent while on the run, or sitting at work bored