I have found that our 23 sails hersel quite nicely if the conditions are right. Waves knock her off pretty quickly, though.
I am a bit confused about the rudder being "hung at an angle". If it is not vertical, it would explain what you are describing. I am not a physicist but am pretty certain that the rudder should be vertical!
As far as the weather helm goes, I assume that you have played with the sail settings and trim. Flatteining the main might help. A little bit of weather helm is good. Do you know if your sail sizes are right?
A clove hitch around the rudder (using the stern cleats) is a nice cheap substitute for an autohelm!