To understand what's going on, do a simple experiment. Fill the sink with water, take a knife, tape it to a bottle and put into the water. Now try to spin it with two fingers. Take another bottle and tape a big cutting board to it and try to spin it. If you use the bathtub you can also give them a push and observe how they track.
This is the primary behavior of the two types of underwater shape, and it can't be changed. The long keel boat is hard to spin, it always wants to go straight. The finn keeled one is easy to spin and will rotate with minimal force.
Turning force comes from the rudder and depends on rudder area, water speed over that area and distance from the pivot point. The farther away the rudder is from the middle of the boat (asuming the pivot point is there, wich of course it isn't) the more authority it has because the lever arm is longer. This means you can use a smaller area to achieve the same turning force. Same thing with speed, the faster the water speeds over the rudder, the less area you need. This you know because as speed increases you need to make smaller rudder movements to achieve the same results.
Your long keeled boat has a small rudder at the end of a longish keel, but not at the extreme end of the boat. This means low rudder authority, the boat wants to go straight and anytime you let go of the rudder the water flowing along the keel pushes it in the lowest drag position wich is inline with the keel. Simply put (it's more complicated actually) the boat overrides the rudder.
The short keel boat has better rudder authority (bigger area and longer lever), the rudder turns the boat and stays into a turn because this is now the lowest drag position. This time the rudder overrides the boat.
Waterspeed over the rudder can be created two ways, you move the boat trough the water, or you accelerate water over the standing rudder. The second case happens when the propeller sends water over the rudder, and your two boats exemplify this perfectly, one has no propwash at all and the rudder "won't bite" until you have a certain speed, while the other responds to the tiller sooner.
In sailing circles a "horn style" is known as a semi-skeg rudder, and is seldom found on smaller boats. That's because if the "horn" is to actually support the rudder it creates complications in building the structure. Cantilevered rudders are much simpler to implement and can easily be made to whitstand whatever force is desired. Of course the "horn" doesn't have to actually be structural in relation to the rudder, and in reality many skegs are only play pretend in this department, more hydrodynamic aid (or hindrance) then actual support.