Here in America, we almost never keep a stallion unless they are being used 
for breeding.  If your stallion is not covering mares or being used for 
collecting semen, there is really no need for him to be a stallion.  
Stallions are a very definite liability when it comes to management.
However, gelding a stallion at a more advanced age may not solve your 
nervousness problem at this point in time.  I have observed geldings who 
were gelded at an older age, say 6-7 years, and still exhibited the same 
characteristics of nervousness and non-recovery as you have described.
My guess is that if you can make it through the qualifying stages of your 
lower distances, and start doing longer distances with this horse he will 
probably settle down and start to work, whether you geld him or not.
Another observation I've made, and confirmed by interviewing people who 
ride stallions in endurance, is that stallions seem to be less "honest" 
than mares and geldings.  My "honest" I mean they will not give you their 
best effort, but they save some energy for themselves.  They keep some 
energy in reserve.
Just my two-cents on stallions.
Linda Van Ceylon
lvanceylon@vines.colostate.edu
phone:  970-491-1428