Afraid you do, since he actually saw the horse. In our area (Southern Oregon) a farm call is around $45. So, if he went to the horse, that doesn't leave much of your $100 for an exam.
Our local favorite vey does a nice basic pre-purchase for about $100, but docs I've used in other parts of Oregon were $300+ without x-rays and ultrasound.
Better pay up and chalk it up to experience. Did you buy the horse?