He is probably right. It can only be an axle seal or brake fluid. If you rbrakes are good and your fluid level has not gone down then its your axle seal.
Just remember, its a piece of rubber around metal. It could have ripped, piece of debris, or could have just worn out. They are not that bad to replace.
How many miles you have on it?