My vet has just successfully castrated Bruce (bar some nibbling on his part!), my Conti buck at 4 months old. As soon as plums made an appearance, essentially! The vets were perfectly happy to do him that young.
He's with his sister, so it was imperative for us to get him done as soon as possible as we're leaving Nyssa until she's older and stronger as it's a more invasive surgery, obviously, before getting her done - at the vets' recommendation.