Personally I would do so straight away. I would not waste any more money paying for consultations with a Vet who appears to have minimal knowledge about Rabbits. In Veterinary Medicine Rabbits are classified as an exotics species. During Vet Med training only a very small part is dedicated to Exotics; and that includes all Exotics, not just Rabbits. So whilst small animal Vets, ie the ones likely to treat domestics pets, might be excellent with regards to their Dog/Cat knowledge, their knowledge about Rabbits, Guinea Pigs etc could be minimal. To further their knowledge about Rabbits it is up to the individual Vet to engage with further learning via self funded CPD courses.Some Vets will admit that they do not have a huge amount of knowledge about Rabbits and offer to refer the client on to a more experienced colleague. But unfortunately a few will just blunder on, not really knowing what they are doing. I would say those sort of Vets are in the minority though, I am certainly not bashing the Profession per se.
Are you in the UK ? If so if you state your approximate geographical location some RU members might be able to recommend a Rabbit Savvy Vet near your location. Also, the RWAF have a list of Rabbit Savvy Vets
https://rabbitwelfare.co.uk/recommended-rabbit-friendly-vets/
Once you have read the link you could call a few local Vet Practices to ask relevant questions to try to determine if any of their Vets are Rabbit Savvy. Unfortunately at the moment quite a few Practices have stopped taking on new clients. There is something of a staffing crisis within the Profession these days. That said, I have recently located a few Practices near me (SW London) who are taking on new Clients.
I really feel that it would be better to cut your losses with your current Vet. Once you register elsewhere the new Practice can request your Rabbits clinical records from the previous practice. But given how obvious it is that your Rabbit has an abnormal growth on his genitals and has been exhibiting symptoms for some time I feel sure that a Rabbit Savvy Vet will want to crack on with getting the problem sorted out.
Good luck
