It sounds like you've already worked out what you want to do
I'm just going to add my spin on it anyway.
When you're thinking about him, its very clear that you already know what you think is best for him.
The issue that is stopping you is issues about yourself, not about him. You're scared of losing him (which is completely understandable), but that's an issue only for you.
Part of being a good animal owner is to look at their needs and what is best for him, which, from all you've said, seems to be a neuter.
If anything happened to him under a GA he would't know. He's not going to worry about it prior to it, and he won't feel any pain or suffer under the GA. If he dies, you are the one that carries the pain, and you will spend your time worrying. He won't suffer, it will be you.
If he died under GA then hopefully, in time, you would find comfort in the fact that you did absolutely what you thought was best for him (which is all any owner can do). If you kept him by himself because you were selfish and scared to get him neutered then likely, later on, you would then feel guilt for that (especially when he died), which can be very painful.
Now, I can see you've made your decision and your decision is based on your complete love for him, and putting his needs before yours. That makes him very lucky. You're a great owner and in the unlikely event that anything happens to him, at least you will know you always tried your hardest and always tried to do what was best for him. He is very lucky to have you