I want to second this. Don't let fear of what might happen make a regret. I have very strong regrets from my youth where people I cared about died and I didn't make the efforts I should have to make better last memories. Just don't do it.

I understand not wanting to have him the way he is now potentially be your last memory of him. When my grandmother died, I refused to see her in her casket for the same reason. Just prepare yourself for how he's going to look before you arrive, and the sting may not be as bad. Also, it might help if you try to remember small things, like his smile when he sees you've come to visit him.

It might also not be so bad to talk to your family about what happened with your grandmother and if the worst happens, to give them a moment to make sure he's okay to be seen before you enter the room. There's nothing wrong with this. Just, please, OP, don't suffer from the regrets I've had to suffer from. My grandmother died almost 15 years ago and I still sometimes start crying uncontrollably when I remember that I blew off the last chance I had to give her a hug and tell her I love her and tell her what she means to me. Please, don't be me.