Oh yes, I can understand feeling that way. I always feel guilty for pressuring people to scene with me when I think they don't really want to, too.

It's not really fair if they sorta jump a situation demanding your involvement on you without really talking to you about it. It's usually not that hard to just go, "Hey, mind if we do this." And likewise if you always play along, they should play along occasionally, too. Or negotiate the circumstances of how something could happen.