You don’t have to answer to everyone for everything you do. You’ll make mistakes and do things that other people don’t want you to. But there’s a big difference between things we wish someone didn’t do (like switch lanes in front of me in traffic), and things that they actually shouldn’t do (like drive along while texting). Just because someone doesn’t like what you did doesn’t mean you were wrong to do it.
If you’re a person who cares a lot about other people and likes to make sure you don’t upset people, this can be hard to get your head around. But you can’t please everyone. Don’t hurt yourself trying. For every person you find that wanted you to do a particular thing, you’ll just as easily find one that wanted the opposite from you.
By the way, it’s not selfish to do something for yourself. It’s selfish to do things for yourself at the expense of others. Taking care of yourself can actually benefit others. You only have a finite amount of energy and capacity for giving out before you find you have nothing to give. If someone expects you to give more, that doesn’t mean you actually can, any more than you can get a thousand miles out of one tank of petrol just by wishing you could.
Listen, it’s not your job to keep the seven billion plus people on this planet happy, no matter how conscientious you are. You have to draw a line and focus your attention on the things you can physically do. Be considerate of others, but not tied to the idea that they have to like what you do, because you can’t predict their reaction even if you are deliberately thinking of them.