You can't live like that. You can't please everyone and you can't be liked by everyone, people will always talk and seek for a flaw in you, even if you are a saint. You just need to feel indifferent about such persons and bla bla talk. As long as you don't harm anyone, you shouldn't care what people think or say