Yeah... I wouldn't bother about it... it's more or less like someone talking bad to you about someone else: you can't really trust on it. The most you can do is stay alert and see by your own eyes, but never trust on it.
Anyway, just something that happened to me recently: I always had small reputation, like 3 dark green dots (mostly for my "noob guide")... until recently, when
I posted a joke, nothing bad. It was _probably_ that person to whom the joke was directed, but I don't know and don't care.
Then, probably the same person (there is no way to tell) rated me four times with bad rep. and then I was full red ("officialy hoffed"). I didn't give a s**t, if you allow me that. And then, there is nothing like others helping you. Look at the picture:

I say: Just do what you think is right. Forget the rest.
