Even if your brother don't see him often and yes he won't say anything to anyone about her sister, you want to be in the safest position. So tell your brtoher about what that boy has been upto. He is there to help you, and you want to be helped! So what's the problem? Well, actually he or your father (i.e. family members) are in best position to help you; and we here can suggest only what to do.
I am actually worried about how they would react.
Well, InshAllah, ill let them know abt it.