Yes, you are over-reacting. Just change your phone number, if you want to avoid any more calls. They probably got it because you have it posted in one of your social profiles online, or it's in a phone directory online that they just searched and found.
Change your email address as well. They rarely if ever email anyone, though.
They will not go looking for you in real life, so relax. Nobody will show up at your door. They never confront any victim in real life. It is all done online, and they are only interested in CASH, not YOU personally.
Chill, brother.