Should I watch out for this guy?

You need to sit down and talk to him about it. Don't be afraid to do so. He seems sweet if he's never done anything to hurt you or disrespect you. Just tell him that you feel intimidated by him and his experience that he has had with women. If he has been heartbroken that many times, maybe this time you can be the first woman that he's been with to not break his heart. Just try talking to him and telling him how you feel about it.