Shall I avoid him from now on?

Don't say 22 is late. It is not. Do you really want to lose your virginity to someone that is not in it for the long haul? If he is looking for a casual screw, that is fine, but when someone loses their v-card it should be special in some way. You know there can be some other way besides to achieve ual satisfaction too, you know. (Maybe oral) All in all if you think this dude will end up hurting you just drop him. Don't regret it.