Am I fooling myself with this relationship?

You have let him treat you like crap for so long that it has become a habit for him.Unless you stop him from doing those remarks he wont stop.Stand up for yourself now or he's gonna keep doing it.