I'm thinking of ending my relationship?

Love is complicated but absolute. If you don't love being around him or are having to change your self or habits to make him happy, then that is grounds to end it. You need some one who accepts you. Be your self and you will end up with some one that will make you happy.