>>86871231The TL;DR is that his entire life was a lie, and the bald guy emotionally manipulated him to give him a superiority complex just so he could break him, and then indirectly use him to hurt Sakura
More specifically, he found out he was from a lineage of Mages early on in his life, and tied his entire sense of self-worth around that, studying and researching tirelessly through the books and materials laying around in his mansion. At some point Sakura gets adopted into his family and he assumes it's just a normal orphan, since he feels superior being a super special mage from a super special lineage, he feels bad for her and decides to look out for her, which lasts several years.
Then at one point he stumbles into the REAL atelier of his family, where he sees his father and grandfather instructing Sakura, and learns that not only is she going to be the inheritor of his family, but also that he himself is actually a complete dud incapable of magecraft. It is also mentioned in passing that this is the LAST time his father ever talks to him, since the cat is out he just doesn't feel like bothering dealing with him again.
Coming to terms with the fact that he's worthless (which Zouken consistently hammers on), Sakura pitying him, the only real mage in town scorning him, being essentially soft-disinherited and abandoned by his family, having no one he feels close to (Shirou is passive as fuck, so he doesn't approach him), etc., leaves him in a downward spiral, that leads from him being a pretty great dude with a shitty attitude, into the completely worthless mess devoid of any redeeming qualities we see in-game.