Maybe it's the fact that Diamond was my first Pokémon game (I was a fan way back in Gen II mind you, I just didn't have a handheld to play the games on.) but I actually LIKED Barry. I found he was both someone you loved to hate for his overconfidence, but he was also a character you liked for his quirkiness, and I quite enjoy how he was basically your typical speedrunning tryhard.
I'd have to say my least favourite Rivals were the ones in X and Y. I mean, they were great characters on paper, but in practice, they felt kinda bland and badly paced. To list an example: there is one scene where everyone meets up, and rather then exchange friendly banter, Serena/Calem basically says "All right, let's all battle.", and then they just... battle. No response, no "THAT'S A GREAT IDEA HERP DERP.", they just turn and battle. And when the battle is over, everyone just won't stop rubbing the PC's dick/cunt over how amazing he/she is rather then talk about themselves and how they're doing. It's infuriating, because they had great character ideas with opportunity to create character development like what they did with Silver or Hugh.