The differences are
>exclusive pokémon
>exclusive mega stones
>Xerneas and Yveltal
Since you can easily trade for a female pokémon from the other version on /vp/ or whatever, you should go for the version which has your favorite mega stones:
Pokémon X
>Mega Charizard X (if you choose Charmander from prof. Sycamore)
>Mega Mewtwo X
>Mega Pinsir
>Mega Tyranitar
>Mega Manectric
Pokémon Y
>Mega Charizard X (if you choose Charmander from prof. Sycamore)
>Mega Mewtwo X
>Mega Heracross
>Mega Houndoom
>Mega Aggron
Of course you can always trade stones with someone, but it's harder than trading for pokémon. As for Xerneas/Yveltal, I don't see them as very important since I only catch them and box them, so it's up to you.