In universe, it is stated the limit is how many Pokémon one can take care at the same time.
Catching more would teleport the new catch away.
I know this comes from in-game rule to make it entertaining. How is it enforced? I'll say standard Pokéball-shop technological implementation, but what/how to prevent bad faith actors to craft balls without said limitation and storm a shop by unleashing 257 zubats inside?