>>49232305OP I will actually give you a real answer since I was researching this myself a couple days ago after seeing a thread somewhere else. Retro Dodo on YT reviews pretty much every one of these so if you want a more in depth recommendation go watch those.
Anyway the best bang for your buck seems to be the Anbernic RG351P. It's made for GBA games specifically but also plays everything up to N64/PS1. It's supposed to support DS and PSP also but it has very poor performance on those. It's $90-100.
https://anbernic.com/products/rg351p-anbernic-retro-game-ps1-rk3326-64g-open-source-system-3-5-inch-ips-screen-portable-handheld-game-console-rg351gift-2401?variant=40986799407268Or if you want to drop an extra $50 you can upgrade to the RG351MP which has wi-fi and a metal shell.
https://anbernic.com/products/anbernic-new-rg351mp-retro-games-builtThere's also the Powkiddy Q20 Mini which isn't as nice as the RG351P but it's only $40 and roughly the size of a Gameboy Micro.
https://powkiddy.com/products/powkiddy-q20-mini-open-source-2-4-inch-oca-full-fit-ips-screen-handheld-game-console-retro-ps1-new-game-players-childrens-giftsI'm planning to get one of these but haven't decided yet.