I just want to have one bike, but want to use it all throughout the year as effienctly as possible

So I want a frame that can both run fatbike tires and 23mm roadbike tires, and everything in between

I know running 23 mm roadbike tires on a fatbike frame will look funny but I don't care. I just want to have multiple wheelsets that I can swap out for whatever the environment demands.

Why should I not do this?