Most oldfags dropped off at gen 5, as that was when most people aged 5-10 during Pokemania (Red/Blue-Gold/Silver) had hit ages where they most likely weren't playing Pokemon religiously (estimated age ranges of mid teens to mid twenties). The games had just given us nice remakes of Gens 1 and 2, and Gen 4 mainline games were noticeably poorer quality due to design fatigue and decisions like expanding cross-gen prevos/evos. I can't speak for everyone, but I just didn't bother with Gen 5 at the time because at nearly 500 Pokemon, the series felt very full and the push for more event Pokemon and cross-gen trade evolutions in a generation I had next to no friends still playing the games killed any drive to pick up Gen 5.
I have fond memories of attempts to get 32 people met in the underground at the summer YMCA, and failing because the games were out of vogue with my age group. It only makes sense that a few years later the games were dead to all but the tightest enthusiasts.
Coming back to it now, I have a hard time playing them as the amount of dialogue is insane. People give gen 7 a hard time for cutscenes but the rambling attempts at philosophy in Gen 5 are an utter snore, making black and white the FIRST pokemon games I have never even reached the elite four on. Adding an exclusive dex was interesting, but the games themselves were a slog for me. The sequels were better, and I was able to finish those. They saved the late-phase DS releases from obscurity. I firmly believe if the sequels had not come out, gen 5 would be like gen 6 - they would exist, and nobody would have too much to say about them one way or the other.