Sort of, though Wii and Wii U are a bit more complicated than that so this somewhat of an oversimplification. The ELI5 answer is that some hardware components are directly upgraded and can run in a compatibility mode, other components are just the original hardware thrown in separately.
New3DS is the most recent and most notable exception. It's directly upgraded 3DS hardware, but the CPU downclocks to run at 3DS specs on all legacy titles (and there are almost no native New3DS games so this upgrade was pretty pointless). Softmodding can unlock the full clockspeed, and most games do work fine this way but there are a few rare bugs.
I expect Switch 2 will just be the same architecture upgraded, because that's a lot easier to do now, while the old style of true redundancy would inflate costs too much today. It's also worth noting that Switch titles already expect variable performance in order to support handheld and docked modes, so I doubt much would break if allowed to overclock. But I could also see Nintendo not even trying to support it if even one bug might exist somewhere.
That is 22 first party releases. The answer to your question was "yes", not "not really".