AFAIK, DVD players don't actually do widescreen switching as such, you have to set the player to 16:9 or 4:3 manually.
Widescreen DVDs are recorded anamorphically, as you said. As I understand it, when the DVD is mastered, the picture is squashed horizontally into a 4:3 frame. When the DVD is played, the player has to expand the frame to 16:9 (if the option is set in the menus). If you play a 4:3 DVD, that will also get expanded to 16:9.
I suppose that the reason why they can't do auto WS switching is that the player has no idea if the DVD is anamorphic WS or 4:3, since the actual data is the same.
Maybe WS switching was left out of the DVD spec on purpose, to make it a truly international standard. Maybe other broadcast systems don't use line 23 signalling, or is it the fact that line 23 doesn't actually exist on a DVD (it's inside the "overscan" area for PAL 625).
Presumably for scart pin8 switching to work reliably, the device still has to decode the PAL line 23 signal from the original frame.
If I'm wrong about this, will someone please explain how it really works