I think that the actual differences in visual quality are so uneven that you cannot choose one console over the other based on that. The Xbox has obvious advantages over the PS2 when it comes to texture management and graphical effects, but they are not used entirely to their best advantage.
Out of the best looking Xbox games and the best looking PS2 games, the Xbox emerges the victor only based on specific elements the PS2 does not manage and that are still only of interest if you like the game. The Xbox does not eliminate all the flaws seen on the PS2, and some flaws seem more common on the Xbox, such as unstable frame rates and tearing.
The Xbox has a significant advantage over the PS2 when it comes to sound. Since the sound processor is powerful and fully independent, it has enough capacity to encode Dolby Digital 5.1 in real time - something the PS2 does not (there are games which use low spec DTS 5.1 on the PS2, but they are very rare). The support for Dolby Digital 5.1 has been made universal by Microsoft so most if not all Xbox games support it. Some PS2 games have cut scenes (one ...or more

) in Dolby Digital 5.1 but in-game sound in Dolby Surround.
At the end of the day, in order to decide which system to go with, you must take a very close look at the existing game library and the upcoming games. Be prepared that there will always be flaws on the PS2 like low-res textures, a lack of advanced shading effects, and 2.0 or matrixed surround. If you feel you can live with that and it appears the PS2 has a library more in your interest, the PS2 might be more of interest.
Not that some developers have chosen to support all the current systems and will release many titles on all three consoles. These titles will vary slightly depending on which console is prioritized and who is making the game on each system. The Xbox may earn better graphics and sound, but when it comes to content it might be quite random which system gets the best version.
If you feel split between the consoles - and a lot of serious gamers do - I think the question is not which one to get, but which one to get
first. I have both the Dreamcast, PS2 and Xbox and would not have traded any of them for the other.