It can also be due to when the xbox is chucking a lot of stuff around in its memory: you can get lags when the ram can't keep up with the cpu.
depending on your spec, pcs suffer from this so this is what you might be seeing. Some games that use the xbox harddrive (like morrowind and halo) "glitch" or stutter when they're loading -- better this than load from dvd. It's as close to seamless as you're going to get.
Vice City just suffers from slowdown. Try driving the tank in GTAIII and see what I mean.