I would have though that a good CF card would easily outstrip Sony MS Duo for speed. You will only notice it once the camera buffer fills up, though.
Good CF cards are quite cheap now at about £30 for 4GB of Sandisk Ultra II memory. It may not be worth going to CF if you have loads of Duo cards, or loads of other devices that could utilise your cards.
I have a Canon 400D, and use an SD to CF converter (similar to the Duo to CF one included with the A100), as I have a lot of SD memory. I have some Sandisk Extreme III CF cards as well now - these are faster than the SD cards, but not always noticeable in everyday use.