Apple do a couple of bits to help you do the AV trick but this is a pricey route. As Steve says it depends what TV/Amplifier you have. They do a Component lead (five plugs to Red x2, Blue, Yellow, White) and a Composite lead (three plugs Red, White, Yellow).
Both leads are £35 in store but come with a usb/charger (I think). As far as I can see both should give you full AV through your system for vids and songs but unfortuately I don't think they will transfer Apps onto the big screen like you can with the equivalent PSP leads (now that would have been cool). The Universal Dock is another £35 and not essential though it does have a 3.5mm socket as well leaving the iPhone free. It also comes with a basic remote control.
In theory the Component leads should produce a better picture but either way don't expect HD!
I'm still debating which way to go. The simpler Composite lead is more universally available and offers the opportunity to play back through, for example, a hotel TV and should even work with a scart adapter.
I've not tried any of this myself yet (maybe this weekend

) so check it out first. IMHO this an area where the usually competent Apple staff are a bit sketchy - care required.