IMHO audio is much more subjective. Good, well respected amps generally do not exhibit neutral sound, but have a small amount of tonal lift. They often change their tonal balance at different volumes, as many operate at class A at low levels and class AB or D at higher levels. As AB & D have inherently higher levels of distortion, how an amplifier handles itself in this mode often decides its character.
Having impulse tested a few back in the 90's when I worked as an audio designer, Most consumer amps had a warm (Lifted) bass and a little roll off at the high frequency end - usually as a result of an ultrasonic filter. We built an amp with dead flat response from less than 10Hz to over 200KHz using a switched mode PSU and some meaty output devices. Honestly ? It sounded "OK", but improved massively when we warmed it up a bit and actually increased the distortion at higher levels!
Valve amps are another subject entirely. The most critical component is the output transformer, imparting the frequency response (Never flat) and also the graceful overload (Magnetic saturation being much nicer than electronic clipping) Of course, good, well matched valves are needed to drive the thing in the first place!
It there was one amp design that worked perfectly, we would all buy the same one! Speakers do not present a static load to the amp, so the way the amp drives different speakers will also change. Hence, certain amps suit certain speakers.
In short, great sounding amps are well designed and use quality components. But, and its a big but, the speaker imparts much more character upon the sound than the amp driving it.