Okay, let's go back to first principles.
What you would normally think of as being a CD player contains a number of different components, but you can usefully divide what it does into two separate functions:
1) Read the digital sound data off the disc (i.e. convert a series of little holes in the disc surface into a series of corresponding electrical pulses).
2) Convert the digital data into an analogue electrical signal (i.e. convert a series of electrical pulses into two smoothly-varying voltages, one for left and one for right).
The result of stage 2 is the output from the player (via Phono/RCA sockets, etc.) and is what you feed into the amplifier.
In some high-performance systems you actually have two separate devices performing those two functions. This is not unlike what happens when you have a DVD player feeding a home cinema processor: the player reads a digital signal off the disc, the processor decodes it and converts it into analogue signals that can then be amplified. In such a two-box device, the part that reads the disc is called the "transport" and the part that does the conversion is called the "DAC".
"DAC" stands for "Digital to Analogue Converter" and it's a general name for any sort of device that converts digital signals into analogue ones. Most CD players will have some kind of DAC built in. But you can also buy devices that are
only DACs: they convert digital signals to analogue, but they aren't able to read a disc - they have nowhere you can put a disc. Instead you use a separate CD or DVD player as a transport, and take a
digital output signal from the player, and feed it into the DAC, then feed the output of the DAC into the amplifier.
Now, obviously, the transport part of a CD player costs money. So, if you spend (say) £500 on something that is only a DAC, it will do a much better job of being a DAC than the built-in DAC of a £500 CD player - because in the case of an all-in-one player, the £500 is buying you the transport as well as the DAC.
The obvious question now is, does the quality of the transport affect the sound quality? The answer to that is "sometimes". But certain types of DAC produce sound whose quality is more or less independent of the transport. These are what I'm calling a "reclocking" or "rebuffering" DAC - and I don't really want to get into precisely what that means

but just accept that it means they're not fussy about the quality of the transport.
Okay, so, if you already have some kind of CD player or DVD player which has a digital output, then the best value-for-money upgrade may be to buy a reclocking DAC and use your existing player as a transport.
One example of a reclocking DAC is the Benchmark DAC1, and it's a particularly good value example, which sounds as good as DACs costing two or three times the price, and as good as all-in-one CD players costing 5 or 6 times the price. As I said before, this device is also capable of functioning as a headphone amp and as a pre-amplifier (i.e you can feed the output straight into a power amp). Whether it does as good a job of being a headphone amp as your Solo does, I don't know - that's something you could only determine by trying. But, even if it isn't quite as good, it may well be good enough.
One thing it won't do is act as a headphone amp for any sound the DAC itself is not generating - I don't know if this is an issue.