So I can't fathom...

Alter3go

Member
...how there can be an audible difference between modern 320kbps variable MP3s (or whatever Spotify is using for "very high quality" these days) and lossless, FLAC, SACD, or anything else really. Now I know this has been talked about to death, and I have read some of the threads. And I know people have strong opinions about these things. But being the type of scientific person whose immersed himself in 1000s of research papers, I want to address the heart of this question, and that's why I made this thread:

Is there a single, competent, double-blind study, that demonstrates that human beings, even audio engineers and other related professionals, can actually make any of these distinctions? In short, I am not asking for your opinion at all (not that its invalid or anything), or how you view the science of audio and what conclusions you can draw (again, not that any of that is invalid). To satiate my own curiosity, I am asking what double-blind studies of this are on record, how sound the methodology is, the sample sizes used, and what the results were. In the threads I read, I saw studies that said that this distinction could not be made. I want to know if that there are any -large scale- studies that say anything to the contrary.

P.S. Please don't kill me. Thanks.
 

oscroft

Member
I can't point you to any tests, sorry, but I can tell you why FLAC etc can sound better than MP3. It's because MP3, even at 320kbps, throws away parts of the audible signal - in the words of Wikipedia, It uses "inexact approximations and the partial discarding of data". That is, it does not retain the full data that is produced by Nyquist–Shannon sampling, which is the theoretical minimum needed to accurately reproduce sound.

Whether/how higher resolution sound reproduction beyond Nyquist–Shannon can make a difference is subject to considerably more disagreement.

(I couldn't distinguish 320kbps MP3 from FLAC until I upgraded my DAC recently, but now I can. But it depends on the recording, and I still often can't tell any difference. The closest I can get to a blind test is to use a random playlist of multiple copies of the same piece of music, half of which are MP3 and half FLAC.)
 

Jamie

Distinguished Member

k-spin

Active Member
I haven't read the paper but a quick look on Wikipedia found this study by Dr Joshua Reiss of Queen Mary University London (and a member of the Audio Engineering Society):

Over the last decade, there has been considerable debate over the benefits of recording and rendering high resolution audio beyond standard CD quality audio. This research involved a systematic review and meta-analysis (combining the results of numerous independent studies) to assess the ability of test subjects to perceive a difference between high resolution and standard (16 bit, 44.1 or 48 kHz) audio. Eighteen published experiments for which sufficient data could be obtained were included, providing a meta-analysis that combined over 400 participants in more than 12,500 trials. Results showed a small but statistically significant ability of test subjects to discriminate high resolution content, and this effect increased dramatically when test subjects received extensive training. This result was verified by a sensitivity analysis exploring different choices for the chosen studies and different analysis approaches. Potential biases in studies, effect of test methodology, experimental design, and choice of stimuli were also investigated. The overall conclusion is that the perceived fidelity of an audio recording and playback chain can be affected by operating beyond conventional resolution.
The paper can be downloaded here: AES E-Library » A Meta-Analysis of High Resolution Audio Perceptual Evaluation
 

oscroft

Member

larkone

Member
Do the numbers

CD 1411kpbs
MP3 320kbps
Difference MP3 320 is only 22.6% of the bitrate of a CD
It would have to be a seriously well recorded MP3 not to hear a difference on a half decent system.

But if you cannot hear the difference on a half decent system then lucky you because then you only need to buy a cheap hifi that will save you a fortune
 

Ascotbilly

Active Member
@larkone My summarised comments from another thread.

"Hearing Tidal Hifi, High, Normal and can't differentiate, all sound very good as do other streamed sources. Played same track on Stream, CD, LP. Think I hear differences on CD & LP but not sure if its just as I want them to be better."

Hopefully I'll save myself a fortune !!
 
Last edited:

oscroft

Member
Do the numbers

CD 1411kpbs
MP3 320kbps
Difference MP3 320 is only 22.6% of the bitrate of a CD
That in itself doesn't prove anything.

Does it mean, by extension, that 1mbps would automatically sound better? 10mbps? Wherever it is, there is a limit to the bitrate beyond which there will be no audible improvement.

I do agree that CD at 1411kpbs can produce better sound than MP3 at 320kbps, but my point is simply that the bit rate comparison on its own does not demonstrate that.
 

larkone

Member
That in itself doesn't prove anything.

Does it mean, by extension, that 1mbps would automatically sound better? 10mbps? Wherever it is, there is a limit to the bitrate beyond which there will be no audible improvement.

I do agree that CD at 1411kpbs can produce better sound than MP3 at 320kbps, but my point is simply that the bit rate comparison on its own does not demonstrate that.
I didn't say it does prove anything. What I am suggesting is if you cannot hear a difference then it would be sensible not to spend mega bucks on a high end system because it would be wasted money.
 

oscroft

Member
I didn't say it does prove anything. What I am suggesting is if you cannot hear a difference then it would be sensible not to spend mega bucks on a high end system because it would be wasted money.
Apologies, then. It's just that when you said...
Difference MP3 320 is only 22.6% of the bitrate of a CD
It would have to be a seriously well recorded MP3 not to hear a difference on a half decent system.
...I read that as saying the bitrate difference alone was enough to mean CD should sound better.
 

Hixs

Distinguished Member
Tbh I struggle with the idea what people can't tell the difference between mp3 and CD.
 

Alter3go

Member
Tbh I struggle with the idea what people can't tell the difference between mp3 and CD.
My system is ultimately mid-ranged. I have yet to have the privilege of listening and comparing on a truly high-end, audiophile quality system. Maybe I'll find a difference? Who knows. But what I am looking for right now is something that says that that difference won't be in my head, so I have a good idea of the ultimate upgrade paths I want to take. I was a major in Cognitive Science, and I can't even begin to tell you how much our brain does to fill in our reality to account for the gaps in our senses and limited capacity for attention. For instance, our retina, the part of our eye that sees color, has a paltry 5 million cones, equivalent to a 5 megapixel camera. That's, ultimately, very very sh*tty. Everything that you see is rapidly being filled in by what our brain already knows and understands to give our sight the quality it has (do not take this to mean that there is no point in having a camera above 5 megapixels, this is not true at all). Another example is that in that same retina is the end of our optic nerve. This creates a black vertical line in our visual field, which is then filled in by the brain. There are some tricks you can do to see this vertical line. Do some quick Googling if you're curious.

The brain is doing similar things with sound. But it's harder to measure that cutoff like we can with vision (5 megapixels), and just where and how this filling in is taking place. If you want to see what I am talking about, grab yourself some low-end audio equipment. The lower the better. Maybe even your phone. Listen to a song you know well. It will sound like ass, but you will still find some enjoyment in it, and it won't sound too bad. Now listen to a sound you are sure you have never heard at all, anywhere, ever. Find some obscure hipster band on your streaming that has no listens. When you listen, it will sound sooooooo much worse than the song you did know, however bad that sounded, because your brain is filling in the missing pieces of the one your familiar with. This might be undone if you have a really really high expectation of the song you do know and are looking for those missing pieces. So I would recommend passive listening for this experiment.

Anyway, if anyone has more studies, go for it. I'll take a look at the ones posted.
 
Last edited:

Alter3go

Member
Also, thanks to Jamie and k-spin for proving actual studies instead of just opinion. Double thanks for the meta study, which I will take apart when I have the time and energy, hopefully soon. For the purposes of this thread, I am going to say that the 'hypothesis' is that there is a perceivable difference in higher resolutions (higher than the best MP3-320). Therefore, the null-hypothesis is that there isn't. Now we can properly refer to the null-hypothesis ("The null-hypothesis is true/false") and shorten the verbage of our posts.

P.S. k-spin, almost any study can be downloaded at sci-hub.tw. ;) For instance...saved my ass when I graduated university and lost most journal access.
 
Last edited:

Jamie

Distinguished Member
The same blogger also did a 16bit be 24bit test that might be of interest.


In fact I think he's done few over the years, you need to use desktop view on his website to get a search option, I can't see one on the mobile view.
 

Alter3go

Member
The same blogger also did a 16bit be 24bit test that might be of interest.


In fact I think he's done few over the years, you need to use desktop view on his website to get a search option, I can't see one on the mobile view.
You're the best-est! No evidence of discernment there...
 

oscroft

Member
But what I am looking for right now is something that says that that difference won't be in my head, so I have a good idea of the ultimate upgrade paths I want to take.
A couple of thoughts, on each part of that. Firstly, I don't think it's possible for you to get proof that any difference won't be in your head. There are technical arguments that convince me that FLAC can be better than MP3, but that doesn't mean it is. And even if it is, not everyone will hear it. I'd also offer a caution here on tests that have been published. There's one I found a while ago (and I can't remember where, sorry), which offered FLAC vs MP3 tests with a well-designed A/B/X protocol. But it had to be done on the web site using a browser, with whatever signal degradation that might produce. I did the test using a digital output on my computer, into a DAC with a headphone amp, and my best pair of headphones. But the sound quality was poor, no matter which sample I listened to. It also used samples that were clearly badly recorded, with very poor dynamic range. One was Hotel California, and there are some horribly compressed versions of that out there and I'm sure this was one. Another was some simplistic boomy rock thing with no subtle detail whatsoever. My music system sounded a lot better even on lower-res MP3, and there was no way the samples on this test (coupled with the route through the interwebs) were good enough to distinguish MP3 from FLAC.

But the next thing, ultimate upgrade paths... I'm currently nearing the end (well, that's what I tell my wife) of an upgrade path, and the upgrades I've made so far have significantly improved the sound of my system. For digitial music, getting an Audiolab M-DAC to replace my Cambridge Audio DacMagic Plus made a very big difference. The change was spurred by turntable upgrades that made my vinyl listening so much better than my existing digital setup that I stopped listening to digital. The M-DAC upgrade was the change that meant I could hear differences between MP3 and FLAC for the first time (which is no use to you, I know, but I'll just note the event). But here's the thing. At no point had it been my intention to make my system good enough to hear the difference between MP3 and FLAC. That was entirely incidental. The new DAC sounds a lot better than the old with MP3 too, and that was actually my goal. Just about everything that can play MP3 these days can play FLAC, so my suggestion would be to just think about better equipment that should make everything sound better, regardless of format. My DAC will handle hi-res up to 192/24 too, though I haven't played any yet (I need to rearrange things to get a USB connection). I have no idea whether I'll hear any difference when I try it again now (I couldn't with my old DAC), but if it does then it's just more incidental good fortune. At no point have I bought any new equipment specifically to try a new format (at least not since MP3).

The bottom line for me is that the range of qualities of original recordings, mixing and mastering is far more varied than the difference between MP3 and FLAC, or CD and hi-res. So just listen to the music.
 
Last edited:

JoyDivision1982

Standard Member
I find FLACs on good headphones and a DAC sound much more open and airy than 320kbps MP3. If you compare the wave forms of an MP3 and FLAC file of the same recording you can clearly see the difference.

I don't stress over it though, but for my HIFI needs I have no reason at all to use MP3 or other lossy compression. I have more than enough space or bandwidth. However on my phone with its internal DAC I am more than happy with 320kbps Spotify Premium.

I've noticed this in clubs as well, if the DJ uses a laptop it often sounds awful, if they use CDs or viny the sound system comes alive. I suspect this is because they are using low bit rate MP3s on the laptop.
 

phil t

Well-known Member
...how there can be an audible difference between modern 320kbps variable MP3s (or whatever Spotify is using for "very high quality" these days) and lossless, FLAC, SACD, or anything else really...……………

But being the type of scientific person whose immersed himself in 1000s of research papers, I want to address the heart of this question, and that's why I made this thread:...………………...

In short, I am not asking for your opinion at all (not that its invalid or anything), or how you view the science of audio and what conclusions you can draw (again, not that any of that is invalid). …..
So you're not interested in talking about the science behind audio, considering it to be an opinion, but you're okay to talk about peoples opinions that have collated as part of study?

The link that was posted was an interesting read, but I'm not sure it conclusively proves much, too many variables.
 

dannnielll

Well-known Member
My system is ultimately mid-ranged. I have yet to have the privilege of listening and comparing on a truly high-end, audiophile quality system. Maybe I'll find a difference? Who knows. But what I am looking for right now is something that says that that difference won't be in my head, so I have a good idea of the ultimate upgrade paths I want to take. I was a major in Cognitive Science, and I can't even begin to tell you how much our brain does to fill in our reality to account for the gaps in our senses and limited capacity for attention. For instance, our retina, the part of our eye that sees color, has a paltry 5 million cones, equivalent to a 5 megapixel camera. That's, ultimately, very very sh*tty. Everything that you see is rapidly being filled in by what our brain already knows and understands to give our sight the quality it has (do not take this to mean that there is no point in having a camera above 5 megapixels, this is not true at all). Another example is that in that same retina is the end of our optic nerve. This creates a black vertical line in our visual field, which is then filled in by the brain. There are some tricks you can do to see this vertical line. Do some quick Googling if you're curious.

The brain is doing similar things with sound. But it's harder to measure that cutoff like we can with vision (5 megapixels), and just where and how this filling in is taking place. If you want to see what I am talking about, grab yourself some low-end audio equipment. The lower the better. Maybe even your phone. Listen to a song you know well. It will sound like ass, but you will still find some enjoyment in it, and it won't sound too bad. Now listen to a sound you are sure you have never heard at all, anywhere, ever. Find some obscure hipster band on your streaming that has no listens. When you listen, it will sound sooooooo much worse than the song you did know, however bad that sounded, because your brain is filling in the missing pieces of the one your familiar with. This might be undone if you have a really really high expectation of the song you do know and are looking for those missing pieces. So I would recommend passive listening for this experiment.

Anyway, if anyone has more studies, go for it. I'll take a look at the ones posted.
5 million cones does not equate to a 5 mega pixel camera... The entire point is that the eye is capable of focusing on a very small section of the image field, and then later to subject that5 million on any other section. It can therefore with enough time subject any image with basically infinite resolution.
But the major point you are making is that the brain learns to fill in what it does not know. I often think this is why people may prefer the imperfections of vinyl.
 

Alter3go

Member
So you're not interested in talking about the science behind audio, considering it to be an opinion, but you're okay to talk about peoples opinions that have collated as part of study?

The link that was posted was an interesting read, but I'm not sure it conclusively proves much, too many variables.
"Opinion" backed by double-blind, properly processed data, passed through statistical analysis, ceases to be mere opinion. The foundation of the scientific method is replication, so, if enough studies are created that replicate the the methodology and produce the same conclusion, and the methodology is considered to be sound, this is what, in general, the scientific community considers "fact". Certainly, one or two studies is not enough to determine fact, but to equate forum opinions with the process of double-blind empirical investigation is absolutely laughable at best.

5 million cones does not equate to a 5 mega pixel camera... The entire point is that the eye is capable of focusing on a very small section of the image field, and then later to subject that5 million on any other section. It can therefore with enough time subject any image with basically infinite resolution.
But the major point you are making is that the brain learns to fill in what it does not know. I often think this is why people may prefer the imperfections of vinyl.
There's no point into getting into the nuances of the retina and going on a long tangent that strays away from the subject matter, when in regards to my conclusion, it seems its something we both agree on that point, that the brain is doing a MASSIVE amount of filling in.

As far as vinyl, I posted somewhere else a pretty good theory as to why some people prefer vinyl and others digital. I can definitely make a separate thread about that.
 

phil t

Well-known Member
"Opinion" backed by double-blind, properly processed data, passed through statistical analysis, ceases to be mere opinion. The foundation of the scientific method is replication, so, if enough studies are created that replicate the the methodology and produce the same conclusion, and the methodology is considered to be sound, this is what, in general, the scientific community considers "fact". Certainly, one or two studies is not enough to determine fact, but to equate forum opinions with the process of double-blind empirical investigation is absolutely laughable at best.
You seem to be confusing quotable scientific fact with opinion , as in :-

I can't point you to any tests, sorry, but I can tell you why FLAC etc can sound better than MP3
Notice the word CAN. Lots of variables, including the resolution of the system.

Of course, for a test as described above to be considered meaningful, the resolution of the system would have to be high enough to be able to produce all differences between files, else you system limit the test. Realistically every test would have to be conducted under tight conditions, ie consider acoustic properties of the room, the transducer, the calibration of equipment, temperature equipment and ambient, distancing between subject and transducer, etc. These conditions would have to be replicated within agreed tolerances.

Can I hear the difference between hi res and cd? Not convinced I can, so I buy (when I buy) hi res so as to start with the best. Having said that, a poor recording/mix can null any advantage of hi res before you even start.
 
Last edited:

oscroft

Member
...so, if enough studies are created that replicate the the methodology and produce the same conclusion, and the methodology is considered to be sound, this is what, in general, the scientific community considers "fact". Certainly, one or two studies is not enough to determine fact...
And that's where you're not going to find what you seek. The hi-fi world is notorious for almost never doing proper double-blind tests - try getting any cable enthusiast in the hi-fi press to agree to a double-blind test, for example.

Those multiple CD vs hi-res vs FLAC vs MP3 studies are just not there. I've only ever found a small number, and in my opinon they have all been flawed in some way. So I don't think the replicated sound methodology is there either.

I think you're going to have to find some other basis on which to decide your upgrade path.
 
Last edited:

JasonPSL

Active Member
"Opinion" backed by double-blind, properly processed data, passed through statistical analysis, ceases to be mere opinion. The foundation of the scientific method is replication, so, if enough studies are created that replicate the the methodology and produce the same conclusion, and the methodology is considered to be sound, this is what, in general, the scientific community considers "fact". Certainly, one or two studies is not enough to determine fact, but to equate forum opinions with the process of double-blind empirical investigation is absolutely laughable at best.



There's no point into getting into the nuances of the retina and going on a long tangent that strays away from the subject matter, when in regards to my conclusion, it seems its something we both agree on that point, that the brain is doing a MASSIVE amount of filling in.

As far as vinyl, I posted somewhere else a pretty good theory as to why some people prefer vinyl and others digital. I can definitely make a separate thread about that.
The issue with comparing the visual analogy to hifi is that the brain has to make assumptions and frequently gets it wrong - as with the visual attentive test with the gorilla and passing the basket ball test - google it if you are unaware of just how easy the brain can get confused.
When the brain does not have enough information, it will fill in the gaps. The question is, will it do so accurately (more likely if you know the piece of music well) or with errors. Also, this puts more stress on the brain to concentrate and work out what is happening. You could ague either way whether this leads to better listening or worse.
 

Alter3go

Member
The issue with comparing the visual analogy to hifi is that the brain has to make assumptions and frequently gets it wrong - as with the visual attentive test with the gorilla and passing the basket ball test - google it if you are unaware of just how easy the brain can get confused.
When the brain does not have enough information, it will fill in the gaps. The question is, will it do so accurately (more likely if you know the piece of music well) or with errors. Also, this puts more stress on the brain to concentrate and work out what is happening. You could ague either way whether this leads to better listening or worse.
You're in the ballpark, but you still have a fundamentally flawed assumption: that the brain isn't doing filling in when it does have enough information. As I said earlier, attention is another major facet here. Compared to the barrage of stimuli coming from the world, our capacity for attention is very limited. This both stresses certain stimuli at the expense of others, and also causes our brain to completely disregard other stimuli. But even when we are attent to a full set of information, the brain is still filling in many details. A lot lot lot of detail. Remember, on top of it all, the thalamus is filtering sh*ttons of information out from the stimulii we're taking in, before any processing at all is ever done, leaving huge potential gaps. This process is called sensory gating. Errors in sensory gating is strongly linked to schizophrenia. Our minds simply cannot handle the full weight of information and stimuli from the world around us, in any sensory modality, including hearing. Expectation, of course, can drastically alter, change, add to, or subtract from, the process of our brain filling in this information. Be aware of that before you're sure, oh so sure, that you're hearing, feeling, or even seeing, some difference. Remember the old trope about playing this or that song backwards having satanic lyrics? They're there if you listen and look for them. There's a reason we do double blind with any decent study (or at least single blind). Therefore, the visual analogy stands, even if I don't have the specifics of the retina right.
 

Alter3go

Member
And that's where you're not going to find what you seek. The hi-fi world is notorious for almost never doing proper double-blind tests - try getting any cable enthusiast in the hi-fi press to agree to a double-blind test, for example.

Those multiple CD vs hi-res vs FLAC vs MP3 studies are just not there. I've only ever found a small number, and in my opinon they have all been flawed in some way. So I don't think the replicated sound methodology is there either.

I think you're going to have to find some other basis on which to decide your upgrade path.
Well, I really must admit, that is quite the problem. But, in our little corner of the internet, in a subform in a fourm not really dedicated to audio at all, us few souls can do our best to compile any and all scientific data in this regard, however meager it may be. Onward troopers!
 

The latest video from AVForums

Podcast: New Denon AV Products, HDMI 2.1, 8K, DTS:X Pro Sound Q&A with Denon Product Specialists

Trending threads

Latest News

Philips TV and Bowers & Wilkins extend partnership
  • By Andy Bassett
  • Published
Denon unveils CEOL N11 DAB music system
  • By Andy Bassett
  • Published
Audeze announces Penrose wireless gaming headphones
  • By Andy Bassett
  • Published
Movies preferred to TV shows during lockdown in UK
  • By Andy Bassett
  • Published
Ruark announces R3 all-in-one music system
  • By Andy Bassett
  • Published
Top Bottom