The Traders' Den  

  The Traders' Den > Where we go to learn ..... > Technobabble
 

Notices

Technobabble Post your general Need for Help questions here.
Lossy or Lossless?
Moderators

Reply
 
Thread Tools
  #1  
Old 2017-07-20, 11:06 PM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
FLAC fingerprint md5 mismatch - looking for some help

Hi all

So.....I have received three separate file sets in the past couple of days (at another bt site), all of the same artist, all from the same seeder. Each set has an md5 and an ffp; in each case the md5 verifies fine but in each case the ffp gives this message for every track: "failed verification (checksum in file header matches, but checksum of audio data does not match)."
The weird thing is that even when I use TLH to create a new ffp, that ffp yields the same message. I hope I've explained the problem adequately; does anyone have any insight into this phenomenon? The files will decode fine but there is apparently no way to create an ffp that will verify properly.

TIA for any help or insight.

Last edited by TTDaddy; 2017-07-23 at 12:20 PM.
Reply With Quote Reply with Nested Quotes
  #2  
Old 2017-07-23, 11:30 PM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

Anyone? I suspect this is a technical question having to do with how ffp files are created...but I'd really like to get to the bottom of this before these sets are spread any further.

Thanks in advance for any help!
Reply With Quote Reply with Nested Quotes
  #3  
Old 2017-07-24, 05:28 PM
dorrcoq's Avatar
dorrcoq dorrcoq is offline
Champion of the Silent
TTD VIP
727.47 GB/6.81 TB/9.58
 
Join Date: Nov 2004
Re: FLAC fingerprint md5 mismatch - looking for some help

Are those from the Waterfront Blues Festival in Portland? I downloaded one - JD McPherson - that had these issues. I reported it and got it pulled, but the "fix" only corrected some of the files. Strangely they all play fine.
__________________
DON'T MESSAGE ME FOR RE-SEEDS. I DO NOT DO THEM!
Reply With Quote Reply with Nested Quotes
  #4  
Old 2017-07-24, 08:30 PM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

Quote:
Originally Posted by dorrcoq View Post
Are those from the Waterfront Blues Festival in Portland? I downloaded one - JD McPherson - that had these issues. I reported it and got it pulled, but the "fix" only corrected some of the files. Strangely they all play fine.
No, these are from Steve Hackett shows fro 1980/81. As in your case, the files play fine but the ffp fails verification. I just am trying to fix that issue...but the files decode correctly.
Reply With Quote Reply with Nested Quotes
  #5  
Old 2017-07-25, 12:44 PM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

Anyone have any input here? The error message is one thing; but when I re-create the ffp, the values are exactly the same yet even the new file fails verification with the same message. HELP!!
Reply With Quote Reply with Nested Quotes
  #6  
Old 2017-07-30, 07:24 PM
spidergawd's Avatar
spidergawd spidergawd is offline
Peninsulanwolf
278.96 GB/560.91 GB/2.01
 
Join Date: Dec 2011
Location: Metro Day Twa
Re: FLAC fingerprint md5 mismatch - looking for some help

Quote:
Originally Posted by TTDaddy View Post
Anyone have any input here? The error message is one thing; but when I re-create the ffp, the values are exactly the same yet even the new file fails verification with the same message. HELP!!
What happens if you use flac's "test" function?

flac -t *.flac

If the checksums in the header don't match the checksums of the audio data, it will fail the test. If it passes, then that would sound like something messing with TLH, especially if

metaflac --show-md5sum *.flac

produces the same values as in the ffp file.

I don't use Windows or TLH, I use Slackware Linux and have had to edit and re-write my ffp-checking shell script every time I come across an ffp file with weird text formatting or other simple things that can screw it up, but that doesn't sound like your situation since TLH wouldn't use formatting that TLH can't understand. IMO if your files pass flac's test and also pass the md5 check, I'd consider the files good and chalk this up to a fluke or bug.

Good luck
Reply With Quote Reply with Nested Quotes
  #7  
Old 2017-07-30, 10:14 PM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

Quote:
Originally Posted by spidergawd View Post
What happens if you use flac's "test" function?

flac -t *.flac

If the checksums in the header don't match the checksums of the audio data, it will fail the test. If it passes, then that would sound like something messing with TLH, especially if

metaflac --show-md5sum *.flac

produces the same values as in the ffp file.

I don't use Windows or TLH, I use Slackware Linux and have had to edit and re-write my ffp-checking shell script every time I come across an ffp file with weird text formatting or other simple things that can screw it up, but that doesn't sound like your situation since TLH wouldn't use formatting that TLH can't understand. IMO if your files pass flac's test and also pass the md5 check, I'd consider the files good and chalk this up to a fluke or bug.

Good luck
I used the "test" function in FLAC frontend, and in every case (all files in all four shows from the same uploader) I got:
"ERROR: MD5 signature mismatch"

The weird thing is that TLH will generate a new ffp that is identical to the one contained in the fileset.
Reply With Quote Reply with Nested Quotes
  #8  
Old 2017-07-31, 08:13 PM
spidergawd's Avatar
spidergawd spidergawd is offline
Peninsulanwolf
278.96 GB/560.91 GB/2.01
 
Join Date: Dec 2011
Location: Metro Day Twa
Re: FLAC fingerprint md5 mismatch - looking for some help

Ok, as I've said I don't use TLH and don't know for sure what's going on, but I'll tell you what it sounds like to me: the audio data of the files got altered somehow after the encoding but before the md5 was generated, and TLH is simply outputting the (no longer valid) checksums from the files' headers (metaflac outputs the MD5 signature from the STREAMINFO block). Possibly the files got truncated somewhere along the way before they were uploaded?

shntool's "hash" mode will output md5 signatures of just the decompressed audio portion of the files (as opposed to a wholefile signature), same as a flac fingerprint. They should match. If not, something has happened to either the audio portion or the fingerprint portion of the files. If you create an ffp file from the shntool hash output, it should verify in TLH. The question then would be WTF happened to the flacs?

Corrections appreciated.
Reply With Quote Reply with Nested Quotes
  #9  
Old 2017-07-31, 10:12 PM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

Quote:
Originally Posted by spidergawd View Post
Ok, as I've said I don't use TLH and don't know for sure what's going on, but I'll tell you what it sounds like to me: the audio data of the files got altered somehow after the encoding but before the md5 was generated, and TLH is simply outputting the (no longer valid) checksums from the files' headers (metaflac outputs the MD5 signature from the STREAMINFO block). Possibly the files got truncated somewhere along the way before they were uploaded?

shntool's "hash" mode will output md5 signatures of just the decompressed audio portion of the files (as opposed to a wholefile signature), same as a flac fingerprint. They should match. If not, something has happened to either the audio portion or the fingerprint portion of the files. If you create an ffp file from the shntool hash output, it should verify in TLH. The question then would be WTF happened to the flacs?

Corrections appreciated.
Thank you for taking the time to respond. I understood about half of that ;-)
The files decoded to wav (using "decode through errors") with the md5 mismatch error. I'm not sure what the seeder would have done to the files AFTER encoding; there are no tags showing in foobar so I don't think that's the issue. I don't use shntool so I can't follow your advice. It's also weird that not one of those who have downloaded these shows (222 downloads so far) has mentioned this issue on that site.

Thanks again for your input. Not sure what else to do at this point.
Reply With Quote Reply with Nested Quotes
  #10  
Old 2017-07-31, 10:25 PM
rspencer's Avatar
rspencer rspencer is offline
TTD Hoarder
TTD Staff
326.11 GB/317.06 GB/0.97
 
Join Date: Oct 2007
Location: the outer limits
Re: FLAC fingerprint md5 mismatch - looking for some help

Quote:
Originally Posted by TTDaddy View Post
It's also weird that not one of those who have downloaded these shows (222 downloads so far) has mentioned this issue on that site.
Not really. So many people download shows and never listen to them. Terrabytes of data never again used.

No real idea on what's causing your issue, but I seem to recall that happening to me once and the file names were the issue. There was a work-around to get them to verify correctly even with the header mismatch.
__________________
My main list .... My masters

Quote:
Originally Posted by dcbullet View Post
Sometimes im dense.
Quote:
Originally Posted by dcbullet View Post
I hate not being a real man.
Reply With Quote Reply with Nested Quotes
  #11  
Old 2017-08-01, 04:51 PM
roann's Avatar
roann roann is offline
TTD Staff
1.24 TB/594.30 GB/0.47
 
Join Date: Nov 2004
Location: Planet Earth
Re: FLAC fingerprint md5 mismatch - looking for some help

Quote:
Originally Posted by TTDaddy View Post
Hi all

So.....I have received three separate file sets in the past couple of days (at another bt site), all of the same artist, all from the same seeder. Each set has an md5 and an ffp; in each case the md5 verifies fine but in each case the ffp gives this message for every track: "failed verification (checksum in file header matches, but checksum of audio data does not match)."
The weird thing is that even when I use TLH to create a new ffp, that ffp yields the same message. I hope I've explained the problem adequately; does anyone have any insight into this phenomenon? The files will decode fine but there is apparently no way to create an ffp that will verify properly.

TIA for any help or insight.
1. After encoding a .wav file the FLAC encoder stores the checksum of the raw WAVE data (the so-called flac fingerprint or ffp) in the header of the .flac file.

2. When TLH "creates" a ffp it just reads the checksum stored in the header of the .flac file.

3. When TLH verifies a flac fingerprint it re-calculates the checksum of the audio data and compares this fingerprint with the one stored in the .flac file's header. If the checksums do not match you'll get the error message "failed verification (checksum in file header matches, but checksum of audio data does not match)". So the checksum in the file's header is incorrect.

I think the creator of the .flac files has used a bad encoding program because all the files fail to verify. Note that the md5 checksums match because they were created after creating the (invalid) .flac files.
Reply With Quote Reply with Nested Quotes
  #12  
Old 2017-08-03, 05:33 PM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

I've been corresponding with the seeder/uploader, and he has been remarkably cool with my questioning. I'm going to share what he wrote to me. I'm not using his name so I don't feel like I'm betraying a confidence; as the files decode successfully (albeit with the md5 mismatch error message), I'm going to stop bothering him about this issue. Now I'm just trying to understand what's up, as it does not appear that he's doing anything to the FLACs after creating them. He wrote:

"Well, maybe all this time I've been doing something wrong, I don't know? I've been taking my original master cassettes and loading them into my Tascam deck that is connected to my PC and recording the audio in raw .WAV format via WavePad. I then use the split functionality to break into tracks and save them as individual FLAC files at 16 bit 96kHz sample rate. Once the files have been compiled I use TLH to create the MD5 and ffp checksums (pass every time) and the torrent file. I've posted other artists on other trackers and never had anyone mention there is an issue. Am I not doing something right?"

Two things:
1 - I am apparently the only person experiencing this issue, which worries me.
2 - Is it possible that the files being 16/96 is making TLH choke on them? Note that when I use TLH to check for SBEs, it returns the message "information about audio data is not available" for each track.
I'm truly at sea here so any ideas or advice are welcome. Thanks for putting up with my pedantic nature!
Reply With Quote Reply with Nested Quotes
  #13  
Old 2017-08-04, 07:46 AM
spidergawd's Avatar
spidergawd spidergawd is offline
Peninsulanwolf
278.96 GB/560.91 GB/2.01
 
Join Date: Dec 2011
Location: Metro Day Twa
Re: FLAC fingerprint md5 mismatch - looking for some help

1 - Your md5 passes, so you got the same files as everyone else. They either didn't check or didn't care.

2 - You can't burn audio CDs directly from 96 khz files, so checking for SBEs is somewhat pointless.
Reply With Quote Reply with Nested Quotes
  #14  
Old 2017-08-04, 11:03 AM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

Yes, I know I can't burn straight from 96k files; it was just an exercize to see how TLH would interact with the files. The uploader has told me that he will be uploading another SH show, in 48k. We'll see if that proves to be the issue. Update to follow...

Quote:
Originally Posted by spidergawd View Post
1 - Your md5 passes, so you got the same files as everyone else. They either didn't check or didn't care.

2 - You can't burn audio CDs directly from 96 khz files, so checking for SBEs is somewhat pointless.
Reply With Quote Reply with Nested Quotes
  #15  
Old 2017-09-06, 11:05 AM
TTDaddy's Avatar
TTDaddy TTDaddy is offline
713.91 GB/3.62 TB/5.20
 
Join Date: Dec 2006
Re: FLAC fingerprint md5 mismatch - looking for some help

OK, I'm going to put this to bed now. But just in case anyone is interested.....I got a PM from a member here who gave me the following info (I'm paraphrasing so as not to violate his confidence):

There's a free version of WavePad available (for non-commercial use only); I tested it.

1. The "Split" function works properly.

2. The file parts you get when splitting the original . wav file can be saved in FLAC format (this is what the seeder of your files did). Problem: These .flac files do not have a standard .flac file header as defined by the makers of the FLAC format. Among other things WavePad does not write the flac fingerprint to the file header. TLH assumes that the file header is in standard format and reads the part of the header that usually contains the checksum. That's why the "created" fingerprints are wrong.

Conclusion:
The .flac files WavePad create are ok (the encoding of the WAV data is correct, you can play the files, you can decode them), but the file headers are non-standard. So if you want to get correct checksums of the audio part of the .flac files using TLH you have to create the shntool md5 fingerprints (st5)."

All is well, just wanted to button this topic up. Thanks all!
Reply With Quote Reply with Nested Quotes
Reply

The Traders' Den > Where we go to learn ..... > Technobabble

Similar Threads
Thread Forum Replies Last Post
Weird... IPs don't match - Buzz Durden-Hail Technobabble 2 2008-09-08 12:52 PM


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forums


All times are GMT -5. The time now is 04:21 PM.


Powered by: vBulletin, Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004 - , TheTradersDen.org - All Rights Reserved - Hosted at QuickPacket