The Traders' Den  

  The Traders' Den > Where we go to learn ..... > Technobabble
 

Notices

Technobabble Post your general Need for Help questions here.
• Lossy or Lossless?
Moderators

Reply
 
Thread Tools
  #16  
Old 2005-09-28, 01:46 PM
feralicious's Avatar
feralicious feralicious is offline
dare to discover
63.44 GB/133.91 GB/2.11
 
Join Date: Nov 2004
Location: occasionally
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Quote:
Originally Posted by range_hood
Thatīs Blondie July 9, 1979, isnīt it?
These files are rotting on my harddisk since I downloaded. I asked dgkdgk (the taper and seeder) to reseed them and reported the torrent to the DaD staff. No response from the seeder, a mod over there said the torrent is ok as long as no other leecher has a problem with it.

If you listen to the problematic tracks you will hear digital errors, so please donīt trade this further without noting that in the info file.

The ffp-txt file matches the "fingerprint infomation" in the flac files, but the audio content has different checksums. Maybe the seeder flaced the files without verify?
In TLH "Verify checksum file" (ffp) does only compare the textfile to the "fingerprints" stored in the flacs. Where "Test encoded files" (same as flac.exes Verify) decodes the audio, calculates the checksums and compares that to the "fingerprint information" in the flacs.
This "Test encoded files" is comparable to a st5 checksum verification.

What I do is checking if all checksums do match and "test encoded files" (all in TLH) - additional listening to questionable files a bit closer and noting all problems. And maybe persuading the seeder, as he seems to be the taper, to reseed a fixed show.
Yep that's the one. So how come when I fix the SBEs, processing all the files, the new files are fine and the flac ffp/checksum matches the audio checksum?

And so really basic ffp and verifying just those is useless then. Why even have them? Now I know to verify the files and not just run the ffp verify. Had I not happened to do it on this show I would've just checked using the ffp and thought it was fine. That's bad because I actually spent (too much) time trying to understand all this and SBEs, just think about all those others who don't even look into it.
__________________
feralicious goodies
Don't get even.
Get odd.

....

Last edited by feralicious; 2005-09-28 at 01:52 PM.
Reply With Quote Reply with Nested Quotes
  #17  
Old 2005-09-28, 02:42 PM
range_hood's Avatar
range_hood range_hood is offline
lsmf
11.44 GB/10.00 GB/0.87
 
Join Date: Apr 2005
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Quote:
Originally Posted by feralicious
Yep that's the one. So how come when I fix the SBEs, processing all the files, the new files are fine and the flac ffp/checksum matches the audio checksum?
On re-encoding (fixing sbes) you calculated new checksums of the actual audio content.

Quote:
Originally Posted by ferelicious
And so really basic ffp and verifying just those is useless then. Why even have them? Now I know to verify the files and not just run the ffp verify. Had I not happened to do it on this show I would've just checked using the ffp and thought it was fine. That's bad because I actually spent (too much) time trying to understand all this and SBEs, just think about all those others who don't even look into it.
Thatīs a good point. The function should be disabled in TLH for ffp in the next release. Maybe ban *.ffp and use *.st5 as standard. TLH development seems not to be that vital.
You have to have fingerprints as a textfile, so users can compare their sources without having to download the whole show.
Reply With Quote Reply with Nested Quotes
  #18  
Old 2005-09-28, 02:56 PM
AAR.oner's Avatar
AAR.oner AAR.oner is offline
TTD Staff
1.11 TB/1.41 TB/1.27
 
Join Date: Feb 2005
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

nope...i've not been clear... i understand what ya'll are saying...

i received a show ina trade...the only checksum included was a ffp.txt...when i went to verify it, it failed...so i do some research and find the "real" ffp online, checked it against my copy of the show, and it verified correct...so i compare the two ffp's...seems the ffp.txt i got in the trade had been messed up [i think a few of the letters/numbers got jumbled in the fingerprint text itself]...hence my orig point:

when creating a lossless fileset [pref. flac of course ] on a show for the first time, best to create an st5 checksum, and not a ffp.txt *only*...they are more prone to "user-error" than a .st5 or even a regular .ffp...
__________________
TTD's Gear Lust Forum -- info & reviews on taping gear
The Basics of EQing
Reply With Quote Reply with Nested Quotes
  #19  
Old 2005-09-29, 10:09 AM
Five's Avatar
Five Five is offline
189.30 GB/594.78 GB/3.14
 
Join Date: Oct 2004
Location: Canada
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

after briefly reading this thread, here's what I can tell you...

when a FLAC file is created, the ffp for that file is stored in the header at the time the audio is compressed.

creating a ffp merely copies this checksum to a .txt file (using FLAC frontend) or a .ffp (using TLH). the .ffp file is just like the .txt file, just with a different extension.

when a .ffp (or ffp.txt) file is verified using TLH, it takes the .ffp file and compares the checksum associated with each file with the checksum stored in the header. it doesn't actually look further than that. That's why it takes such a short time to do this function.

FLAC frontend can't verify .ffp files this way.

when you use the "test" function on FLAC frontend, it decompresses the audio to a temp file, generates a checksum and compares this with the checksum which was stored in the header at the time the file was created. TLH can also do this, using the "test encoded files" function.

okay, here's the tricky part:

when you use TLH (or shntool) to verify an .st5 file, TLH decompresses the audio to a temp file, generates fresh checksums and compares these with the checksums contained in the .st5 file and doesn't check the headers of the FLAC files at all.

also, when you fix a SBE, the file is decompressed, altered, then re-encoded to FLAC and therefore a fresh checksum is generated and inserted into the header.

so the error from flac frontend "MD5 signature mismatch" should also be reported from TLH when you use the "test encoded files" function, and it means that the checksums contained in the headers of those files don't match the checksums generated on the spot from the WAV audio decompressed to a temp file.

when you made the new .ffp it took all the checksums from the headers just like the seeder did, therefore they are the same.

then you make an .st5 and it shows the true checksums for the files, some of which have incorrect checksums stored in their headers at the original time of compression.

the files you fixed were re-encoded and new headers were written which you tested as being accurate.

so, to fully verify a FLAC file, you have to do a couple things:

1) check that the checksums contained in the headers match the checksums of the audio contained within. you can do this using either TLH "test encoded..." or FLAC frontend verify/test.

2) if there is a .ffp file, check that it matches what is contained in the headers using TLH "verify checksum" or by generating a fresh fingerprint using FLAC frontend and comparing by eye.

3) if there is a .st5 file, verify it using TLH "verify checksum file". If you want to do the same using only FLAC frontend, run a test to make sure the headers match the files, generate a ffp.txt, then compare the checksums by eye with the .st5 checksums.

is that confusing enough?

remember, I'm only on coffee #1...
__________________
Checksums Demystified | ask for help in Technobabble

thetradersden.org | ttd recommended free software/freeware webring
shntool tlh eac foobar2000 spek audacity cdwave vlc

Quote:
Originally posted by oxymoron
Here you are in a place of permanent madness, be careful!
Reply With Quote Reply with Nested Quotes
  #20  
Old 2005-09-29, 10:11 AM
Five's Avatar
Five Five is offline
189.30 GB/594.78 GB/3.14
 
Join Date: Oct 2004
Location: Canada
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

and I find it very strange that the checksums in the header don't match the audio in some files... that's a problem that happened during compression, and I've never seen this before and am not able to re-create this sort of a problem. My gf always thought I was paranoid to bother testing FLACs so thoroughly, turns out its a good idea!
__________________
Checksums Demystified | ask for help in Technobabble

thetradersden.org | ttd recommended free software/freeware webring
shntool tlh eac foobar2000 spek audacity cdwave vlc

Quote:
Originally posted by oxymoron
Here you are in a place of permanent madness, be careful!
Reply With Quote Reply with Nested Quotes
  #21  
Old 2005-09-29, 10:52 AM
AAR.oner's Avatar
AAR.oner AAR.oner is offline
TTD Staff
1.11 TB/1.41 TB/1.27
 
Join Date: Feb 2005
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

i apologize for my hijacking of this thread Fera...sorry

i'ma start a new thread re: st5 vs ffp vs md5, WANT ya'lls thoughts opinions on it tho...cheers!
__________________
TTD's Gear Lust Forum -- info & reviews on taping gear
The Basics of EQing
Reply With Quote Reply with Nested Quotes
  #22  
Old 2005-09-29, 03:40 PM
range_hood's Avatar
range_hood range_hood is offline
lsmf
11.44 GB/10.00 GB/0.87
 
Join Date: Apr 2005
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Quote:
Originally Posted by Five
and I find it very strange that the checksums in the header don't match the audio in some files... that's a problem that happened during compression, and I've never seen this before and am not able to re-create this sort of a problem.
I only can imagine this on encoding flacs while other programs are loading the cpu; without checking "Verify" in flac frontend.
Reply With Quote Reply with Nested Quotes
  #23  
Old 2005-09-29, 09:47 PM
feralicious's Avatar
feralicious feralicious is offline
dare to discover
63.44 GB/133.91 GB/2.11
 
Join Date: Nov 2004
Location: occasionally
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Well now I'm wondering about shn files. Do you need to both verify the md5 and "test" the files? I have been since I came across this problem but it feels like overkill.

Aaron... don't worry... I never thought you had hijacked it.

range_hood... I reported that torrent and they pulled it. Of course it had already run its course and hundreds are trading it at this very moment, but hey...
__________________
feralicious goodies
Don't get even.
Get odd.

....
Reply With Quote Reply with Nested Quotes
  #24  
Old 2005-10-01, 01:32 PM
feralicious's Avatar
feralicious feralicious is offline
dare to discover
63.44 GB/133.91 GB/2.11
 
Join Date: Nov 2004
Location: occasionally
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Quote:
Originally Posted by feralicious
Well now I'm wondering about shn files. Do you need to both verify the md5 and "test" the files? I have been since I came across this problem but it feels like overkill.
Still wondering if I'm wasting my time. Can I just do the "test encoded files"? I'm going to convert them to flac anyway before I archive.
__________________
feralicious goodies
Don't get even.
Get odd.

....
Reply With Quote Reply with Nested Quotes
  #25  
Old 2005-10-01, 01:58 PM
range_hood's Avatar
range_hood range_hood is offline
lsmf
11.44 GB/10.00 GB/0.87
 
Join Date: Apr 2005
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Quote:
Originally Posted by feralicious
Still wondering if I'm wasting my time. Can I just do the "test encoded files"? I'm going to convert them to flac anyway before I archive.
On "Test encoded files" the files are decoded to a temp directory. As you convert them to flac anyway (shn>wav>flac) the test is included in the conversion (say, itīs no test anymore ).
The md5 check is a must, to determine if the files are the same as the seeders ones.
Reply With Quote Reply with Nested Quotes
  #26  
Old 2005-10-01, 02:08 PM
feralicious's Avatar
feralicious feralicious is offline
dare to discover
63.44 GB/133.91 GB/2.11
 
Join Date: Nov 2004
Location: occasionally
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

What if I'm going straight from shn>flac? I use foobar.

I swear I think I have all this down then something like this comes up and confuses me again.

So... if the md5 is a must, then why do people think you don't need them for flacs? I don't see what the difference is if "testing" isn't enough for shns why would it be for flacs?

When someone says "whole file md5" that's not the same as an st5, right? That's like an md5 of all the tracks?
__________________
feralicious goodies
Don't get even.
Get odd.

....
Reply With Quote Reply with Nested Quotes
  #27  
Old 2005-10-01, 04:26 PM
range_hood's Avatar
range_hood range_hood is offline
lsmf
11.44 GB/10.00 GB/0.87
 
Join Date: Apr 2005
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Quote:
Originally Posted by feralicious
What if I'm going straight from shn>flac? I use foobar.
If no error shows up on reencoding, then the files are fine in terms of "properly decoding".
To compare if the audio contents match after converting you can use a batchfile (by uhclem). Sure, if you fix sector boundary errors, the files will not match anymore.
_____________
@echo off
for %%T in (*.shn) do shntool cmp %%T %%~nT.flac
pause

_____________

Or you create st5s of shns and flacs and compare by eye:
967c57a1a178f952f931a5a090252c8d [shntool] ns2002-11-18t01.shn
bff785557d40fff55dea7e5784583d31 [shntool] ns2002-11-18t02.shn
967c57a1a178f952f931a5a090252c8d [shntool] ns2002-11-18t01.flac
bff785557d40fff55dea7e5784583d31 [shntool] ns2002-11-18t02.flac

Quote:
Originally Posted by feralicious
So... if the md5 is a must, then why do people think you don't need them for flacs? I don't see what the difference is if "testing" isn't enough for shns why would it be for flacs?
On flacs "test encoded files" (TraderLittleHelper) or "Verify" (FlacFrontend) the files are decoded, the checksums calculated and compared to the checksum in the header in just one step. If thereīs a problem on decoding, it will show up; if the checksums do not match, it will show up.
Shns do not have checksum informations included. Thatīs why you need an textfile.

Quote:
Originally Posted by feralicious
When someone says "whole file md5" that's not the same as an st5, right? That's like an md5 of all the tracks?
st5 is the calculated checksum of the decompressed (decoded) audio content only.
itīs the same as a flac fingerprint, just in another text format but also usable for other audio formats like shn, ape, ....

ffp:
beatles-urt-d1t01.flac:95fd6f5e85fab21a1a84e9447af3d6bd
beatles-urt-d1t02.flac:b0ae6b35acc4f4eea9783154444ee829
st5:
95fd6f5e85fab21a1a84e9447af3d6bd [shntool] beatles-urt-d1t01.flac
b0ae6b35acc4f4eea9783154444ee829 [shntool] beatles-urt-d1t02.flac

In wholefile md5s of audio files a checksum of the whole file is calculated (including headers, tags, etc.; no decoding is done).

Last edited by range_hood; 2005-10-01 at 04:31 PM.
Reply With Quote Reply with Nested Quotes
  #28  
Old 2005-10-01, 07:01 PM
feralicious's Avatar
feralicious feralicious is offline
dare to discover
63.44 GB/133.91 GB/2.11
 
Join Date: Nov 2004
Location: occasionally
Re: Flac Fingerprint vs. st5 / Flac Frontend vs. TLH ???

Okay, thanks range_hood.

I used to bitcompare in foobar when I converted shn>flac, but now I make st5 of the shns first, then of the flacs and visually compare. I'll try uhclem's batch file again, but last time I tried it I couldn't get it working, though I think I was writing it out myself so I'm sure I just couldn't preoperly figure out the commands.

I swear if only I didn't care about all this I could've had all my 700gb archived already! It's annoying how much slop is getting passed around that has to be fixed.
__________________
feralicious goodies
Don't get even.
Get odd.

....
Reply With Quote Reply with Nested Quotes
Reply

The Traders' Den > Where we go to learn ..... > Technobabble

Similar Threads
Thread Forum Replies Last Post
flac fingerprint problem - corpusse Technobabble 2 2009-03-18 12:04 PM
Question about Flac fingerprint errors..... - tWreCK51 Technobabble 7 2008-02-20 06:58 PM
NO Text in Flac Fingerprint - reese Technobabble 4 2008-01-08 12:14 AM
Really Dumb Flac Fingerprint Questions... - velvetsailor Technobabble 1 2007-02-26 04:33 AM
FLAC Fingerprint not working! - analog414 Technobabble 6 2006-09-04 11:35 AM


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forums


All times are GMT -5. The time now is 06:44 PM.


Powered by: vBulletin, Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004 - , TheTradersDen.org - All Rights Reserved - Hosted at QuickPacket