![]() |
|
Technobabble Post your general Need for Help questions here.
• Lossy or Lossless? Moderators |
![]() |
|
Thread Tools |
#1
|
||||
|
||||
Checksums Demystified (.st5, .ffp and .md5)
TTD Requirements:
SHN/APE: .st5 fingerprints posted in announce thread, should also be included with the show. Since SHN format is not capable of self-testing, some form of checksums must be included. FLAC: .st5/.ffp fingerprints posted in announce thread, should also be included with the show. DVD: .md5 checksums posted in announce thread and included with the show. Checksums (.st5, .ffp and .md5) Checksums are used for the purpose of verifying the integrity and identity of a digital copy of audio. There are many kinds of checksums but only a few that are commonly used by music traders: .md5 wholefile checksum. verifies that files are identical in every single way, and is therefore sensitive to compression setting, file format et al. .st5 aka SHNtool md5. this is a checksum taken from the decompressed audio only. the advantage to this is that it works with any lossless codec at any compression setting. This is the preferred checksum type to be included with FLAC, SHN or APE files. .ffp aka FLAC fingerprints. These checksums are virtually identical to .st5. some differences are explained below, under "Verifying a checksum file" and "Creating a .ffp file". For audio, I highly recommend using FLAC and including .st5 and .ffp no wholefile .md5 necessary. For Mac use xAct. Tutorials here: Making ShnTool md5s using xACT http://u2lynne.sandsmuseum.com/checksums.html For PC use TLH: http://tlh.easytree.org/ TLH checksums tutorial Testing lossless files: This function runs a simple test to see if the files will decode properly. 1. Go to Test encoded files > Add 2. Use Files of type to select the lossless file type. 3. Select all and hit open. ![]() 4. Check that the information displayed in the list is correct. 5. Press Test 6. The result will take a few minutes to display. ![]() Verifying a checksum file: TLH can verify .st5, .ffp and .md5 files. You can simply double-click the checksum file to begin testing, or: 1. Go to Verify checksum file > Add 2. Use Files of type to select checksum type. Load the checksum file. 3. Press Test. 4. This will take a few minutes. ffp compares the checksums in the FLAC header without checking the audio. ![]() ![]() ![]() .md5 checks for any change whatsoever in the whole file (useful for DVD). .st5 checks the decompressed audio only. ![]() ![]() md5 for DVD cannot find the files, what do I do? To verify a playable video DVD with checksums which are on your hd, generate md5 checksums (see below) directly from the DVD and then test those against the files which are on your hd. Another problem that sometimes occurs is that the paths stored in the .md5 file do not tell TLH to look inside the VIDEO_TS folder. So if the .md5 file for your DVD looks like this: Code:
8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS.BUP 8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS.IFO e40bd45c55f783d519c2a5c9aa088e98 *VIDEO_TS.VOB e11aaf0c14501bbf15d776a34218c205 *VTS_01_0.BUP e11aaf0c14501bbf15d776a34218c205 *VTS_01_0.IFO da29301176f55de40d79be5cba560d06 *VTS_01_0.VOB 99ba41ff0fec88e3def109b6cb824dcb *VTS_01_1.VOB 6c20fa2d2232106310d61dce30cd3e29 *VTS_01_2.VOB Code:
8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.BUP 8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.IFO e40bd45c55f783d519c2a5c9aa088e98 *VIDEO_TS/VIDEO_TS.VOB e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.BUP e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.IFO da29301176f55de40d79be5cba560d06 *VIDEO_TS/VTS_01_0.VOB 99ba41ff0fec88e3def109b6cb824dcb *VIDEO_TS/VTS_01_1.VOB 6c20fa2d2232106310d61dce30cd3e29 *VIDEO_TS/VTS_01_2.VOB Advanced: Another possibility is to burn a standalone-playable data DVD which contains info, artwork (if any) and .md5 checksums inside a folder called EXTRAS_TS. To do this, create three folders in your compilation: AUDIO_TS EXTRAS_TS VIDEO_TS The VIDEO_TS folder contains the obvious, the AUDIO_TS folder must be there and remain absolutely empty, and the EXTRAS_TS with the art/info/checksums inside. When the .md5 file is placed inside of an EXTRAS_TS folder, the path must be formatted as: Code:
8d50cba158b40a75dd32cada8964aef7 *../VIDEO_TS/VIDEO_TS.BUP 8d50cba158b40a75dd32cada8964aef7 *../VIDEO_TS/VIDEO_TS.IFO e40bd45c55f783d519c2a5c9aa088e98 *../VIDEO_TS/VIDEO_TS.VOB e11aaf0c14501bbf15d776a34218c205 *../VIDEO_TS/VTS_01_0.BUP e11aaf0c14501bbf15d776a34218c205 *../VIDEO_TS/VTS_01_0.IFO da29301176f55de40d79be5cba560d06 *../VIDEO_TS/VTS_01_0.VOB 99ba41ff0fec88e3def109b6cb824dcb *../VIDEO_TS/VTS_01_1.VOB 6c20fa2d2232106310d61dce30cd3e29 *../VIDEO_TS/VTS_01_2.VOB thanks to fatoldpig and yoho for suggesting this stuff! Running a len (length) check: This gives some helpful details about the set's compatability with audio cdr format. This is primarily used to check for SBEs (Sector Boundary Errors), and that the files are 16bit/44.1kHz. For more detailed information about the meaning of the data, check here: http://www.etree.org/shnutils/shntoo...oc/shntool.txt 1. Go to Audio file Details > Add 2. Use Files of type to select the lossless file type. 3. Select all and hit open. ![]() 4. Check that the information displayed in the list is correct. 5. Press Show details ![]() 6. Copy and paste into your info .txt file. Code:
length expanded size cdr WAVE problems filename 2:15.26 23875196 --- -- ---xx gl2005-03-12t01.flac 5:22.52 56923148 --- -- ---xx gl2005-03-12t02.flac 4:14.46 44913836 --- -- ---xx gl2005-03-12t03.flac 6:17.55 66632204 --- -- ---xx gl2005-03-12t04.flac 3:40.49 38923292 --- -- ---xx gl2005-03-12t05.flac 2:03.55 21826604 --- -- ---xx gl2005-03-12t06.flac 23:54.58 253094280 B (totals for 6 files, 0.4980 overall compression ratio) Creating a .st5 file: Test the files and run a len check before you do this. st5 is a new name for something known as a SHNtool md5. it is a checksum for verifying only decompressed audio from a lossless file. .st5 is identical to .ffp except formatting of the text. 1. Go to Create checksum file > Browse 2. Select the folder containing the lossless files you want to fingerprint. Press Add use Files of type to change the correct format. Select all the files you want to fingerprint and press Open 3. From the dropdown menu on the right, select md5 fp then hit Create ![]() 4. The checksums will take a few minutes to calculate. ![]() 5. Save your .st5 to the same folder. ![]() Code:
c0437a74a414351fb1fb4e4cece99e06 [shntool] gl2005-03-12t01.flac 1822d349a6be042a49af9bb0085e093c [shntool] gl2005-03-12t02.flac 5028343d2820679c23a99cc0e2f6e1f5 [shntool] gl2005-03-12t03.flac c485a617b3af2e4267753d334b709fa2 [shntool] gl2005-03-12t04.flac fd6e94f4c89685e8b41e4df2c073fb48 [shntool] gl2005-03-12t05.flac 2c3d244df2a8cb4aab370e26e4f3f393 [shntool] gl2005-03-12t06.flac Test the files and run a len check before you do this. 1. Go to Create checksum file > Browse 2. Select the folder containing the FLACs you want to fingerprint. Press Add use Files of type to change format to FLAC. Select all the files you want to fingerprint and press Open 3. From the dropdown menu on the right, select flac fp then hit Create ![]() 4. Since the ffp checksums were calculated during initial coding they can be very quickly extracted from the headers and put into a list. FLAC files should also be tested to make sure they can be decoded properly (see "Testing lossless files" above). ![]() 5. Save your .ffp to the same folder. ![]() ffp is generated by FLAC itself. during encoding a checksum is generated from the decoded audio only for each file and stored in the header. the checksums are the same as a .st5 (aka SHNtool md5) just the formatting of the .ffp is slightly different. Code:
gl2005-03-12t01.flac:c0437a74a414351fb1fb4e4cece99e06 gl2005-03-12t02.flac:1822d349a6be042a49af9bb0085e093c gl2005-03-12t03.flac:5028343d2820679c23a99cc0e2f6e1f5 gl2005-03-12t04.flac:c485a617b3af2e4267753d334b709fa2 gl2005-03-12t05.flac:fd6e94f4c89685e8b41e4df2c073fb48 gl2005-03-12t06.flac:2c3d244df2a8cb4aab370e26e4f3f393 .md5 checksums are only used for video at TTD. ![]() 1. Go to Create checksum file tab. 2. Press Browse. Select the folder containing the VIDEO_TS folder of the DVD you want to create checksums for. It is best not to select the VIDEO_TS folder, but instead the folder which contains the artwork, info and VIDEO_TS folder. 3. Press Add then use Files of type to change format to all. Double-click on the VIDEO_TS folder, select all the files inside that folder then press Open 4. From the dropdown menu on the right, select md5 sig (default). 5. Review that the files and folder are all correct, then hit Create. 6. Save your .md5 to the folder which contains the artwork (if you have), info and VIDEO_TS folder and not inside the VIDEO_TS folder. In the example, you would want to save to a folder called 1971.09.09 Hampton. Code:
8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.BUP 8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.IFO e40bd45c55f783d519c2a5c9aa088e98 *VIDEO_TS/VIDEO_TS.VOB e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.BUP e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.IFO da29301176f55de40d79be5cba560d06 *VIDEO_TS/VTS_01_0.VOB 99ba41ff0fec88e3def109b6cb824dcb *VIDEO_TS/VTS_01_1.VOB 6c20fa2d2232106310d61dce30cd3e29 *VIDEO_TS/VTS_01_2.VOB
__________________
Checksums Demystified | ask for help in Technobabble thetradersden.org | ttd recommended free software/freeware webring shntool tlh eac foobar2000 spek audacity cdwave vlc Quote:
No members have liked this post.
|
#2
|
||||
|
||||
Re: Checksums Tutorial (.st5, .ffp and .md5)
this is going to get put into the FAQ sometime soon.
please let me know if you find any type-o's/fuckups!
__________________
Checksums Demystified | ask for help in Technobabble thetradersden.org | ttd recommended free software/freeware webring shntool tlh eac foobar2000 spek audacity cdwave vlc Quote:
No members have liked this post.
|
#3
|
||||
|
||||
Re: Checksums Tutorial (.st5, .ffp and .md5)
A couple notes from the Checksums section (comments in bold):
.st5 aka SHNtool md5. this is a checksum taken from the decompressed audio only. the advantage to this is that it works with any lossless codec at any compression setting. This is the preferred checksum type to be included with shn or ape files. .ffp aka FLAC fingerprints. These checksums are virtually identical to .st5. some differences are explained below, under "Verifying a checksum file" and "Creating a .ffp file". For audio, I highly recommend using FLAC and including .st5 and .ffp no wholefile .md5 necessary. Except the 32-digit checksum is going to be exactly the same in the st5 and ffp file. There is really no reason to include both types of checksums for fingerprints, just one type. Here is a quick tutorial on making shntool md5s on a Mac using xACT: 1. Locate files on your computer. ![]() 2. Open xACT to the checksum tab and drag the files into the window. ![]() 3. Hit the Checksum button. ![]() 4. Hit the shntool md5 button (verify is the default). ![]() 5. It prompts you to save it somewhere, find the same folder. ![]() 6. Name the md5 you are creating properly and hit Save (a more common extension these days is ".st5") ![]() 7. It makes the shntool md5s (it will take a couple of minutes). ![]()
__________________
ABT Seeding Policy | ABT Seeding Guide | VBT Seeding Policy | VBT Seeding Guide Software Links HERE | FAQ is HERE | Technobabble forum is HERE. If your port number is in red, you are firewalled! How to Set Up Your Router (so you aren't firewalled) and How to Port Forward on Windows XP Five's Checksums Demystified - everything and anything you want to know about checksums
On a Mac? Get XLD to rip your CDs. Please see this guide - X Lossless Decoder (XLD): How to create flawless CD rips on Mac OS X No members have liked this post.
|
#4
|
||||
|
||||
Re: Checksums Tutorial (.st5, .ffp and .md5)
nice job five
![]() No members have liked this post.
|
#5
|
||||
|
||||
Re: Checksums Tutorial (.st5, .ffp and .md5)
![]() ![]() No members have liked this post.
|
#6
|
||||
|
||||
Re: Checksums Tutorial (.st5, .ffp and .md5)
Quote:
I added in "This is the preferred checksum type to be included with FLAC, SHN or APE files." in the st5 section. The main reason is that when you double-click a .ffp for testing using TLH, it extracts the checksums hidden in each FLAC file's header and compares that to what is in the .ffp file to see if it matches (takes about three seconds). Using the .ffp this way for testing does not test the audio content whatsoever (which could be corrupt after a successful .ffp verification). A second stage of testing is necessary in this case, it is "test" mode, which decompresses the audio to a temp file and generates a fresh hash from this and compares that to the the checksum hidden in the header. So, only after the two-stage test the files are verified. Double-clicking on a .st5 for verification using TLH results in the audio content of the files to be tested being decompressed to a temp file (temporarily) while a fresh hash is generated and compared to the corresponding checksums contained in the .st5 file. So the .st5 test is only one step and verifies both (a) that the files will decompress properly (as with "test" mode) and (b) that the checksums contained in the .st5 file match hashes freshly generated from that decompressed audio. When a .st5 verifies 100% okay everything is tested except the header isn't looked at, thus the .ffp. The .ffp is admittedly a little superfluous, but since it is only 1k and takes about three seconds to generate and verify its not really much to ask. oww, my brain!
__________________
Checksums Demystified | ask for help in Technobabble thetradersden.org | ttd recommended free software/freeware webring shntool tlh eac foobar2000 spek audacity cdwave vlc Quote:
No members have liked this post.
|
#7
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
so, to burn a data disc that is standalone compatable using the EXTRAS_TS method, would select your disc type as DVD-ROM (ISO), DVD-ROM (UDF), or DVD-ROM (ISO/UDF)?
[...and the answer is: DVD-ROM(UDF) ] No members have liked this post.
|
#8
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
There's also the excellent md5check perl script (and compiled version) that's available here:
http://etree-scripts.sourceforge.net/ I installed cygwin on my Win XP machine and use scripts and command line stuff to do all my lossless operations. Much cleaner than dealing with GUIs for me. No members have liked this post.
|
#9
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
I never could get md5check running for some reason, but have heard it is really great, you just need a bit more of a grasp of commandline than I have! If somebody wants to post instructions for that it would be appreciated.
Quote:
__________________
Checksums Demystified | ask for help in Technobabble thetradersden.org | ttd recommended free software/freeware webring shntool tlh eac foobar2000 spek audacity cdwave vlc Quote:
No members have liked this post.
|
#10
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
On a Mac, when using Toast, if I want to add an EXTRAS_TS folder, I go to Data > DVD-ROM (UDF).
__________________
ABT Seeding Policy | ABT Seeding Guide | VBT Seeding Policy | VBT Seeding Guide Software Links HERE | FAQ is HERE | Technobabble forum is HERE. If your port number is in red, you are firewalled! How to Set Up Your Router (so you aren't firewalled) and How to Port Forward on Windows XP Five's Checksums Demystified - everything and anything you want to know about checksums
On a Mac? Get XLD to rip your CDs. Please see this guide - X Lossless Decoder (XLD): How to create flawless CD rips on Mac OS X No members have liked this post.
|
#11
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
ack... okay, maybe its DVD-ROM (UDF) on PC, too. it is the default when you start up Nero. My DVD burner is on my other computer, I can check it later and post back.
__________________
Checksums Demystified | ask for help in Technobabble thetradersden.org | ttd recommended free software/freeware webring shntool tlh eac foobar2000 spek audacity cdwave vlc Quote:
No members have liked this post.
|
#12
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
hey
it is DVD-ROM (ISO) mode in Nero that I am using for the EXTRAS_TS method... works great on comp and standalone ![]() No members have liked this post.
|
#13
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
Quote:
No members have liked this post.
|
#14
|
||||
|
||||
Re: Checksums Demystified (.st5, .ffp and .md5)
![]() No members have liked this post.
|
#15
|
|||
|
|||
Re: Checksums Demystified (.st5, .ffp and .md5)
Great job Jamie.
Mike No members have liked this post.
|
![]() |
The Traders' Den |
![]() |
||||
Thread | Forum | Replies | Last Post | |
Where to download checksums? - popeye | Technobabble | 1 | 2006-12-03 11:24 AM | |
creating checksums - Music 2 My Ears | Technobabble | 2 | 2006-07-05 12:24 AM | |
How do I get Checksums? - Scott | Technobabble | 7 | 2005-09-09 04:54 PM |
Thread Tools | |
|
|