Checksums Demystified (.st5, .ffp and .md5) [Archive]

View Full Version : Checksums Demystified (.st5, .ffp and .md5)

Five

2006-02-09, 03:49 PM

TTD Requirements:
SHN/APE: .st5 fingerprints posted in announce thread, should also be included with the show. Since SHN format is not capable of self-testing, some form of checksums must be included.
FLAC: .st5/.ffp fingerprints posted in announce thread, should also be included with the show.
DVD: .md5 checksums posted in announce thread and included with the show.

Checksums (.st5, .ffp and .md5)

Checksums are used for the purpose of verifying the integrity and identity of a digital copy of audio. There are many kinds of checksums but only a few that are commonly used by music traders:

.md5 wholefile checksum. verifies that files are identical in every single way, and is therefore sensitive to compression setting, file format et al.

.st5 aka SHNtool md5. this is a checksum taken from the decompressed audio only. the advantage to this is that it works with any lossless codec at any compression setting. This is the preferred checksum type to be included with FLAC, SHN or APE files.

.ffp aka FLAC fingerprints. These checksums are virtually identical to .st5. some differences are explained below, under "Verifying a checksum file" and "Creating a .ffp file".

For audio, I highly recommend using FLAC and including .st5 and .ffp no wholefile .md5 necessary.

For Mac use xAct. Tutorials here:
Making ShnTool md5s using xACT (http://www.thetradersden.org/forums/showthread.php?p=300188#post300188)
http://u2lynne.sandsmuseum.com/checksums.html

For PC use TLH:
http://tlh.easytree.org/

TLH checksums tutorial

Testing lossless files:

This function runs a simple test to see if the files will decode properly.

1. Go to Test encoded files > Add

2. Use Files of type to select the lossless file type.

3. Select all and hit open.

http://img221.echo.cx/img221/646/test18ia.th.gif (http://img221.echo.cx/my.php?image=test18ia.gif)

4. Check that the information displayed in the list is correct.

5. Press Test

6. The result will take a few minutes to display.

http://img221.echo.cx/img221/1386/test25xi.th.gif (http://img221.echo.cx/my.php?image=test25xi.gif)

Verifying a checksum file:

TLH can verify .st5, .ffp and .md5 files. You can simply double-click the checksum file to begin testing, or:

1. Go to Verify checksum file > Add

2. Use Files of type to select checksum type. Load the checksum file.

3. Press Test.

4. This will take a few minutes.

ffp compares the checksums in the FLAC header without checking the audio.

http://img288.echo.cx/img288/2185/verifyffp16wb.th.gif (http://img288.echo.cx/my.php?image=verifyffp16wb.gif) http://img288.echo.cx/img288/4466/verifyffp24rv.th.gif (http://img288.echo.cx/my.php?image=verifyffp24rv.gif) http://img288.echo.cx/img288/2671/verifyffp36hk.th.gif (http://img288.echo.cx/my.php?image=verifyffp36hk.gif)

.md5 checks for any change whatsoever in the whole file (useful for DVD).

.st5 checks the decompressed audio only.

http://img288.echo.cx/img288/2999/verifyst513hk.th.gif (http://img288.echo.cx/my.php?image=verifyst513hk.gif) http://img244.echo.cx/img244/1438/verifyst526nm.th.gif (http://img244.echo.cx/my.php?image=verifyst526nm.gif)

md5 for DVD cannot find the files, what do I do?

To verify a playable video DVD with checksums which are on your hd, generate md5 checksums (see below) directly from the DVD and then test those against the files which are on your hd.

Another problem that sometimes occurs is that the paths stored in the .md5 file do not tell TLH to look inside the VIDEO_TS folder. So if the .md5 file for your DVD looks like this:

8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS.BUP
8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS.IFO
e40bd45c55f783d519c2a5c9aa088e98 *VIDEO_TS.VOB
e11aaf0c14501bbf15d776a34218c205 *VTS_01_0.BUP
e11aaf0c14501bbf15d776a34218c205 *VTS_01_0.IFO
da29301176f55de40d79be5cba560d06 *VTS_01_0.VOB
99ba41ff0fec88e3def109b6cb824dcb *VTS_01_1.VOB
6c20fa2d2232106310d61dce30cd3e29 *VTS_01_2.VOB

you should (a) make a backup of the original .md5 then (b) add VIDEO_TS/ to the path (open, edit & save using notepad), and test again. The modified .md5 file for the example posted above would look like this:

8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.BUP
8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.IFO
e40bd45c55f783d519c2a5c9aa088e98 *VIDEO_TS/VIDEO_TS.VOB
e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.BUP
e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.IFO
da29301176f55de40d79be5cba560d06 *VIDEO_TS/VTS_01_0.VOB
99ba41ff0fec88e3def109b6cb824dcb *VIDEO_TS/VTS_01_1.VOB
6c20fa2d2232106310d61dce30cd3e29 *VIDEO_TS/VTS_01_2.VOB

Be very careful not to alter the 32-digit checksums or filenames, just add "VIDEO_TS/" (no quotes) as above.

Advanced: Another possibility is to burn a standalone-playable data DVD which contains info, artwork (if any) and .md5 checksums inside a folder called EXTRAS_TS. To do this, create three folders in your compilation:

AUDIO_TS
EXTRAS_TS
VIDEO_TS

The VIDEO_TS folder contains the obvious, the AUDIO_TS folder must be there and remain absolutely empty, and the EXTRAS_TS with the art/info/checksums inside. When the .md5 file is placed inside of an EXTRAS_TS folder, the path must be formatted as:

8d50cba158b40a75dd32cada8964aef7 *../VIDEO_TS/VIDEO_TS.BUP
8d50cba158b40a75dd32cada8964aef7 *../VIDEO_TS/VIDEO_TS.IFO
e40bd45c55f783d519c2a5c9aa088e98 *../VIDEO_TS/VIDEO_TS.VOB
e11aaf0c14501bbf15d776a34218c205 *../VIDEO_TS/VTS_01_0.BUP
e11aaf0c14501bbf15d776a34218c205 *../VIDEO_TS/VTS_01_0.IFO
da29301176f55de40d79be5cba560d06 *../VIDEO_TS/VTS_01_0.VOB
99ba41ff0fec88e3def109b6cb824dcb *../VIDEO_TS/VTS_01_1.VOB
6c20fa2d2232106310d61dce30cd3e29 *../VIDEO_TS/VTS_01_2.VOB

If you wish you can substitute \ for /. The end result is a DVD which is playable in most standalones & also includes all the stuff which is normally included in a trade. Another big advantage is that you can double-click the .md5 which is inside the EXTRAS_TS folder and TLH will test all of the files on that DVD. Just be very careful that you get everything correct the same way I explained it above or it will not work.

thanks to fatoldpig and yoho for suggesting this stuff!

Running a len (length) check:

This gives some helpful details about the set's compatability with audio cdr format. This is primarily used to check for SBEs (Sector Boundary Errors), and that the files are 16bit/44.1kHz.

For more detailed information about the meaning of the data, check here:
http://www.etree.org/shnutils/shntool/support/doc/shntool.txt

1. Go to Audio file Details > Add

2. Use Files of type to select the lossless file type.

3. Select all and hit open.

http://img244.echo.cx/img244/2027/len17wl.th.gif (http://img244.echo.cx/my.php?image=len17wl.gif)

4. Check that the information displayed in the list is correct.

5. Press Show details

http://img244.echo.cx/img244/1362/len20zm.th.gif (http://img244.echo.cx/my.php?image=len20zm.gif)

6. Copy and paste into your info .txt file.

length expanded size cdr WAVE problems filename
2:15.26 23875196 --- -- ---xx gl2005-03-12t01.flac
5:22.52 56923148 --- -- ---xx gl2005-03-12t02.flac
4:14.46 44913836 --- -- ---xx gl2005-03-12t03.flac
6:17.55 66632204 --- -- ---xx gl2005-03-12t04.flac
3:40.49 38923292 --- -- ---xx gl2005-03-12t05.flac
2:03.55 21826604 --- -- ---xx gl2005-03-12t06.flac
23:54.58 253094280 B (totals for 6 files, 0.4980 overall compression ratio)

Creating a .st5 file:

Test the files and run a len check before you do this.

st5 is a new name for something known as a SHNtool md5. it is a checksum for verifying only decompressed audio from a lossless file. .st5 is identical to .ffp except formatting of the text.

1. Go to Create checksum file > Browse

2. Select the folder containing the lossless files you want to fingerprint. Press Add use Files of type to change the correct format. Select all the files you want to fingerprint and press Open

3. From the dropdown menu on the right, select md5 fp then hit Create

http://img28.echo.cx/img28/568/createst514gb.th.gif (http://img28.echo.cx/my.php?image=createst514gb.gif)

4. The checksums will take a few minutes to calculate.

http://img28.echo.cx/img28/3979/createst528ow.th.gif (http://img28.echo.cx/my.php?image=createst528ow.gif)

5. Save your .st5 to the same folder.

http://img28.echo.cx/img28/4815/createst534xv.th.gif (http://img28.echo.cx/my.php?image=createst534xv.gif)

c0437a74a414351fb1fb4e4cece99e06 [shntool] gl2005-03-12t01.flac
1822d349a6be042a49af9bb0085e093c [shntool] gl2005-03-12t02.flac
5028343d2820679c23a99cc0e2f6e1f5 [shntool] gl2005-03-12t03.flac
c485a617b3af2e4267753d334b709fa2 [shntool] gl2005-03-12t04.flac
fd6e94f4c89685e8b41e4df2c073fb48 [shntool] gl2005-03-12t05.flac
2c3d244df2a8cb4aab370e26e4f3f393 [shntool] gl2005-03-12t06.flac

Creating a .ffp file:

Test the files and run a len check before you do this.

1. Go to Create checksum file > Browse

2. Select the folder containing the FLACs you want to fingerprint. Press Add use Files of type to change format to FLAC. Select all the files you want to fingerprint and press Open

3. From the dropdown menu on the right, select flac fp then hit Create

http://img257.echo.cx/img257/875/createffp13xu.th.gif (http://img257.echo.cx/my.php?image=createffp13xu.gif)

4. Since the ffp checksums were calculated during initial coding they can be very quickly extracted from the headers and put into a list. FLAC files should also be tested to make sure they can be decoded properly (see "Testing lossless files" above).

http://img257.echo.cx/img257/5468/createffp21sz.th.gif (http://img257.echo.cx/my.php?image=createffp21sz.gif)

5. Save your .ffp to the same folder.

http://img257.echo.cx/img257/7285/createffp31dm.th.gif (http://img257.echo.cx/my.php?image=createffp31dm.gif)

ffp is generated by FLAC itself. during encoding a checksum is generated from the decoded audio only for each file and stored in the header. the checksums are the same as a .st5 (aka SHNtool md5) just the formatting of the .ffp is slightly different.

gl2005-03-12t01.flac:c0437a74a414351fb1fb4e4cece99e06
gl2005-03-12t02.flac:1822d349a6be042a49af9bb0085e093c
gl2005-03-12t03.flac:5028343d2820679c23a99cc0e2f6e1f5
gl2005-03-12t04.flac:c485a617b3af2e4267753d334b709fa2
gl2005-03-12t05.flac:fd6e94f4c89685e8b41e4df2c073fb48
gl2005-03-12t06.flac:2c3d244df2a8cb4aab370e26e4f3f393

Creating a .md5 file for a DVD:

.md5 checksums are only used for video at TTD.

http://img432.imageshack.us/img432/4931/md5dvd15gd.th.gif (http://img432.imageshack.us/my.php?image=md5dvd15gd.gif)

1. Go to Create checksum file tab.

2. Press Browse. Select the folder containing the VIDEO_TS folder of the DVD you want to create checksums for. It is best not to select the VIDEO_TS folder, but instead the folder which contains the artwork, info and VIDEO_TS folder.

3. Press Add then use Files of type to change format to all. Double-click on the VIDEO_TS folder, select all the files inside that folder then press Open

4. From the dropdown menu on the right, select md5 sig (default).

5. Review that the files and folder are all correct, then hit Create.

6. Save your .md5 to the folder which contains the artwork (if you have), info and VIDEO_TS folder and not inside the VIDEO_TS folder. In the example, you would want to save to a folder called 1971.09.09 Hampton.

8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.BUP
8d50cba158b40a75dd32cada8964aef7 *VIDEO_TS/VIDEO_TS.IFO
e40bd45c55f783d519c2a5c9aa088e98 *VIDEO_TS/VIDEO_TS.VOB
e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.BUP
e11aaf0c14501bbf15d776a34218c205 *VIDEO_TS/VTS_01_0.IFO
da29301176f55de40d79be5cba560d06 *VIDEO_TS/VTS_01_0.VOB
99ba41ff0fec88e3def109b6cb824dcb *VIDEO_TS/VTS_01_1.VOB
6c20fa2d2232106310d61dce30cd3e29 *VIDEO_TS/VTS_01_2.VOB

Five

2006-02-09, 03:50 PM

this is going to get put into the FAQ sometime soon.

please let me know if you find any type-o's/fuckups!

U2Lynne

2006-02-09, 04:28 PM

A couple notes from the Checksums section (comments in bold):

.st5 aka SHNtool md5. this is a checksum taken from the decompressed audio only. the advantage to this is that it works with any lossless codec at any compression setting. This is the preferred checksum type to be included with shn or ape files.

.ffp aka FLAC fingerprints. These checksums are virtually identical to .st5. some differences are explained below, under "Verifying a checksum file" and "Creating a .ffp file".

For audio, I highly recommend using FLAC and including .st5 and .ffp no wholefile .md5 necessary. Except the 32-digit checksum is going to be exactly the same in the st5 and ffp file. There is really no reason to include both types of checksums for fingerprints, just one type.

Here is a quick tutorial on making shntool md5s on a Mac using xACT:

1. Locate files on your computer.
http://www.thetradersden.org/images/thumb/xactshntoolmd51.jpg (http://www.thetradersden.org/images/xactshntoolmd51.jpg)

2. Open xACT to the checksum tab and drag the files into the window.
http://www.thetradersden.org/images/thumb/xactshntoolmd52.jpg (http://www.thetradersden.org/images/xactshntoolmd52.jpg)

3. Hit the Checksum button.
http://www.thetradersden.org/images/thumb/xactshntoolmd53.jpg (http://www.thetradersden.org/images/xactshntoolmd53.jpg)

4. Hit the shntool md5 button (verify is the default).
http://www.thetradersden.org/images/thumb/xactshntoolmd54.jpg (http://www.thetradersden.org/images/xactshntoolmd54.jpg)

5. It prompts you to save it somewhere, find the same folder.
http://www.thetradersden.org/images/thumb/xactshntoolmd55.jpg (http://www.thetradersden.org/images/xactshntoolmd55.jpg)

6. Name the md5 you are creating properly and hit Save (a more common extension these days is ".st5")
http://www.thetradersden.org/images/thumb/xactshntoolmd56.jpg (http://www.thetradersden.org/images/xactshntoolmd56.jpg)

7. It makes the shntool md5s (it will take a couple of minutes).
http://www.thetradersden.org/images/thumb/xactshntoolmd57.jpg (http://www.thetradersden.org/images/xactshntoolmd57.jpg)

bubbybooshay

2006-02-09, 04:36 PM

nice job five :clap:

AAR.oner

2006-02-09, 05:41 PM

:banger2: :cheers:

Five

2006-02-11, 02:18 PM

A couple notes from the Checksums section (comments in bold):

.st5 aka SHNtool md5. this is a checksum taken from the decompressed audio only. the advantage to this is that it works with any lossless codec at any compression setting. This is the preferred checksum type to be included with shn or ape files.

.ffp aka FLAC fingerprints. These checksums are virtually identical to .st5. some differences are explained below, under "Verifying a checksum file" and "Creating a .ffp file".

For audio, I highly recommend using FLAC and including .st5 and .ffp no wholefile .md5 necessary. Except the 32-digit checksum is going to be exactly the same in the st5 and ffp file. There is really no reason to include both types of checksums for fingerprints, just one type.

thanks

I added in "This is the preferred checksum type to be included with FLAC, SHN or APE files." in the st5 section.

The main reason is that when you double-click a .ffp for testing using TLH, it extracts the checksums hidden in each FLAC file's header and compares that to what is in the .ffp file to see if it matches (takes about three seconds). Using the .ffp this way for testing does not test the audio content whatsoever (which could be corrupt after a successful .ffp verification). A second stage of testing is necessary in this case, it is "test" mode, which decompresses the audio to a temp file and generates a fresh hash from this and compares that to the the checksum hidden in the header. So, only after the two-stage test the files are verified. Double-clicking on a .st5 for verification using TLH results in the audio content of the files to be tested being decompressed to a temp file (temporarily) while a fresh hash is generated and compared to the corresponding checksums contained in the .st5 file. So the .st5 test is only one step and verifies both (a) that the files will decompress properly (as with "test" mode) and (b) that the checksums contained in the .st5 file match hashes freshly generated from that decompressed audio. When a .st5 verifies 100% okay everything is tested except the header isn't looked at, thus the .ffp. The .ffp is admittedly a little superfluous, but since it is only 1k and takes about three seconds to generate and verify its not really much to ask.

oww, my brain!

AAR.oner

2006-03-19, 07:02 AM

so, to burn a data disc that is standalone compatable using the EXTRAS_TS method, would select your disc type as DVD-ROM (ISO), DVD-ROM (UDF), or DVD-ROM (ISO/UDF)?

[...and the answer is: DVD-ROM(UDF) ]

GregY

2006-03-19, 07:14 AM

There's also the excellent md5check perl script (and compiled version) that's available here:

http://etree-scripts.sourceforge.net/

I installed cygwin on my Win XP machine and use scripts and command line stuff to do all my lossless operations. Much cleaner than dealing with GUIs for me.

Five

2006-03-19, 01:16 PM

I never could get md5check running for some reason, but have heard it is really great, you just need a bit more of a grasp of commandline than I have! If somebody wants to post instructions for that it would be appreciated.

so, to burn a data disc that is standalone compatable using the EXTRAS_TS method, would select your disc type as DVD-ROM (ISO), DVD-ROM (UDF), or DVD-ROM (ISO/UDF)?
Just use the default that Nero selects when you boot up, I believe that's "DVD-ROM (ISO)", the setting you would use for burning normal non-video stuff. It works great with my DVD player and it is sweet being able to get all the info & artwork right off the same playable disc. Also great to just double-click the md5 there and the DVD self-tests.

U2Lynne

2006-03-19, 01:23 PM

On a Mac, when using Toast, if I want to add an EXTRAS_TS folder, I go to Data > DVD-ROM (UDF).

Five

2006-03-19, 01:27 PM

ack... okay, maybe its DVD-ROM (UDF) on PC, too. it is the default when you start up Nero. My DVD burner is on my other computer, I can check it later and post back.

Five

2006-04-17, 04:08 AM

hey

it is DVD-ROM (ISO) mode in Nero that I am using for the EXTRAS_TS method... works great on comp and standalone :thumbsup

AAR.oner

2006-04-17, 06:33 AM

hey

it is DVD-ROM (ISO) mode in Nero that I am using for the EXTRAS_TS method... works great on comp and standalone :thumbsup
cheers! i'll give that one a try then Five

karmakat

2006-04-17, 07:02 AM

:clap: Good job Five. About six months too late (for my personal use)...:run:

mike1061

2006-05-16, 11:20 PM

Great job Jamie.
Mike

cmaz

2006-05-17, 03:48 PM

aristote

2007-05-16, 01:07 PM

:thumbsup

Five

2007-05-16, 09:23 PM

i have a question regarding the difference between verifying and testing with TLH. If i verify the files, do i also need to test them? If the files verify, won't they neccessarily decode properly? And what about the check box "full ffp vrf"? When, where, and why do i choose each of these options?

When do i verify, test, and/or choose "full ffp"?

Thanks!
since this tutorial was written there have been some new features added to TLH... if you test with the "full" feature enabled (as it is by default) TLH will check the files every way possible and only that one test/verification is necessary.

shovelhead

2008-04-05, 01:19 PM

WAY COOL!!!! THANK YOU SO VERY MUCH FOR YOUR TIME IN POSTING & GENEROSITY IN SHARING!!! WAY COOL!!!! THANKS AGAIN!!!! :cool::cool::cool::cool:

salmonella

2012-06-27, 10:53 PM

Is there any chance of updating the initial post in this thread? I'm trying to make a SHNTOOL md5 with xACT, however, the interface looks completely different. The example given is not helpful 6 years later.

I sincerely hope nobody takes this as complaining on my part. I know you all want this to be as helpful as possible.

Thank you.

scb

2012-07-01, 01:27 PM

in xACT 2.x, on the Checksum tab, click "Create Checksum File" and choose the "St5" option

AAR.oner

2012-07-01, 03:48 PM

what scb said ^^

select the files you wanna include in Finder, drag to the window on left in xACT [screen below], then select the options circled in red...click "Checksum" button and let it do its thing

Five

2012-07-09, 02:13 AM

SteelStage

2015-08-11, 12:56 AM

jabulon

2015-08-11, 03:18 AM

hey, read the FAQ's and having issues creating uploaded torrent

Have created ffp and st5 with TLH, can't figure out the shntool nor how to get it working

Getting this

The fingerprints provided are not shntool st5s or flac fingerprints

any more help would be appreciated.

Are you talking about audio or video? I assume audio, because you talk about flac. Windows?

ps edit: forget about the shntool, when creating the torrent file just add the ffp and/or st5 that is good enough.

SteelStage

2015-08-11, 07:59 AM

yes audio, flac and yes windows.

when i create the torrent (with utorrent engine), then attach and load it tells me the message i tried above. Maybe my torrent is bad, when i create the torrent, i direct the folder with all files, do i need to add all the files?

jabulon

2015-08-11, 09:24 AM

Try this: Make the torrent file with TLH as well.

open Torrent select create torrent file.
add directory
check private torrent
tracker/announce (select ttd tracker)
save torrent file (set the torrent location)
create

SteelStage

2015-08-11, 09:26 AM

THanks I'll def try that and let u know

SteelStage

2015-08-11, 11:54 AM

Tried TLH for torrent,

getting this error upon upload now: "Forget something? How about some fingerprints, please."

In the file directory i have ffp and st5 files, is that correct?

jabulon

2015-08-11, 12:02 PM

You have to copy and past the ffp (the text - open the ffp-file with text editor /notepad) in the second big box on the upload page

SteelStage

2015-08-11, 01:01 PM

I'm good now thanks for you help man. I will seed some shows

jabulon

2015-08-11, 01:03 PM

:thumbsup

SteelStage

2015-08-11, 01:16 PM

one more thing, the initial seed, that seems to be stuck, any guidance on that?

I did download from the TTD page it was linked in url, then i added it to utorrent as initial seed...see it in download status 99.9 and not in seed page, but it progressed to 'finding peers' this has always baffled me doing this once before

so i think i am stuck at step 11 in audio torrent seed FAQs

jabulon

2015-08-11, 01:30 PM

The 99.9% etc comes when you have changed something after making the torrent. Delete this torrent. Make a new torrent (make sure all files are closed and saved when making the torrent. Then follow the upload procedure. I'll delete your old upload page.

SteelStage

2015-08-11, 01:37 PM

thanks i'll do that

SteelStage

2015-08-11, 01:54 PM

and we're good. Much appreciation and patience jabulon

jabulon

2015-08-11, 01:56 PM

:thumbsup

Blackdeath

2019-12-03, 12:53 PM

hello i need to change a md5 to ffp in a torrent that is already on the tracker?? does a MOD need to delete the current torrent before i upload the new torrent?

TheMysticMuse

2019-12-04, 03:15 PM

hello i need to change a md5 to ffp in a torrent that is already on the tracker?? does a MOD need to delete the current torrent before i upload the new torrent?

Yes, unfortunately when a torrent is created there's nothing you can add/modify in some way. That's why it's super important that when you've created the torrent all the files are 100 % ready to be uploaded. So no, you can't change/add change anything when you've uploaded a torrent. Tell the mods to delete the thread then create a new one with the fully working .ffp. Cheers and good luck!

Blackdeath

2019-12-04, 06:11 PM

Yes, unfortunately when a torrent is created there's nothing you can add/modify in some way. That's why it's super important that when you've created the torrent all the files are 100 % ready to be uploaded. So no, you can't change/add change anything when you've uploaded a torrent. Tell the mods to delete the thread then create a new one with the fully working .ffp. Cheers and good luck!thank you sir..ive been looking through the FAQ for this answer..:thumbsup:thumbsup:thumbsup

gwendibbley

2020-04-15, 09:22 AM

Five

2020-04-15, 03:35 PM

Okey, I have a confession to make.

I haven't always checked the fingerprints and checksums for downloaded lossless audio files.

But now I want to do that, and to filter out the bad ones.
hello Hendrick :wave:

Some of the bootlegs I have do have st5 and ffp, but some only have md5 and some don't even have any fingerprints/checksums.

(I have flac, shn and ape-files)

1. How do I ascertain that my files are correct? TLH -> Analysis -> Test audio encoded files and Check audio files for SBEs?
yes, run this for all files shn, flac, ape. somebody's gonna say 'nobody burns anymore...' well these days I see it more as a secret handshake, when you see shows with SBEs you know its a noob, and to check everything about that show over.

Running "flac -t" in a command prompt in Windows?
use st5 if available, test them all. the st5 checksum is absolute, track01.wav, track01.flac, track01.ape, track01.shn when you generate st5 for any one of them it makes the same number.

if there is no st5:
flac & ape: self-test, then generate st5 and keep together with the files.

shn:
a) if there is .md5 with shn, test with that, then generate .st5 and add it to the folder.
b) if there is no .md5 with shn then open with audacity or some other wav editor and check it over, make sure there's not obvious corrupt data anywhere. listening all the way through if you have time. for 'rush job' tlh shn self-test, which detects shns that are so badly broken they will not fully decompress (iirc).

2. If the above solution(s) return a positive result, can I then create a st5 (and ffp) for those audio files?
yes, certainly!

I have a lot of work ahead of me, but I want to do it right, and preferably only one time (for my collection now, of course I will do this for any future bootleg I download).

Hope you can help me out.

Best reagrds,
Henrik
let me know if this explains well, the topic gets confusing fast.

-Jamie

gwendibbley

2020-04-16, 06:38 AM

gwendibbley

2020-04-16, 10:43 AM

Five

2020-04-16, 03:55 PM

Thanks Five!

I Think I've figured it out..

Still have questions though:

If I discover SBEs and I don't know which flac level the original files where encoded with, what is the best re-encoding level after the fix? What settings for the fix should I use? Fix/pad? Shift forward/backward/round? Skip first N? Pad last file?
ffp is st5 that only works with flac. st5 is ffp for all lossless. it pulls out the raw wave file without the header and analyzes only that.

sbe fix default settings that should be backward shift and pad final track if necessary. oldschool 'proper procedure' is involved and basically profitless. besides, others who downloaded the same set will most likely choose this path of least resistance and their resulting fixed set will have matching st5 as yours without re-downloading. st5 is immune to compression level changes and also lossless codec changes.

it is good to see it for yourself.. take one flac file to a test folder, use tlh to encode level 1, level 6, level 8, decompress a wav, transcode to ape, mkw, shorten and put them all together in tlh then 'create checksums' generate. the same numbers over and over. md5 used this way is all screwed up.

if the md5 doesn't verify just take it out. if the md5 works I would still take it out, site policy is you can take it out or leave it in at your discretion but it must work.

Are there any point in leaving "old" ffp/md5/sfv if I create a st5-file?

Thanks,

H
the original st5 file often contains a date of creation time stamp that should be preserved. I feel it is helpful to collectors.

sfv is used with mp3 sets.. for this stuff, just copy whatever those ppl are doing. I hate the mp3-sfv functionality tlh is a shntool frontend and shntool is [/i]exclusively[/i] lossless, like ttd-etree-etc.

if you generate new st5 it should be identical to old st5 except for the date so you don't need a new st5 so long as the old one works.

shn sets only need st5 in the year 2020 .. md5 is 90s

sorry for the tl;dr .. help me boil this to 25 words or less if possible lol

Five

2020-04-16, 03:59 PM

Another question:

Can I run an analysis on shn-files in TLH? I currently have a bootleg where the analysis and the check for SBEs are ok, but the md5s are returning an error.

Should I create a st5-file and delete the md5-file(s)?

/H
most likely the md5 was working with the pre-fix set and the fixer didn't bother to update it. take out the broken md5, it doesn't need a new one, generate st5 and put that in.

Five

2020-04-16, 04:19 PM

for recompressing FLAC use most recent binaries if using TLH you can go into program files/trader's little helper/CmdlineApps folder and overwrite flac.exe with x86 or x64 version of flac.exe downloadable here:
https://ftp.osuosl.org/pub/xiph/releases/flac/
flac-1.3.2-win.zip 2017-01-01 01:58 1.2M (windows x64 and x86 inside)

I know a lot of ppl are still running the old 2011 version of TLH so its a way to be at least a little more up to date. the newer binaries have better compression with no disadvantage, no effect to st5 and ffp altho you won't be able to hop directly on a torrent containing the older version of the show. md5s will no longer work, just take them out. the flac container is designed to be updated, regularly with 0% change to the wav file inside, verifiable using self-test or st5, ffp (which triggers self-test externally).

if you want to use cmd add -f switch. upgrade your binaries in path.
flac -V -8 -f

Five

2020-04-17, 12:52 AM

you can run shntool len on shn files via tlh or cmd eg
shntool len *.shn > len.txt

change to *.ape or *.flac *.wav for those. even *.mkw if you want

hope this helps :wave:

gwendibbley

2020-04-18, 07:33 AM

I've added a flow chart as an attachment.

Did I get it right?

/H

Five

2020-04-18, 07:05 PM

excellent! :clap:

I love the flow chart, where is my 1981 flow chart ruler? :cool:

I would add a wav editor check and glance at the contents of any show with SBEs but just a three-point check, listen to 5 seconds from here and there. sbe fixes should be notated to lineage as well, someday maybe we could automate this.

TLH was supervised by TTD when it first started.. there used to be a 'full test' button we are trying to get Robert to put that back. The flow chart is tremendously helpful, thanks very much.

I want to map the shortest path of keypresses in TLH to achieve the desired (and necessary) "full test" and we can submit that to Robert and see if we can wake him up.

I am developing a TLH alternative but it is slow going eta 2022 and will put everything in my prototype so we can test and test and test and test and test.

gwendibbley

2020-04-19, 10:35 AM

I am developing a TLH alternative but it is slow going eta 2022 and will put everything in my prototype so we can test and test and test and test and test.

Let me know if there is anything I can do to help. I can beta test the damn thing! :lol:

I am not a software developer but I can try and make a script using it, if that’s possible.

I would love to be able to point to an entire root folder and have the program scan through all the files found in it, sub-folders and all.

/H

Five

2020-04-20, 05:36 PM

fantastic, I accept your offer

it is a script meant to inhabit other frontends. this way it eliminates the need to program a frontend, and the source code that matters always stays available, unlike TLH code which is locked inside.

It has only transcode, re-encode and verify functionality right now. I'm adding the fingerprints functionality but to get it to do that flowchart stuff is :hmm: :hmm: :hmm: atm.

I'll tidy my code post everything here for you in a few days.

gwendibbley

2020-04-21, 01:40 AM

In what language are you writing this code? Vbscript? Perl?

Five

2020-04-21, 02:15 AM

I guess it's called 'unix-like environment'?

https://www.softpedia.com/get/Multimedia/Audio/Other-AUDIO-Tools/Frontah.shtml

https://web.archive.org/web/20071130035411/http://home.vxu.se/mdati00/frontah/

this is the only one that lets me edit the menu the way I want, supports icons, runs multiline code straight out of the box. runs from any folder (best to stay out of Program Files). running very stable for a couple months here.

here is how to read/edit the default script
https://web.archive.org/web/20071130035411/http://home.vxu.se/mdati00/frontah/

I will post 'frontah_script.ini' that I have been working soon.

Five

2020-04-21, 02:38 AM

something else very interesting is the official ttd software called 'audiophilia' that came before TLH
https://sourceforge.net/projects/audiophilia/

there is not much use in downloading because nothing at all works :lol:

the author disappeared and is now living like JD Salinger.

this pic shows everything, all other tabs are blank.

gwendibbley

2020-04-22, 09:55 AM

Hey, what's with the vxu.se-address? Did you study in Sweden?

/H

Five

2020-04-22, 04:43 PM

nono that's the original programmer of frontah. the built-in functionality and visual design comes from him He is called 'madah' on hydrogenaud.io there are several old threads there talking about frontah. it seems like the typical story of a talented young free software developer who had to get back to real life and became uncontactable very suddenly.

so everything in there you've seen so far is his genius nothing of mine.

I was reading a lot of rarewares.org and installing every lossless codec I could get to work into frontah for fun and programming experience, when I realized shntool modules and multiline commands are available here, and that's about where I'm at now.

I recorded multitrack indie stuff, worked 10yrs of dialog editing and know the ttd/etree preferred practices. Not a formal education, but I got a lot of ideas that can translate to 'live show trading' beyond simply cloning TLH.

I removed all lossy functions and routines that could be harmful to the sort of FLAC content that we use. this is a pure lossless multi-frontend, just like shntool is a pure lossless program and there is nothing (yet) that exists in this category. the lossless for lossless' sake category. lossy can be done elsewhere, maybe even another frontah version if you want. it is not hard to find an mp3 encoder, it is hard to find something without it.

my 'script' (custom ini file) is getting very tidy now, but I broke something along the way. no harm, I have a backup that's working and with a little testing I hope to post working ini file here tomorrow.

here is a module you can try for now that I put together. take the frontah_script.ini out of frontah.dat using 7-zip or similar and paste this code to the very end (or anywhere you want for that matter) using notepad/whatever. keep frontah_script.ini in the same folder with frontah.exe

binaries are called from the path or frontah folder.

the is my flac re-encode at level 8 module. I want it to detect whether an overwrite is beneficial or not, as it is it just stupidly overwrites every file need it or not. there are icons, they are not necessary and it is easy to make your own if you want.

[reflac]
IconLarge = reflac32.bmp
IconSmall = reflac16.bmp
ModifiesInput = 1
DisableConvert = 1
LongDesc = Re-encode to FLAC
Menu = -Advanced\Recompress\Re-encode FLAC files
N0Cmd0 = FLAC.exe -V -8 -f -o '%o' -- '%i'
N0InExt = .flac
N0OutExt = .flac
ShortDesc = FLAC
StatusText = Re-encoding
TrackbarText = level 8 verified

Five

2020-04-22, 04:51 PM

and here is the 'verify' module, which will support more in the future.

right now flac, ape, wavpack, optimfrog

and waveform archiver (x86 only, untested) :lol:

this one has to be pasted to the end or at least not right at the beginning for icons to work. I have no idea why.

let me know if these are working for you, or just wait a little while longer for full script, about 50k

[verify]
IconLarge = verify32.bmp
IconSmall = verify16.bmp
ModifiesInput = 0
DisableConvert = 1
DisableTagging = 1
LongDesc = Verify
Menu = -Verify
N0Cmd0 = FLAC.exe -t '%i'
N0InExt = .flac
; N0ErrorCode = flac
N1Cmd0 = MAC.exe '%i' -v
N1InExt = .ape
; N1ErrorCode = monkeysaudio
N2Cmd0 = OFR.exe --check --md5 '%i'
N2InExt = .ofr
; N2ErrorCode = frog
N3Cmd0 = WVUNPACK.exe -v '%i'
N3InExt = .wv
; N3ErrorCode = wvunpack
N4Cmd0 = WA.exe t arc '%i'
N4InExt = .wa
; N4ErrorCode = wavarc
ShortDesc = Verify
StatusText = Verifying
TrackbarText = Test archive integrity

Five

2020-04-22, 07:08 PM

it is necessary to mention monkey's audio frontend since frontah is like an imitation version of that.

if you can be troubled, install v3.97f and look at what a nice frontend that is.
https://web.archive.org/web/20040615110412/https://monkeysaudio.com/files/

there is something terribly wrong with current versions of Monkey's Audio Frontend, be very careful stay away as of this writing. I had to reboot my computer it was so screwed up and it didn't even install.

Five

2020-04-23, 01:17 PM

to witness the problem that ttd staff have with a lack of pure lossless software witness here:
https://wieslawsoltes.github.io/BatchEncoder/

I am sending a n00b to use an encoder to listen to lossless files on his itunes. It is full of aac, mp3 and because he's new he could easily choose those and never hear the lossless files. he'll say "there it is! mp3, that's what I need" and all tapers feel sad inside because we produce lossless files in hopes that it will be listened to as lossless.

If tapers wanted you to hear mp3 they would give mp3 simple as that. The fanatical ttd members cannot control what you do at home but they hope you enjoy the most high fidelity representation of a great night out.

I'll stop carrying on now... here is what I've got, I appreciate anybody who helps test it or just take a look, feature suggestions welcome.

remember eta 2022 .. and next version will at minimum hide the codec museum or best case scenario be something you can switch on and off.

I'll give icons for the main functions on the left, which I will attempt to keep as few buttons as possible. put the bmps and frontah_script.ini in the same folder with frontah.dat and frontah.exe

much later a full icon set will be given, it is not very important right now.

Five

2020-04-23, 01:31 PM

;
; frontah v0.92 beta2 by maddah 2003
; weaponx script v0.011 by fivah 2020
; lossless archiver with interactive museum
;
# MD5 checksums generated by MD5summer (http://www.md5summer.org)
# Generated 2020-04-23 2:25:03 PM

379c0332b33952f4fb44fc4310330143 *frontah_script.ini
...

Five

2020-04-23, 01:35 PM

the binaries are out there, look on Jason Jordan's page in my signature first, reallyrarewares for the obscure stuff. let me know if you're having trouble finding anything and you don't have to install anything you don't care about. i'll get a link list together later.

Five

2020-04-23, 01:37 PM

also I recommend the official shorten 3.6.1 binaries, the 2014 john33 compile on rarewares works faster but cannot encode so its too much of a broken thing for me to trust :lol:

Five

2020-04-23, 01:51 PM

...

gwendibbley

2020-04-28, 12:50 AM

Hey!

Do you want me to write an actual vb/cmd-line script or should I write in actual English how I think the process would be?

/H

Five

2020-04-28, 01:09 AM

yes, in fact I was dreaming about somebody who could do that :D

I'm interested in both, if you could pls. thx

gwendibbley

2020-04-28, 01:50 AM

I can start with actual english and we can discuss if the process flow is what you (we?) want before focusing on the actual script.

/H

Five

2020-04-28, 11:01 AM

yes, please! I need something to do :D

gwendibbley

2020-04-28, 03:10 PM

I’m struggling with the vbscript so I will now focus on the plain English version.

Stay safe!

H

Five

2020-04-28, 05:37 PM

thanks H take your time, get it right

gwendibbley

2020-04-29, 07:58 AM

This vbscript searches through all the folder in the pre-defined root folder for files ending with st5, ffp, md5, sfv and txt (checksums could be inside a file named fingerprints.txt, how to deal with those?).

You could try this for yourself and then look at the output file for the result.

Option Explicit

'Declaration
Dim fso, arr1, File, SubFolder, oItem, strWriteLine, strText, strOutputFile

'System variables
Const ForReading = 1

'User variables
Const rootFolder = "C:\Temp\Test"
Const OutputFile = "C:\Temp\FindChecksumFileoutput.txt"
Const debugging = 1
Const Logging = 1

Set fso = CreateObject("Scripting.FileSystemObject")
Set strOutputFile = fso.CreateTextFile(OutputFile)
arr1 = Array(".st5", ".ffp", ".md5", ".sfv", ".txt")

FindChecksumFile fso.GetFolder(rootFolder)

strOutputFile.WriteLine strWriteLine
strOutputFile.Close

Set fso = Nothing

Sub FindChecksumFile(Folder)
For Each File In Folder.Files
For Each oItem In arr1
If InStr(LCase(File.Name), oItem) > 0 Then
If oItem = ".st5" Then
Checkst5(File)
ElseIf oItem = ".ffp" Then
Checkffp(File)
ElseIf oItem = ".md5" Then
Checkmd5(File)
ElseIf oItem = ".sfv" Then
Checksfv(File)
ElseIf oItem = ".txt" Then
Checktxt(File)
End If
End If
Next
Next

For Each SubFolder In Folder.SubFolders
FindChecksumFile SubFolder
Next
End Sub

Sub Checkst5(inputfile)
'WScript.Echo inputfile.Path
strText = inputfile.Path
strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checkffp(inputfile)
'WScript.Echo inputfile.Path
strText = inputfile.Path
strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checkmd5(inputfile)
'WScript.Echo inputfile.Path
strText = inputfile.Path
strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checksfv(inputfile)
'WScript.Echo inputfile.Path
strText = inputfile.Path
strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checktxt(inputfile)
'WScript.Echo inputfile.Path
strText = inputfile.Path
strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Let me know what you think..

/G

gwendibbley

2020-04-29, 09:41 AM

I’ve gotten a bit further but am stuck now on if there are spaces in the flac filename..

Option Explicit

'Declaration
Dim fso, arr1, File, SubFolder, oItem, strWriteLine, strText, strOutputFile, wso, exe, sout, cmd, strFromProc, strErrorCode, oShell, strReturn

'System variables
Const ForReading = 1

'User variables
Const rootFolder = "C:\Temp\Test"
Const OutputFile = "C:\Temp\FindChecksumFileoutput.txt"
Const Debugging = 1
Const Logging = 1

Set fso = CreateObject("Scripting.FileSystemObject")
Set strOutputFile = fso.CreateTextFile(OutputFile)
arr1 = Array(".st5", ".ffp", ".md5", ".sfv", ".txt", ".flac", ".ape", ".shn")

FindChecksumFile fso.GetFolder(rootFolder)

strOutputFile.WriteLine strWriteLine
strOutputFile.Close

Set fso = Nothing

Sub FindChecksumFile(Folder)
For Each File In Folder.Files
For Each oItem In arr1
If InStr(LCase(File.Name), oItem) > 0 Then
If oItem = ".st5" Then
Checkst5(File)
ElseIf oItem = ".ffp" Then
Checkffp(File)
ElseIf oItem = ".md5" Then
Checkmd5(File)
ElseIf oItem = ".sfv" Then
Checksfv(File)
ElseIf oItem = ".txt" Then
Checktxt(File)
ElseIf oItem = ".flac" Then
VerifyFlac(File)
ElseIf oItem = ".ape" Then
VerifyApe(File)
ElseIf oItem = ".shn" Then
VerifyShn(File)
End If
End If
Next
Next

For Each SubFolder In Folder.SubFolders
FindChecksumFile SubFolder
Next
End Sub

Sub VerifyFlac(inputfile)
cmd = "C:\Temp\CmdlineApps\flac.exe -t " & inputfile.Path

Set oShell = WScript.CreateObject ("WScript.Shell")
strReturn = oShell.Run(cmd, 0, True)
Set oShell = Nothing

If CInt(strReturn) <> 0 Then
strText = inputfile.Path & ";ERROR"
Else
strText = inputfile.Path & ";OK"
End If

strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checkst5(inputfile)
'WScript.Echo inputfile.Path
'strText = inputfile.Path
'strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checkffp(inputfile)
'WScript.Echo inputfile.Path
'strText = inputfile.Path
'strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checkmd5(inputfile)
'WScript.Echo inputfile.Path
'strText = inputfile.Path
'strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checksfv(inputfile)
'WScript.Echo inputfile.Path
'strText = inputfile.Path
'strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub Checktxt(inputfile)
'WScript.Echo inputfile.Path
'strText = inputfile.Path
'strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub VerifyApe(inputfile)
'WScript.Echo inputfile.Path
'strText = inputfile.Path
'strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Sub VerifyShn(inputfile)
'WScript.Echo inputfile.Path
'strText = inputfile.Path
'strWriteLine = strWriteLine & strText & vbCrLf
End Sub

Five

2020-04-29, 12:17 PM

thanks so much :clap: :clap: :clap:

I'll come back and start looking at it soon.. this helps me to relax, thx :D

gwendibbley

2020-04-29, 12:30 PM

I was thinking that maybe we should write an entire new program, written in C#. With a GUI and everything..

/H

PS. Maybe we should continue this in a private conversation, like email..

gwendibbley

2020-04-29, 02:20 PM

Five

2020-04-29, 09:17 PM

thanks H!

here is the page
http://shnutils.freeshell.org/shntool/

Five

2020-04-30, 05:51 PM

it is shn flac ape and some more on jj's page look there at the link thanks.

I'm not quite sure how to use a vbscript, is that visual basic?

gwendibbley

2020-05-01, 07:10 AM

1. Open a command prompt.
2. cd to the location where you have placed the vbs-file.
3. Write “cscript.exe [name of the vbs-file]” without the quotes.

You must edit the file to have it point to the right location, or paste a folder in your temp-folder with flac and text-files.

H

Five

2020-05-01, 07:08 PM

okay, I'm working on it!!

gwendibbley

2020-05-05, 12:35 AM

Five,

you need to clear your Inbox..

H

Five

2020-05-05, 07:15 AM

thx h I'll be on it soon..

gwendibbley

2020-05-11, 04:20 AM

thx h I'll be on it soon..

How’s that going for ya?

/H

mike1061

2020-05-17, 03:30 PM

If I can be of help, let me know. I'd love helping. That said, I have no idea how to code anything.

Five

2020-06-05, 01:13 AM

How’s that going for ya?

/H

I haven't had any time for this recently,

I have some free time coming up, I study your code a bit .. :wave:

thanks again H, I will not drop the ball, it just takes time for a beginner coder like me to dive in, I get lost easily.

in coming months we will hopefully have a beta and it would be great just to get somebody to make sure everything works.

thanks,

Mike

:cool:

gwendibbley

2020-06-07, 09:30 AM

I haven't had any time for this recently,

I have some free time coming up, I study your code a bit .. :wave:

thanks again H, I will not drop the ball, it just takes time for a beginner coder like me to dive in, I get lost easily.

in coming months we will hopefully have a beta and it would be great just to get somebody to make sure everything works.

thanks,

Mike

:cool:

I was more thinking about you clearing your inbox..

Can I send you a private message yet?

H

Five

2020-06-07, 12:15 PM

lol okay I should do that as well :redface:

I like posting this stuff out here since only two people read anyways and there might be more contributors come along..

gwendibbley

2020-06-08, 11:14 AM

lol okay I should do that as well :redface:

I like posting this stuff out here since only two people read anyways and there might be more contributors come along..

I was thinking about the response to your mail..I can’t respond to you privately because your inbox is full..

H

Five

2020-06-28, 07:27 AM

sorry bro its open now i look forward to hearing from you :D

gwendibbley

2021-03-15, 11:20 AM

How's it going?

/H

Paladin

2021-07-18, 11:05 PM

to create MD5 with foobar2000

https://foobar.hyv.fi/?view=foo_audiomd5
with ffmpeg.exe
https://www.gyan.dev/ffmpeg/builds/packages/