View Full Version : MD5 checksums question

2006-03-14, 03:57 AM
I downloaded a show and none of the flac files downloaded passed the md5 checksum comparison. The show plays fine in Windows Media Player. But I am concerned that, with a lacking integrity, what I have downloaded I have inadvertently passed on to others in uploads. I guess this is a drawback of P2P bitstreaming. I can't tell that what I am receiving has integrity in the first place. I am trying to download parts of the show again to see if I can get a clean copy. Is having to download more that once common?

2006-03-14, 07:14 AM
Read this from the FAQ:

Note: Wholefile md5 checksums are the type generated by programs such as mdsum, mkwact, and others. Please read on to understand why these are forbidden as the primary form of file verificaton at The Traders' Den.
The most rudimentary form of md5 fingerprinting is to run the algorithm on an entire file. By comparing two md5 checksum values, one will be able to confirm that the files that generated the values are the same. In the early days of lossless file trading on the internet, this was used to track recordings encoded to the Shorten (.shn) file format. This very effective method of ensuring that all files remain identical from user to user is still popular today. However, there are some fundamental weaknesses to this method, many that have arisen with the advent of .flac and .ape lossless audio formats as the preferred trading format of many users. Both of these newer formats permit the use of file tags so that a user can add artist, date, venue, source, or any other information they want to the file's header for display in their preferred media player. While adding or removing these tags doesn't change the audio at all, it does alter the file to include the additional header data, and therefore changes the file's checksum. Two files would thus appear different in a checksum comparison while being acoustically identical.

Furthermore, there is certain additional data that can be stored in .wav files that .shn and .ape allow to remain during compression, but which .flac sees as unecessary and potentially harmful to your system and thus removes it. Again, it would be possible to convert from .shn to .flac and back and have two acoustically identical files result in different wholefile md5 checksum values.

Yet another weakness to relying on wholefile md5 checksums is the need to use seek tables in .shn files. The earliest version of Shorten's codec did not allow for the files to be seekable during playback. As a "band-aid" for this oversight, they allowed for the addition of a seek table that would be appended to the file to provide the necessary information to make the file seekable. The problem, again, is that by adding this information a user changed the file but not the audio. This problem has been further compounded by the recent addition of alternate versions of seek tables, so that even files with seek tables can contain one of a few versions and thus have differing md5 checksum results while being musically identical.

Wholefile md5 is an effective method of comparing files, but has been made somewhat obsolete by the more versatile methods of audio file verification I'll discuss next.

So there are a number of reasons why your MD5 won't check out. The files could have been renamed (happened to me numerous times), the seeder could have added tags to the FLAC files, the MD5 could be for SHN or APE files and now you have FLAC, etc. etc.

Sometimes seeders will alter the files before seeding the show (tags, file names, SBE corrections, etc.) and not create a new MD5 or FFP before seeding. Ask around in the thread where you downloaded the show and see if anyone else is having problems, contact the seeder, etc.

2006-03-14, 07:36 AM
any checksums included with a torrent should work

since this is FLAC the files can be self-tested, if they pass that test then they are fine and it is the checksums included which are incorrect.

if you have never run "test" mode on FLAC files there are some instructions in the link "checksums demystified" in my signature

you should also ask in the thread for this torrent as TheMamba suggests

2006-03-14, 06:46 PM
TheMamba, Five, thanks for the guidance and suggestions.