Updated for version 1.72.

Included in this zip file are:
1) MP3Utility.exe
2) MP3Utility.ini
3) Instructions For Adding MP3Utility To WE Context Menu.txt
4) Readme.txt (this file)

Readme.txt - User Notes for MP3Utility

To examine this file, maximize Notepad and turn the Word Wrap on (under the Edit menu selection).

I highly recommend skimming through this entire Readme file before running MP3Utility as it contains a lot of information on how to best use the program for your specific needs.  It's also a pretty quick read.


-------------------------------------------------------------------------------------------------


Table of Contents
Part I    - Summary
Part II   - Installation/QuickStart
Part III  - Philosophy and Program Functionality/Description of Options
Part IV   - Testing Directories of MPEG Audio Files
Part V    - Examining/Testing Individual MPEG Audio Files
Part VI   - Ideas on Fixing Damaged Files
Part VII  - Cataloging Directories of MPEG Audio Files


-------------------------------------------------------------------------------------------------


Part I - Summary

I have always wanted a utility that would help me (i) test the integrity of and (ii) catalog my collection of mp3 files.  Not being able to find a suitable utility available, I decided to try my own hand at creating one.  MP3Utility is the result of that effort, and it can do the following:
1) Display detailed information on just about any MPEG audio file (not just MPEG Version 1.0, Layer III files).
2) Test an MPEG audio file for synchronization (and other) errors.
3) Identify and process both Xing VBR and MusicMatch VBR files.
4) Intelligently parse and display those pesky leading IDV3v2 tags.
5) Batch test a directory of files, with the ability to move corrupted files to a pre-specified directory for further testing.  Can also recursively test files in subdirectories under the top directory specified.
6) Generate a detailed audio catalog formatted for direct import into Microsoft Excel.  This includes all the MPEG and Tag information displayed on the main screen (you can then use Excel for further formatting/sorting of the data).

Other MP3Utility features:
1) Written in Visual C++, which makes it small and fast (I can test a 4.0 MB file in about one second on my 450 MHz Pentium II).
2) The main screen is fully resizable so that you can make the Output Log as big as you want (one of the biggest complaints I had early on was that the Output Log was too small).
3) The utility provides detailed error messages that identify the frame, byte position and time position of synchronization errors.
4) All test output can be saved in a text file for later use.
5) Clean, readable output (It drives me nuts when people display large numbers without commas!)
6) Clean install/uninstall.  The utility will unzip into it's own "mp3utility" directory and won't make any changes to the registry or any other system files.  Also, MPUtility will not attempt to place icons/shortcuts on your Desktop, Start Menu, Quickstart Toolbar, System Tray, etc.  To uninstall, simply delete the MP3Utility directory.
7) Uses standard Windows file access functions, so you can test files/directories located on remote computers (note, however, that LAN access may slow down MP3Utility as all files tested will need to be read across the LAN).
8) Supports drag-and-drop and right-click context menu selection from Windows Explorer.

Caveats:
1) The testing components of this utility provide a lot of technical information and are not designed for use by novices.  In order to understand the error messages from the testing functions, you will need to be familiar with the basics of MPEG audio file structure (for an in-depth primer on MPEG audio frame headers and tags, a must-see is Predrag Supurovic's "MPEG Audio Compression Basics" at:
http://www.dv.co.yu/mpgscript/mpeghdr.htm).
2) Keep in mind that I wrote this utility with the idea of being able to examine and test the file structure of MPEG audio files, specifically looking for synchronization errors.  All of the testing done by this utility is done using information from the frame headers.  I do NO examination of the actual audio data itself (other than to make sure it's there).  Thus, while this utility can tell you if the file structure is damaged, it cannot tell you anything about the quality of the encoded audio stream.
3) While I included the logic for recognizing and testing VBR files, I have not written any logic for determining a specific time index within this type of file.  As a result, the indicated time of a sync error will be off for VBR files.
4) Yes, the icon was shamelessly stolen from Winamp.  If someone out there has any talent designing icons, please send me one before I get sued by NullSoft (which is now part of AOL - yes?)

Finally, if you use this utility, please let me know your thoughts/comments (email address at bottom).  I've spent a lot of time putting this together; it would be most gratifying to know if people actually use it.  Thanks.

Peter F. Smith


-------------------------------------------------------------------------------------------------


Part II - Installation/QuickStart

Installation - basically none!  Unzipping the installation file will create a subdirectory named MP3Utility and all files needed to run MP3Utility will be copied into that directory.  To uninstall, just delete the directory.

Instructions on how to add MP3Utility to the right-click context menu in Windows Explorer can be found in the file entitled: "Instructions For Adding MP3Utility To WE Context Menu.txt"

Quickstart - most people will want to use this utility to recursively scan an entire directory structure of mp3 files and move any corrupted files found to a separate directory for further testing.  Note that a file flagged as corrupted may sound perfectly fine depending upon (1) the seriousness of the errors, (2) the ability of your mp3 player to perform error-correction, and (3) your own sensitivity to clicks and pops.  Ultimately you will need to listen to the area of the file that's corrupted and make your own determination.  My own philosophy is to get rid of any files that are corrupted or poorly encoded.  However, other people may feel differently.

NOTE: Be careful with the "Move Files" option as if you have a lot of corrupted files, MP3Utility will move them all to the specified directory.  To be safe, the first time you run MP3Utility on a large directory structure, run it with the ""Recursively test subdirectories" option on and the "Move Files" option off and see how many files are flagged as corrupted.  Otherwise you could end up with all your files moved from a carefully constructed sub-directory structure into one huge directory.

To recursively scan a directory and move corrupted files to a specified directory:
1) Click "Test Directory" radio button
2) Click the "Load" button and select the top-level directory for testing mp3 files
3) Click the "Options" button.
   - click the "Recursively test/catalog subdirectories" checkbox
   - click the "Move files with errors/warnings to the following directory:" checkbox
   - click the "Browse" button and select the directory to move files with errors or warnings
   - click the "OK" button to close the Options dialog box
4) Click the "Start" button

MP3Utility will report any corrupted files it finds, and it will move those files to the directory you specified.

After segregating files with error/warnings into one directory, you can test them individually with MP3Utility using the "Test File" function to locate the time indices of all corrupted segments.  You can then use your favorite mp3 player to listen to the specified segments and determine the seriousness of the problem.  (Note that "Test Directory" stops analyzing a file once it encounters the first error, thus you need to use the "Test File" function to locate all the errors in a specific file).


-------------------------------------------------------------------------------------------------


Part III - Philosophy and Program Functionality/Description of Options

Philosophy

A major problem with testing mp3 files is that determining the seriousness of a problem is highly subjective, i.e., a slight click that might be barely noticeable to one person might be particularly annoying to another.  Also, mp3 players handle errors differently.  For example, WinAmp has some pretty sophisticated error-handling features that minimize distortion when playing a corrupted file, whereas the CreativeLabs Nomad (a portable mp3 player) has been known to lock up when encountering any type of synchronization error.

Another issue is that some users want "perfect" files for use in archiving, whereas others just want a file that sounds "good enough".  I fall into the former camp, and my philosophy in writing MP3Utility has been to flag any type of corruption I can find.  I do this because experience has shown that even minor corruption in a frame header can point to serious problems in the audio data.  As a result of this additional error checking, MP3Utility will generally flag more instances of corruption than other utilities.

To assist the user in determining the seriousness of corruption, MP3Utility provides both a detailed error description as well as the time index of the problem.  This allows the user to go back and listen to the corrupted sections and then decide whether to keep the file or not.

Program Functionality/Description of Options

When selecting a function that works on an individual file, the MPEG and Tag Information groups will appear on the Main window and the Output Log will shrink in size accordingly.  When selecting a function that works on a directory, the MPEG and Tag Information groups will disappear to make more space for the Output Log.  The main window is fully resizable and when resizing, the Output Log will expand/contract accordingly.  Also, when testing individual files, the "Hide MPEG Info" checkbox will appear on the Main Window.  Clicking this checkbox will hide the MPEG and Tag Info groups on the main window, expanding the size of the Output Log.

From a functional standpoint, it seemed to make sense to have the default directories for file testing and directory testing (and cataloging as well) "follow" each other.  Thus, if you use the "Test Directory" function and browse to a new directory and then later load a file with "Test File", you will start in the directory selected in "Test Directory".  My sense is that this is the functionality most people will want.

Listed here are explanations for each of the options available on the Options screen.

1) "Recursively test/catalog subdirectories" checkbox.  When checked, MP3Utility will test/catalog all the files in the selected directory, as well as recursively process all subdirectories.  Note that files are processed in alphabetical order, by directory.

2) "Auto-start testing on external selection".  When selected, MP3Utility will automatically start testing any files loaded via drag-and-drop or selected from the right-click context menu in Windows Explorer.  This does not apply to files loaded via "internal selection" (i.e., files selected by clicking on the Load button on the main window).

3) "Clear log before each run".  When checked, the Output Log will be cleared every time the Start button is clicked.  Generally you will want this on, however some people have asked for the ability to accumulate and save a log of multiple tasks.

4) "Ignore warnings/errors in last 1% of file".  The real problem here is that a lot of people are tacking on all kinds of junk to the end of mp3 files and I haven't figured out a way to intelligently parse through all the tags and combinations of tags (not to mention corrupted tags) that I have come across.  In addition, there seems to be a popular encoder out there that produces mp3 files with a truncated last audio frame (is this in the spec somewhere?).  I also suspect that some of the multitude of mp3 tagging utilities available are causing corruption as well.  Most importantly, I have found that in almost every case, errors detected in the last 1% of the file can be safely ignored as they in no way impact audio playback.  As such, I finally gave up trying to figure this out and simply put in the option to ignore errors in the last 1% of the file.  However, I do suggest (at a minimum) that you always listen to first few seconds and last few seconds of any mp3 file (even after testing shows it as ok).  This is because there is no way for MP3Utility to tell if the beginning or end of a file has been lost due to a partial download or other error.

In any case, I have given users the ability to turn off the "Ignore warnings/errors in last 1% of file" if they don't like it (but expect a lot of additional files to show up with warnings/errors).  BE ESPECIALLY CAREFUL IF YOU TURN THIS OPTION OFF AND YOU ARE USING THE "MOVE FILES" FEATURE OF MP3UTILITY AS IT IS LIKELY MP3UTILITY WILL FLAG 30-40% OF YOUR FILES AS CORRUPTED AND THEN MOVE THEM TO THE "CORRUPTED FILE" DIRECTORY.

5) "Move files with errors/warnings to the following directory:" checkbox.  Allows you to automatically move all corrupted files found to a specific directory for further testing.  The Browse button allows you to select the directory.  Note: you should choose a directory on the same disk drive as the directory to be tested, as moving a file to a different location on the same disk only requires moving a directory entry, whereas moving a file to a directory on a different disk requires a wholesale copy followed by a delete (much slower).  Also, the directory to hold corrupted files should not be within the set of directories you are testing (MP3Utility will still work if you do this, but any corrupted files in the corrupted file directory will be reported as errors and will be reported as moved, even though they will remain in the same place).

6) "Number of bytes to search for initial sync".  The number of bytes from the beginning of the file (or if there is a leading ID3v2 tag, after the tag) to search for the first frame header.  Generally there shouldn't be anything else at the beginning of the file, but experience has shown it prudent to allow for this (some files have "wav" file headers, etc.).  It is doubtful you will ever need to change this value.

7) "Number of bytes to attempt resync".  The number of bytes after sync is lost to search for a matching frame header.  Again, it is doubtful you will ever need to change this value.

8) "Catalog Field Delimiter".  The delimiter used to separate fields when using the cataloging function.  The easiest to use for import into Excel is the tab character, but I have left the ability to use another character in case someone needs it.  Be careful not to use a character that appears in the any of the tags of the mp3s you are cataloging, as this will cause a tag field to be artificially divided in two.

9) "ID3v2 Tag - Maximum field length:".  I put this limit in for a bunch of different reasons.  First, I've seen some rather lengthy comment fields in ID3v2 tags.  As such, I felt people may only want to see, say, the first 50 characters or so when they use the cataloging feature.  In addition, I'm pretty sure the length of a cell entry in Excel is limited to a specific number of characters (the exact length depends on the version of Excel you're using).  Thus, I assumed it would be a bad idea to try to import a file with fields longer than the maximum cell length allowed by Excel (although I didn't try this to see what would happen).  Finally, I included the maximum field length as a safety feature.  Since the length of the fields in an ID3v2 tag can (in theory) be huge (up to hundreds of megabytes), I wanted to make sure if MP3Utility encountered a corrupted file it wouldn't try to read in and display a multi-megabyte tag field.


MP3Utility will save its settings between program runs.  The settings saved include the following:
1) Main window size, position and state.
2) The selected function.
3) The last directory tested.
4) The directory where the output log was last saved.
5) The state of the "Hide MPEG Info" checkbox.
6) All the entries in the Options Dialog.

Like the author of AudioGrabber, I decided not to store any program settings in the registry but rather save this data in an old-style "ini" file.  This has the advantage that to uninstall MP3Utility, you can just delete the MP3Utility directory and don't have to worry about removing registry entries.  By the way, AudioGrabber is a CD-Ripper for Windows that I find indispensable and highly recommend.  You can check it out at:
http://www.audiograbber.com-us.net


-------------------------------------------------------------------------------------------------


Part IV - Testing Directories of MPEG Audio Files

This basically performs the same level of testing as for individual files, however the program will stop testing a file after encountering the first sync error and then move on to the next file.  Also, error reporting is less detailed.  To test a directory, select "Test Directory" in the "Select Function" group box and then click on "Load" to choose a directory.  The directory chosen will be displayed to the right of the load button.

Click on the "Options" button and select the options you want (see Section III of this document for a description of available options).

To begin testing, press the "Start" button.  Output will be recorded to the Output Log.  The "Start" button becomes the "Cancel" button during processing, and you can click it if you wish to cancel testing.


-------------------------------------------------------------------------------------------------


Part V - Examining/Testing Individual MPEG Audio Files

Generally, I expect most people will use the "Test Directory" facility to examine a directory of MPEG files and then use the Individual File Test to more thoroughly examine those that are indicated as damaged.

To examine an individual file, simply select "Test File" in the "Select Function" group box and then click on "Load" to choose a file.  Detailed MPEG statistics and tag information will then be display on-screen.  To begin testing, press the "Start" button.  Output will be recorded in the Output Log.  The "Start" button becomes the "Cancel" button during processing, and you can click it if you wish to cancel testing.

At the end of processing, MP3Utility will display a summary consisting of the number of frames processed, the number of padded and unpadded frames, and whether the bitrate was constant or variable.

Probably the biggest problem users will have with this utility will be deciphering the error messages, and this complexity is mostly due to the wide variety of junk that people are inserting before and after the audio stream (especially companies like MusicMatch that tack on multi-kilobyte tags to the end of audio files).  As a result, MP3Utility may generate warning messages that may not indicate problems.  I will outline the messages and their meaning below.

1) Basic Errors

Error: Unable to open file.
 - either file is read protected by another application or it was moved after being initially loaded into MP3Utility.

Error: File too short, or
Error: End of file encountered in first audio frame.
 - file is less than the size one audio frame.

Error: Can't locate first frame header.
 - MP3Utility couldn't find a valid frame header within the number of bytes specified in the Options Dialog.

2) Synchronization Errors

Warning: Last audio frame truncated.
 - probably the most common warning/error and can ignored in almost all cases.  Simply indicates that the last audio frame is shorter than indicated in the frame header (it seems a lot of encoders do this)

Warning: Last audio frame too long.
 - can probably be safely ignored.  Usually due to invalid or extended tag data.

Error: Sync error reading frame header xxx.
 - this is the one to look out for and means there is a potentially serious error in your audio file.  The one instance where this is probably not a problem is when it occurs right at the end of the file, and in this case it is generally due to invalid or extended tag data (e.g., multiple tags tacked onto the end of the file or the aforementioned and dreaded MusicMatch "Brava Software Inc." tag).  As it would be nearly impossible to check for all potential problems, I make the simple assumption that if a sync error occurs within the last 1% of a file, it is due to tag data problems and, as a result, the sync error is only recorded as a warning.

After encountering a sync error, MP3Utility will attempt to resynchronize with the next valid frame header.  If successful, the program will display the location of the frame header found and calculate the appropriate error information on the short/long frame preceding it.  Resynchronization will fail if the program either reaches the end of the file or doesn't find a valid header within the specified number of bytes in the Options Dialog.

For more information on one common sync error and how it can be fixed, see: "Part IV  - Ideas on Fixing Damaged Files".


-------------------------------------------------------------------------------------------------


Part VI - Ideas on Fixing Damaged Files

When MP3Utility encounters a sync error, the first thing you should do is determine if it is significant.  Just crank up your favorite player (I use Winamp) and move to just before the indicated time of the error.  Note that Winamp reads about two seconds ahead of what's currently playing, so you should see the red "sync error" light flash approximately two seconds before you actually hear the error.  In some cases, a reported sync error may be inaudible or so small that you may not care.  If this is not the case, you have two options:
1) Find a better copy of what you're looking for.
2) Attempt to fix the problem yourself.

Generally, I strongly recommend option #1 as editing an mp3 file using a binary editor is usually not worth the time and effort.  However, I provide some ideas below based on my own experience for those who choose option #2.


I have discovered that many sync errors are caused by the second byte of a frame header being invalid.  This is probably the case if you get an error message that indicates a frame is twice as long as it should be.  For example, examine the following error:

Error: Sync error reading frame header 768 expected at byte 320,575.  Approx. time: 0:20 (8% through).
Resync successful - Frame header 768 found at byte 320,993.
Frame 767 (bytes 320,157 - 320,992) long by 418 bytes (expected 418 bytes, found 836 bytes).

Frame 767 is indicated to be twice as long as it should be.  Thus, it is likely that the frame header expected at byte 320,575 is corrupted and is not being recognized as a valid frame header.  In many cases, this is caused by the second byte of the frame header being set incorrectly (the second byte indicates the MPEG Version, the Layer and whether the file contains CRCs - thus it should be the same throughout the file).  The easiest place to get the correct value is from the first frame header (any valid frame header will do, but MP3Utility displays the offset to the first frame header).  Simply reset the second byte of the corrupted frame header, retest it with MP3Utility (to make sure you did it correctly) and then listen to the audio to see if it cleared up the problem.

How do I edit an MPEG audio file?
In my mind, the best (and only) way to tinker with an MPEG audio file is with a binary editor (I use and recommend Hex WorkShop by BreakPoint Software).  A word of advice for anytime you edit a file with a binary editor: Always keep a backup copy of the original until you are sure the edited copy is good!  One other note: I designed MP3Utility to automatically reload the audio file each time you click on "Start".  This is so you can leave the file loaded in both an editor and MP3Utility and, after saving changes in the editor, you can switch to MP3Utility and hit Start to test it again (without having to re-load the file manually).

Another common sync error is what I refer to as the "two byte long/two byte short" problem.  This happens when a frame is two bytes too long and then a subsequent frame (usually two or three frames hence) is two bytes too short.  Unfortunately I don't have any easy fix for this one, but present it in case someone else has any ideas.  The error message for this problem will look like the following:

Error: Sync error reading frame header 3,998 expected at byte 2,086,435.  Approx. time: 1:44 (40% through).
Resync successful - Frame header 3,998 found at byte 2,086,437.
Frame 3,997 (bytes 2,085,913 - 2,086,436) long by 2 bytes (expected 522 bytes, found 524 bytes).

Error: Sync error reading frame header 4,000 expected at byte 2,087,481.  Approx. time: 1:44 (40% through).
Resync successful - Frame header 4,000 found at byte 2,087,479.
Frame 3,999 (bytes 2,086,959 - 2,087,478) short by 2 bytes (expected 522 bytes, found 520 bytes).

Obviously, this section could benefit from the experiences/advice of others.  Please email me if you have any helpful additions.


-------------------------------------------------------------------------------------------------


Part VII - Cataloging Directories of MPEG Audio Files

Ever wanted to catalog your files by bitrate or by time length?  Ever wondered how many of your files are variable bitrate versus constant bitrate?  Ever wonder how many are Stereo versus Joint Stereo?  Well, my cataloging feature is the answer.

When developing this functionality, I wanted to make sure users could easily take the data and further customize it for their needs.  As such, I chose to format the output specifically for easy import into Microsoft Excel under the hopefully valid assumption that most people would have easy access to this application.  Users can then use the formatting and sorting features of Excel to organize/display the information as they wish.  I also decided to include basically everything that anyone could ever want, as it is an easy task to delete any columns not needed.  Cataloging will provide the following information for each file:
 - Directory name
 - File name
 - File size
 - Last modified date/time
 - Offset to first frame header (in bytes)
 - Est. number of frames
 - Est. time in M:SS format
 - Est. time in seconds
 - MPEG version
 - Layer
 - Bitrate
 - Sample rate
 - Channel mode

 - ID3v1 tag found (yes/no)
 - Tag version
 - Title
 - Artist
 - Album
 - Track #
 - Year
 - Genre
 - Comment

 - Leading ID3v2 tag found (yes/no)
 - Tag version
 - Tag length (in bytes)
 - Title
 - Artist
 - Album
 - Track #
 - Year
 - Genre
 - Comment
 - All ID3v2 tags (see description below)

 - Unsynchronisation flag (true/false)
 - Extended header flag (true/false)
 - Experimental indicator (true/false)
 - Footer present (true/false)


All ID3v2 tags: This is a list of all ID3v2 tag headers found.  Following each tag header in parentheses is the tag length in bytes (this is the full tag length, including the tag header).  Tag headers are separated by semicolons.  For a list of the standard tag headers, see section 4 of the document located at:
http://www.id3.org/id3v2.4.0-frames.txt


You can also select the field delimiter character to be used in generating the catalog - see the Options Dialog (MP3Utility uses the tab character by default, which is what Excel uses by default for importing).

To catalog a directory, select "Catalog Directory" in the "Select Function" group box and then click on "Load" to choose the directory.  The directory path will be displayed to the right of the Load button.  To begin cataloging, press the "Start" button.  Output will be recorded in the "Output Log" box (remember, the output is formatted for importing into Excel, it's not very readable in its raw state).  Click the "Cancel" button if you want to cancel cataloging.

When the cataloging is complete, press the "Save" button and save the catalog as a text (.txt) file.

Importing into Excel:
1) Start Excel
2) Click File/Open and select the .txt file you created above (you may need to change the "Files of type:" box to "Text Files" or "All Files" to display text files)
3) The Import Wizard should start automatically.  If you used the tab character as the field delimiter, you can just click "Finish".

Notes on cataloging:
1) The "Length in M:SS" field will be imported into Excel as a time format, allowing you to use Excel addition to sum song lengths (thanks to Koen for this suggestion).
2) Any non-printable tag field characters such as tabs, carriage returns, linefeeds, etc. are displayed (and recorded for cataloging purposes) as spaces.
3) The "Last Modified Date/Time" is imported as a date/time format within Excel, so you can use Excel to format it any way you want (i.e., just the date, time in AM/PM format, etc.)
4) All the numeric data are imported as numbers (not text); thus you can use Excel for formatting or to perform additional calculations.


Enjoy! (And please let me know what you think)

Peter Smith
web: http://www.geocities.com/mp3utility
email: mp3Utility@hotmail.com
