# Description:
#   
#   This script takes two files,( a phrase list which contains 'bar' delimited
#   phrases and their 'equivalents' or translations, and a file which contains
#   a list of URL's of sound files and the textual equivalents of the spoken words which those
#   files contains). These files are then combined in a format that can be used
#   by a Javascript program such as 
#     http://www.geocities.com/matth3wbishop/projects/gencat.html
#   This Javascript based web page is designed to provide a kind of 'tutoring engine'
#   which allows the user to practice his or her language skills in a foreign language.
#
# Parameters"
#   referenceFileName  
#     the name of the file which contains the urls of sound files of spoken words and
#     phrases and the textual equivalents of those
#     words. The file should be in the format "[url]textual equivalent"
#     where [url] is the URL of a sound file beginning with http:// and enclosed in
#     square brackets
#   translationFileName
#     the name of the file which contains translation 'pairs' of words and phrases
#
# Example:
#   /var/www/utils/merge-gencat-refs-and-trans.sh my-ref-file.txt my-tran-file.txt
#     This will write a merged file to standard out.
#   
# Notes:
#   This script currently takes several minutes for 3500 line files.
#
#   The files that contain lists of phrases are currently and confusingly named
#   'wordlist-something.bsv'. The file suffix .bsv stands for 'bar separated values'
#   
#   The sound files being dealt with here are from the Generalitat de Catalunya's
#   web-site oriented mainly to teaching foreign university students how to speak
#   Catalan. Currently the URL references for the sound files of spoken English words
#   and the same for the catalan words are all in the same file (intercat-ref-en-ca.txt)
#
#   So in order to extract either the 'english word' sound-files or the 'catalan word'
#   sound-files, we need to look at the characteristics of the names of the sound-files
#   Fortunately there appears to be a consistent naming convention used.
#   Catalan sound-files seem to begin with 'c' or 'slc' whereas English sound-files appear
#   to begin with just a plain number [0-9] or with an 'a' character. So an English sound-file
#   should look something like this:
#     http://www.intercat.gencat.es/guia/audio2/02019.rm
#     http://www.intercat.gencat.es/guia/audio1/a011.ra
#   And the Catalan ones should look something like this.
#     http://www.intercat.gencat.es/guia/audio1/slc04.ra
#     http://www.intercat.gencat.es/guia/audio2/c02021.rm
#
#   However, the javascript 'enhanced' web-page for which we are generating an array
#   uses a system to reduce the amount of data that has to be downloaded to the users
#   browser the first time that the person accesses the web-page. This system involves
#   'tacking on' the base url of the sound files AFTER the page has loaded (considering that
#   all the sound-files, in this case, have the same base URL. The base URL used in this
#   case is http://www.intercat.gencat.es/guia/audio.
#
#   The 'grep' line which appears to work to extract the catalan URLs from the URL references
#   files is as follows
#      grep -E 'guia/audio[0-9]+/s?l?c' 
#   And the English grep line should be
#      grep -E 'guia/audio[0-9]+/[a0-9]' 
#
#   This script is relatively speaking quite slow because it is performing a 'look-up'
#   type of function on a reasonably large file (3500 lines).
#
#   The resulting javascript array should be in the format 
#     sound-file-partial-url||textual-equivalent-of-sound-file||translation-of-textual-equivalent
#   So an example line from the array might look like
#     3/a014.ra||Fins ara||See you soon||
#   You can see that the first part of the URL has been removed (and will be added later by the
#   javascript in the web-page. The value delimiter string '||' is used only because it is 
#   hopefully very rare in any text file containing phrases and their translations.
#
#  Unfortunately the Data which is contained in the files 'wordlist-xx-xx.bsv' is not 
#  completely 'clean'. This is owing to the data-extraction process which I did
#  and which can be done again with the script 'get-intercat-data.sh'
#
#  It might be possible to make this script faster by using the 'look' program
#  rather than 'sed'
#  
# See Also:
#   There are a set of scripts designed to get and transform the 'gencat' data.
#   get-gencat-references.sh, get-gencat-data.sh, reform-...
# Author:
#   m.j.bishop

 
 if [ "$2" = "" ]
 then
   echo "usage: $0 referenceFileName translationFileName "
   cat $0 | sed -n "/^[ ]*#/p" 
   exit 1;
 fi

 sReferenceFileName="$1"
 sTranslationFileName="$2"

   #-- Output the array name definition
   #-- The english grep line
   #--grep -E 'guia/audio[0-9]+/[a0-9]' | \
     # grep -E 'guia/audio[0-9]+/s?l?c' | \ 
   echo "var aaAllPhrasesData = new Array("
   grep -E 'guia/audio[0-9]+/s?l?c' $sReferenceFileName > temp.txt
   for ii in $(cat temp.txt | \
      tr ' ' '#');
   do
     
     word=$(echo $ii | sed "s/^.*\]//g" | tr '#' ' '); 
     ref=$(echo $ii | sed "s/.*\[.*guia\/audio\(.*\)\].*/\1/g");
     echo "word=$word:"
     #echo "CURRENT WORD=$word";
     #-- For a 'catalan tutoring' array use the line below
     sed -n "/$word/ s@^\(.*\)|\(.*\)@\"$ref||\2||\1||\",@gp" $sTranslationFileName | head -1;
     #-- For an English to Catalan tutoring array use
     #sed -n "/$word/ s@^\(.*\)|\(.*\)@\"$ref||\1||\2||\",@gp" $sTranslationFileName | head -1;
   done 
   #-- Remove the final 'comma' 
   #-- but piping all of the above through another sed means that 'bash' cannot display 
   #-- anything until the whole text stream has been finished. 
   #sed "$ s/,[[:space:]]*$//g"
   #-- Close the array brackets
   echo ");"