#***********************************************************************************************************************************
# File       : MyThoughts.txt
# Type       : Plain text
# Created    : 04/06/2004 by Markus Kemp
# Last change: 05/27/2004 by Markus Kemp
#
# Description: A file in which to stow away the results of my own personal mental diarrhea sessions. Any brilliant and not so 
#              brilliant ideas regarding the testsuite program go here. =P
#
#************************************************************************************************************************************


Testcase sample - samba client installation on a UNIX rig
_____________________________________________________________________________

proceeding:

1) Set gcc path. Not sure if this is really necessary since we could just make an absolute call to gcc.
2) Step into samba directory and run ./configure --prefix=<target dir>. Catch potential failure using eval.
3) Same for make and make install.
4) Log any text output to a text file.

PROBLEMS:

1) Comparing the actual output file to a file containing the expected output (that is, the output expected for a successful installation), implies
   that the expected output file was generated in the exact same environment on the exact same machine. This is a pretty stupid restriction IMO.
   We __could__ parse the actual output file for patterns indicating a failure to successfully install samba and assume that everything went well if
   no such patterns have been found. The problem here is to know exactly which patterns to look for.
2) How do we reliably get the gcc path? We could use the shell command 'which', but this will only work for testcases that run in a UNIX-like 
   environment (UNIX, Cygwin). On Windows machines we might have to crawl through the system registry to obtain that information. Yuck! =(
3) This is more a general problem than anything to do with samba in particular, but do we want our config file to be environment-specific or not? 
   Meaning, should one config file work for all machines and platforms, or are there going to be cases where a config file must be specially tailored
   for one particular machine? This would apply if we were to introduce a <dependencies> tag, which lists the paths of all applications/plugins/etc
   that are required to run the testcase. These paths may vary from host to host which is why a host-specific approach would be necessary.
   
   
   
Thoughts on how to control remote execution of the testsuite on an array of hosts from one central 'issuing' host ( 05/12/2004 )
_____________________________________________________________________________________________________________________________________

Approach 1:
-----------

    The testsuite program (along with the testcase subscripts and config files) must be present in a shared network folder reachable
    by all hosts that are to run the testsuite program. The hosts that are supposed to run the testsuite program shall henceforth be
    referred to as 'executing hosts' whereas the host that initiates the remote execution is called the 'issuing host'.
    Now what happens is the issuing host reads a file that lists all executing hosts then iterates through the list and for each executing
    host opens a secure shell connection to that host, starts the testsuite program and closes the connection immediately upon doing so.
    It is vital that the testsuite program be started as a background process and that any program output be redirected to some file (or /dev/null)
    or else we'll be unable to exit ssh.
    This way the testsuite can be fired up on a large amount of hosts in rapid succession using a single ssh connection without having to wait
    for each instance of the testsuite to finish before going to the next issuing host. 
    If we want the executing hosts to send any log files to the issuing hosts we will have to find a way of letting the executing hosts know
    WHO the issuing host is. Off the top of my head, I can think of the following ways to do this:
    A) Pass the name of the issuing host as a script parameter.
    B) Create a temporary file, write the name of the issuing host to that file and place the file in the same location the testsuite program
       resides in. The executing hosts can then look that file up and extract the name of the issuing host from it.
    C) Just pour any log output straight into the testsuite folder? Hmmm, I'm not sure if that's a good idea. 
    
    PROBLEMS:
    
    1) This is all nice and dandy as long as we're only operating on nix machines but as soon as one of the executing hosts (or the issuing host
       for that matter) runs Windows or some other OS that lacks a ssh server, this approach goes down the crapper.
       
Approach 2:
-----------

    Instead of relying on ssh ( for which there is no counterpart on Windows machines, and if there was one it would most likely be incompatible with
    a UNIX secure shell anyway ) we could use a mechanism that is present on both systems. Communication via sockets comes to mind. The bad news is that
    this would be a __LOT__ more complicated, error prone and probably less efficient. Another problem is that while I can see how a process on the exe-
    cuting host, once running, can be controlled from/by the issuing host, it is not quite clear to me how (if) the process can be __started__ from
    the issuing host. As of now I can only think of a solution that requires a "testsuite daemon" or "testsuite server" to be constantly running in the 
    background on every potential executing host. The issuing host can then connect to such a "server" and send it some command upon which the daemon on the
    executing host starts the execution of the actual testsuite program. The idea of a daemon that needs to be running at all times on every single executing
    host is not particularly inviting to me, though.
   
    
    PROBLEMS:
    
    1) This approach requires one TCP/IP connection per executing host to remain open for the entire duration of the testsuite program. If the number of 
       executing hosts is very large we may run out of available socket connections and thus be unable to open any further connections ( I don't know
       if this could happen but I can imagine it ). We could just wait until some of the open connections are closed and then proceed to contact the
       remaining executing hosts, but still if we have a REALLY large number of hosts this could take all day!
       
       !! Wait a second, that's a bunch of baloney!! It's not actually true that the connection has to stay open for the duration of the testsuite program!
       It just needs to be open long enough so that the issuing host can tell the executing host to run the testsuite program and with which parameters. If
       we also tell the executing hosts how they can contact the issuing host, each executing host could itself open a connection to the issuing host once
       it has finished execution of its instance of the testsuite program and send the results back to the issuing host! However, this means that the 
       issuing host, too, will need some kind of server functionality since it must be able to listen for and accept incoming connection requests from 
       executing hosts.
       
    2) Permissions. I'm not sure which kind of permissions a user/program needs to be able to open TCP connections to remote hosts.
    
    
Approach 3 (added 05/13/2004):
------------------------------

    An alternative to approach 2 would be to use RPC's. We would basically write an RPC method to be called by the issuing host to fire up the testsuite
    program on the executing hosts. RPC's should be implemented on both UNIX and Windows machines. I am, however, unsure if a special package needs to
    be installed for that or if that functionality is coded into the OS itself.
    
    PROBLEMS:
    
    1) I'm assuming that when making a remote procedure call, a connection between the client making the call and the server that executes it stays open
       for the duration of the call. If this is the case we're again faced with the possibility that we might run out of available connections if there's
       a very large number of executing hosts.
       
    2) Although RPC daemons are certainly available for both platforms it appears that they're not necessarily installed or at least not running on some
       hosts.
       
       
    3) (added 05/27/2004) Apparently we'd have to write a dedicated RPC-daemon to run on every potential executing host. What's more, there's no standard
       RPC-implementation for Perl, just a whole bunch of mostly inofficial modules of varying quality and maturity.
       
    => RPC shelved for now! We're gonna go the ssh-route.

    
     
Arguments to pass to individual subscripts or the interpreter running them ( 05/27/2004 )
__________________________________________________________________________________________________


Several types of arguments
----------------------------

    * Fixed arguments. These are arguments that must be passed to __ANY__ subscript. 
    
        a) Path of the logfile to which to direct any output of the programs controlled by the script. 
        b) Path of the perl interpreter to use??? ( not sure if I need this. I might just hardwire this into the script. )
        c) Path of any external lib dirs.
        d) Path of the internal lib dir ( this too could be hardwired ).
        
    * Script-specific arguments. If any and how many of these are passed to a subscript really depends on the individual testcase.
    
    
    
Logfiles ( 05/27/2004 )
__________________________

    Q: What kind of local logfiles should we consider?
    A: * Per-testcase logs that log program output __AND__ subscript output.
       * Per-testcase logs that log program output only.
       * Per-testcase logs that log subscript output only. 
       
       * One log per executing host that logs the overall status of the testsuite on that specific host.
       
    Q: What kind of global logfiles?
    A: * Maybe one global log on the issuing host that logs the overall status of the whole thing ... dunno yet.
    
    !: We need one directory in the testsuite's main dir that is writable/excutable/readable to all executing hosts and the issuing host.
       Inside that directory, the executing hosts are going to place their logfiles ( how do we differentiate the logs of individual hosts
       and testcases from one another? ). When the testsuite is finished, the program on the issuing host can move/copy the logs there to
       another directory ( path can be passed as an argument to the main script ) or otherwise process the logs.
       
    Q: Should we ensure that this directory is empty at each start of the main program?
    A: Probably, yeah. This is easy enough anyway. Just delete everything in there and fail if this isn't possible.
    
    
Communication of subscript programs with the testsuite program and the main program on the issuing host ( 05/27/2004 )
_______________________________________________________________________________________________________________________________

    Q: Why would they need to?
    A: * Inform the testsuite program on an executing host that an individual testcase has finished.
       * Inform the main program on the issuing host that an individual testcase has finished ( or that a certain executing host
         has finished all its testcases? ).
         
    =(   Meh, this can get pretty messy. Do we really need that? Imagine a scenario with 50 executing hosts and 100 testcases. 
         If the issuing host has to keep track of all those testcases ... ugh!
         
    Q: Anyway, what options do we have? 
    A: * Sockets. Complicated, large overhead, error-prone, might run out of available connections.
       * System-V-IPC. Nuh-uh, no way!
    
    --------------------------------------------------------------------------------------------------------------------------------------------------
    |  * Place an empty file with a unique name into some universally reachable directory whenever a testcase has finished. The main program         |
    |    periodically checks this directory for newly arrived notifications. Hey, that sounds good! =) !!!!!!! Come to think of it, the file         |   
    |    doesn't even need to be empty. We could write a short summary on the concluded testcase in there ... time elapsed and stuff like that.      |
    |________________________________________________________________________________________________________________________________________________|    
         
Starting individual subscripts from within the testsuite script ( 05/27/2004 )
________________________________________________________________________________

    So far we're running the subscripts via system(), which is a blocking call. What this means is that the testsuite script will stop at
    each call to system and wait until the started subscript has exited. This may take any amount of time. It would be better if we did a fork
    and then exec() instead of system(). That way we could fire up the subscripts in rapid succession, not waiting until one script has returned
    before we start the next.
    
    
    
Writing a Utilities module with frequently needed functions ( 05/27/2004 )
__________________________________________________________________________________

    Q: What kind of functions would we put in there?
    A: * getDirectoryOfThisScript( $rstrWriteToThisString, [$strAppendThis] )
       * pairedVec2Hash()
       * hash2pairedVec()
     

