ACTIVITIES OF MATTHEW BISHOP FOR THE ELLA ASSOCIATES ALEXIS PROJECT
A Quick Summary And Overview.
The majority of the work done by m.j.b can be seen in three directories
/var/www/alexis-info/docs /var/www/internal and /var/www/utils
All the directories are available from the Web at the urls
http://www.ella-associates.org/alexis-info/docs/
http://www.ella-associates.org/utils/
http://www.ella-associates.org/internal/
Also, the directory /usr/local/bin/ contains some general administrative
scripts written my m.j.b. This file also gets used as a kind of 'scratch-pad'
of ideas which occur to me, and which aren't coherent enough to go
anywhere else. Some times you will see a [*] set of characters where they
shouldn't be. This is because of my dodgy text to html conversion scripts.
Just ignore them.
Quite frequently in the text below, I refer to a computer known as the
'Local Server'. This is somewhat confusing since the work 'Local' in this
sense does not mean 'local' in the English sense of the work (close by, or
not foreign etc) but in the Spanish sense of the word as in 'Shop'. This
is because N.Reddel has/ rents a 'space' or 'studio' which in Spanish terms
is known as a 'local'. The 'local server' is the server located in this
Local which is located in Calle Blanqueria in Barcelona. This computer
is running Debian Linux with various Server Applications installed:
Apache, Tomcat, MySql, Alexis ...
Dates and Descriptions
* March 2, 2003
Created Hotmail accounts for testing of James, in particular, the problem
concerning the delivery of multiple email's to hotmail accounts. This problem
was brought to our attention by Robert Grundy.
The hotmail accounts are as follows (e.g. [email protected])
ella1111, ella1357, ella2468, ella3579, ella7531
all accounts have a password of 'bacibaci'
It is possible to deliver mail to the accounts using the 'raw' SMTP
protocol by connecting to a server like 'mx3.hotmail.com' where
the number after the 'mx' part could be 1,2,4 etc, on port 25.
The 'Putty' application can be used for this by selecting the 'raw'
protocol option. For yahoo you could try 'mx1.mail.yahoo.com'.
Specific error messages for James when delivering mail can be found in
/usr/bin/james/apps/james/logs/mailet.log
An archive containing questions and answers for the James Mail Server can
be found at
www.mail-archive.com/[email protected]">www.mail-archive.com/[email protected]
An email was sent through James successfully to the hotmail accounts
listed above using Outlook express and a comma separated list of email addresses.
The answer to the James multiple hotmail delivery problem is possibly that if one
incorrect email address is present for a particular destination (eg hotmail)
then James fails to deliver any of the email's for that particular destination.
However James does successfully deliver mails in the same 'batch' to other
destinations (such as yahoo).
It is possible that in a sense this is a hotmail problem. For example,
when multiple recipients are specified using raw SMTP protocol to
somewhere like mx3.hotmail.com, if one of those recipients is not
valid, then it appears that none of the recipients will receive the
message. According to rfc821 and rfc2821 this appears to be incorrect
behaviour. Yahoo Mail for example does not have this problem. It
delivers to the valid addresses even if on of the mailboxes is not
valid. However, an SMTP mail server could overcome this problem by
sending the message to each mailbox individually, which James apparently
does not do
Wrote small Java program to check hotmail accounts
It is located at /root/checkhotmail.class but it only works on Windows (!)
Perhaps this is the readLine() method of the BufferedReader class. It is
possible that the line separator property for Unix needs to be set in
order for the readLine() method to word properly.
* March 3, 2003
Researching security problems for the 'local server'
Examining the Debian Security How-to/ Manual.
There are Debian tools, such as 'nessus' which launch attacks against
a Server Computer in order to evaluate how vulnerable that machine is.
* March 4, 2003
Not much, thinking about the vi editor.
Discovered 'netcat', invoked as 'nc' which is a network analysing tool
Which allows scripts to makes TCP connections.
* March 5, 2003, Wednesday
Working on FTP security:
Discovered 'netcat', invoked as 'nc' which is a network analysing tool
which allows scripts to makes TCP connections.
Have noted an extremely large amount of stuff in the syslogs which looks
dubious. It looks roughly like this.
Mar 5 19:20:03 www sm-msp-queue[22513]: h21BU1gk005619:
to=postmaster, delay=4+06:50:00, xdelay=00:00:00, mailer=relay,
pri=55566438, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0,
stat=Deferred: Connection refused by [127.0.0.1] Mar 5 19:23:01
www /USR/SBIN/CRON[22517]: (mail) CMD ( if [ -x /usr/sbin/exim
-a -f /etc/exim/exim.conf ]; then /usr/sbin/exim -q ; fi)
It is probably generated by the 'james' mail server but why?
Created a directory /var/guest-ftp-setup/ which is a mirror of the
/home/vanessa directory which nick created to setup guest ftp. The
/var/guest-ftp-setup/ directory has the same permissions and ownerships
as the /home/vanessa/ directory
created a shell script which automates the process of establishing a
guest ftp login account. This script is located at
/usr/local/bin/make-ftp-guest
and can be invoked as (from any directory)
make-ftp-guest theUserName
* March 6, 2003, Thursday
I am investigating the possibilities of 'sslwrap' to deal with the problem
of a secure ftp connection. There is a Debian package for this and reasonably
OK information.
From reading of the www.wu-ftpd.org mailing list archives it appears that
version 2.8 of wu-ftpd will support ssl (or TLS?) but that this is not
a stable version yet. The 'local server' is running version 2.6.2
www.landfield.com/wu-ftpd/mail-archive/wuftpd-questions/2002/Aug/0119.html
has a list of ftp clients which support secure ftp (ssl/tls)
The secure ftp problem is solved very simply; The sshd daemon, which is already
running, includes support for the Secure Copy Protocol (SCP). All that is
required is a Microsoft-windows client that is able to connect to the sshd server.
Two good clients are available. The 'Putty' psftp command line client, and
the WinSCP GUI client (which appears to be based on the putty client).
The WinSCP client can be found at winscp.vse.cz/eng/
and the putty client can be found at
http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
All access to the 'Local Server' as the 'root' user should be made using
these clients rather than a standard ftp client connecting to the
wu-ftpd server. The normal users of the 'local server' can continue to
connect using the 'wu-ftpd' server, since their passwords are not as
critical to the security of the system.
* March 7, 2003, Friday
not much. Am looking at the cvs system, and whether the 'Local Computer'
has a cvs server currently running. The Alexis Code needs to be moved
into a cvs system.
Other things to do: create a swing applet/ application to look up
the validity of Hotmail Email Addresses. [note: this was eventually done
but not as an applet. It was done as a cgi-script, which is can be found
from the URL http://www.ella-associates.org/alexis-info/docs/resources.html ]
Type 'info cvs' for the main cvs user documentation or
type 'lynx /usr/share/doc/cvs/html-info/cvs.html' for an html
version.
* March 8, 2003,
Inorder not to have to use the -d switch with the
cvs checkout command, you need to set the CVSROOT environment variable.
The question I have is, where is the best place to set this variable?:
In the .bashrc file for each user, in the /etc/profile 'system wide' file,
or somewhere else?
Summary of importing code into cvs and setting up a module. This process
was extracted from the cvs manual (cederqvist). Lines beginning with a
'hash' are comments or non literal intructions.
#-- Set up the CVSROOT environment variable so that you don't have to type
#-- '-d /var/lib/cvs' every time you use a cvs command. The CVSROOT variable
#-- tells the cvs client where it is to find the cvs 'repository', which is
#-- where files and their versions histories are kept.
CVSROOT=/var/lib/cvs
export CVSROOT
#-- The above lines only work for the current login session. The
#-- CVSROOT environment variable is not set for future sessions.
#-- change directory into the top level directory where your code
#-- directory structure resides. For example, cd /alexis
cvs import -m "Some Comment" directory/path/within/repository YourCompanyName SomeTag
#-- The directory path mentioned above should not contain a leading forward
#-- slash (it needs to be a relative directory path). If the directory path
#-- within the repository does not already exist then it appears to be automatically
#-- created when you issue this command.
#-- Check out the Repository modules file 'modules' with
cvs checkout CVSROOT/modules
cd CVSROOT
#-- Text edit the 'modules' file to add a line like
#-- yourModuleName repository/relative/path/to/code
#-- Now, commit the changes to the modules file with
cvs commit -m "Some Comment about what you have done" modules
#-- Now, get rid of your working copy of the CVSROOT directory
#-- (This can't be done when you're actually in that dir, obviously)
cd ..
cvs release -d CVSROOT
Now you can checkout a working copy of the code with
cvs checkout yourModuleName
In other words you don't need to worry about the directory
structure within the cvs repository. This assumes that you are working
on the same machine as the cvs repository. See below if this is not the
case.
To set up the cvs server (to allow for cvs access across a network or
across the internet), do the following, roughly:
Add a line to /etc/inetd.conf refering to the pserver
Restart inetd with kill -HUP <process id>
Create a file /var/lib/CVSROOT/passwd with one line such as
anonymous:
This will need to be changed to stop any old person making changes to
the Alexis code over the internet. Look at the actual file to see a
better way to do this.
Try connecting from a different computer with
cvs -d :pserver:[email protected]:/var/lib/cvs checkout geocities
where geocities is a test module which matthew set up.
http://ccvs.cvshome.org/fom//cache/124.html
Here is a short spiel on how to setup a cvs pserver with xinetd
Created two accounts for the cvs pserver (which is the client/server version of
cvs, in other words the accounts to be used when accessing the 'local server'
computer from across the network or across the internet.)
The two accounts are
password hidden
The cvs server can be accessed with the following command line
cvs -d :pserver:[email protected]:/var/lib/cvs login
To create these accounts, I had to cut and paste an encrypted password from
the /etc/shadow file (since we are using 'shadowed' password, otherwise it would
be the /etc/passwd file). I used the 'adduser' program to generate the encypted
password, even though it was not necessary to actually add a user to the system
in order to use cvs. See the 'cederqvist' manual for more details.
You could also try this to generate the password (from the cvs-home faq)
perl -e 'print crypt "password", "sa"' >>$CVSROOT/CVSROOT/passwd
All this appears to be working in Netbeans as well. By using the menu
items 'versioning/mount version control/cvs' and then following the
the steps of the wizard. The 'Connection Method' Box of the Wizard is the
most tricky. Leave the connection method as 'password server'
Server Name = ella-associates.org
User Name = nicko
Repository = /var/lib/cvs
port = 2401 (no need to change this)
password = as supplied elsewhere
* March 9, Sunday
not much
* March 10, Monday
not much. NetBeans OpenApi familiarisation
* March 11, Tuesday
xml and Netbeans research
* March 12, 2003, Wednesday
not much, fixed network card dongle connection
looked at the chakirya server source code.
* March 13, 2003, Thursday
updated this file.
A unusual Apache quirk: the default document for a directory, for
example 'index.html', gets 'cached' by the web-server, so that even
when you change the index.html file, or delete it, the changes are
not apparent, unless the user specifically requests the document by
name, e.g. www.ella-associates.org/index.html. Requests to
www.ella-associates.org/ will yield the old document. I presume this
is a feature rather than a bug.
This is a strange problem. I even tried restarting the server with
'apachectl restart' but the old index.html file is still being served by
the web-server. It just wont go away. When I put and index.html file in
nhe 'test' subdirectory, the same thing happens but after it is deleted
it eventually goes away. Could this be some odd client side problem
with Internet Explorer refusing to not cache the document? Who knows.
A slight 'gotcha' with restarting the apache webserver. You actually
have to be in the /etc/init.d/ directory or the 'apachectl' script
throws an error (cant find some file)
This situation is especially odd considering that I have changed the
index.html file for the document root directory /var/www/ before.
* March 14, 2003, Friday
A wild guess: could the problem above have something to do with
Tomcat? Installed 'vim', an enhanced version of 'vi'
The network card connection for this computer is slightly dodgy.
Sometimes it is necessary to lightly press downwards on the network
card connector in order to get it to work, or even to balance a
not-too-heay object on the connector cord. The network card connector
relies on a kind of 'pincer' action on the network card connection
in order to work. But the gap between the two sides of the 'pincer'
is now slightly wider than it should be, hence the problem.
The Apache caching problem is incredibly infuriating. In MS Internet Explorer, in the
menu, tools-->options [general tab, temporary internet files box]
-->settings-->check for newer version of pages-->every visit
By setting this option, the situation is improved slightly, but the
old index.html is still being served
Created a file 'ella-associates.org/docs/resources.html' which contains
some url links to resources related to the Alexis project.
Created some basic filter scripts in /var/www/utils/ to turn text into html
In the /etc/rc files there are references both to apache and to apache2.
This does not seem wise. How does the operating system know which daemon
to start up at boot-time?
Created an apache password file at /etc/apache/passwd/ella-members-passwords
This is for allowing http access to the /var/www/internal/ directory.
Created a password protected directory tree at http://www.ella-associates.org/internal/
The index.html file for http://www.ella-associates.org/ has finally actually
been 'registered' by Apache. I really have no idea why. In other words the problem
which I have been refering to, at length, about how the 'index.html' file which
was served by the Apache Server was old, not up-to-date, and in some cases didn't
even exist any more, that problem, has now mysteriously gone away.
Created a set of filter scripts in the directory /var/www/utils/ which turn
various types of text documents into html documents. Some of these filters
are 'diary2html.sh', plaintext2html.sh, and linkdoc2html.sh. The filters
also add links to the Google automatic translation engine for Spanish, French,
Italian, German, and Portuguese.
* March 15, 2003, Saturday
Installed pretty good privacy, pgp on the Almetlla De Mar laptop. The outlook express
plugin does not appear to be working but it doesn't really matter.
I used the 'netword associates' pgp, version 6.5 something I think. And I got
it from http;//www.pgpi.org/
Played tennis with Alex. Lost 6-3.
Created a 'quick guide' to pgp for any other Ella Associates person who needs to
install it
* March 16, 2003, Sunday
The problem refered to above about apache or MS I.E. not serving or displaying the most recent versions of
pages, seems to intermittently re-occur.
Tried to start the telnetd daemon from within inetd, by enabling the telnetd line in
/etc/inetd.conf and doing a 'kill -HUP 1496' to restart inetd. The telnet daemon started
and permitted a login but on successful login says 'cant execute /etc/ftponly'
I wanted to start the telnetd for the purpose of debugging the DNS set-up.
I probably don't need telnet to test dns anyway. The 'dig' tool is better.
Am reading the DNS Howto which is at http://www.tldp.org and at http://langfeldt.net/DNS-HOWTO/
But, its not very helpful about how DNS works, and its written in poor english with
plenty of mistakes. Nevertheless it is useful. There seems to be a reasonable manual
which is installed with the 'BIND' package. This manual is written by the people who
create and maintain BIND. It is available in pdf and html format. Also, perhaps refer to
the document http://www.ella-associates.org/alexis-info/docs/dns-learning-materials.html
* March 17, 2003, Monday
ill, didn't work
* March 18, 2003, Tuesday
ill, didn't work
* March 19, 2003, Wednesday
Started the document /var/www/alexis-info/docs/the-ramble.txt
This is supposed to be a simple explanation of Alexis ideas aimed at
some-body like Marcus.
James Info: to manage use 'telnet localhost 4555' and login with
'root' and 'jimi57'
* March 20, 2003, Thursday
Still sick
* March 21, 2003, Friday
Still sick.
made email users 'mail' and 'webmaster'
Made a summary of the Chakriya source code with the following command-
# (echo "<html><body><h1>Chakriya Source Code</h1>"; for f in *.java; do echo "<h4>$f</h4>"; cat $f | \
indent | /src-highlite/bin/source-highlight.exe --src-lang java --out-format html; done; \
echo "</body></html>") > all.html
I wanted to then use this highlighted code (which is html) as input for the 'htmldoc'
processor. The htmldoc program (GPL on unix) is able to convert html to Adobe pdf format.
However, the htmldoc program does not recognise all html tags and for this reason the
pdf output was 'scrambled' or 'garbled', that is, there were sections of the pdf text
which contained overlapping and super-imposed words and also some of the source document
words were missing. In addition the 'indent' program is designed to format or indent
c code rather than Java code and this also caused the output to be less than good.
* March 22, 2003, Saturday.
Still ill.
Investigating the Chakriya Source Code further. Added slightly to the document
'ChakDoc.txt' which is an attempt to document the Chakriya Server, but is still
very rudimentary.
Using the 'curl' tool to investigate the Chakriya Server from the command line.
I am having difficulty actually working out the xml syntax or dtd for the requests.
This syntax can be obtained from the 'output' windows of the Alexis Client Application
It seems possible to communicate with the Chakriya Server with something like
curl -d "<alexisRequest><login-request><blah></blah></login-request></alexisRequest>" \
www.ella-associates.org/alexis/alexisServer
This is obtaining a response from the server, sometimes and empty <alexisResponse> tag
and sometimes a Stack Trace server error, if an illegal tag is introduced.
* March 23, 2003, Sunday
Not much
* March 24, 2003, Monday
Working on security breaches that mustafa ([email protected]) has found.
Mustafa appears to have been able to enter the 'local server' through the
ftp server and appears to have been able to enter each of the guest accounts.
He does not appear to have gained root access, but was able to upload files.
Investigating Nicks problem of the javascript tree view component.
At www.treeview.net there is a good example of a pure javascript treeview component
that seems to work well, but there is no xml data-binding kind of capability, although
it should be resonably straightforward to write something to do that.
* March 25, 2003, Tuesday
made a user 'webdeveloper/webdeveloper' who's home directory is /var/www/
This was to allow josephine to work on the 'ella-associates.org' site.
However this is probably not the best or most secure way of doing this.
The security problem seems to be that the ftp server is allowing
anonymous ftp, and a script that I wrote, 'make-ftp-guest' had
a bug in it that chrooted all guests to the /home directory instead
of to the /home/[username] directory. This could be the reason that
mustafa was able to put files in everybody's home directories.
Prevented anonymous ftp access to the 'local server'. Also prevented
access for the 'root' user.
* March 26, 2003, Wednesday
The 'enter/return' key on this laptop is becoming unresponsive (has improved since).
Created a small email program set-up how-to for ella-associates.org
which is at http://www.ella-associates.org/alexis-info/docs/ella-email-how-to.html
* March 27, 2003, Thursday
Created a set of backup scripts at /usr/local/bin such as
'backup-ella-web.sh' and 'backup-cvs-repository.sh'. These scripts backup
the ella-associates.org website and the local-server cvs repository
respectively. They also 'rotate' the previous backup copies.
These scripts are scheduled in the /etc/crontab file to run automagically
at certain intervals (approximately once every 2 days in the early morning)
My documentation pages and alexis information have been moved to
http://www.ella-associates.org/alexis-info/
* March 28, 2003, Friday
Expanded the document 'javascript-treeview.txt'. Continuing to investigate the
use of javascript tree-view components in a web-page.
* March 29, 2003, Saturday
More Java/Script Tree View Component Research. Going to a meeting in Barcelona
with Nick tomorrow, 5pm
* March 30, 2003, Sunday
Went to a meeting in Barcelona. Did not work
Some of the things suggested for me to do were:
Implement an IDE Auto-updating feature for Alexis
Implement a schema validation for Nick XML tag editor. Implement Colors
for the tag editor. Implement some chat system or discussion forum for
Alexis. Investigate XML Databases.
Send the pgp how-to link to everybody in the 'team'. Send the XML explanation: Done
link to everybody. Set up 'grapevine.ella-associates.org' and 'language.ella-associates.org'
(possibly not urgent because they won't be used immediately.)
Do the necessary massaging of the Lexicographical test data so that it can
be inserted into Alexis.
Investigate a Tomcat 'glitch' which appears to stop large numbers of accesses
in very short periods of time. This behavior was reported by Nick.
* March 31, 2003, Monday
Returned from Barcelona. Did not work
* April 1, 2003, Tuesday
Not a great deal. Extended 'The Ramble'. Revised some docs.
Created some tables of contents.
* April 2, 2003, Wednesday
Revised Documentation. Began to look at Tagged Editor
* April 3, 2003, Thursday
Looked at available xml editors. Reviewed some web collaboration software.
* April 4, 2003, Friday
More Collaboration software. See the review at 'resources.html' Can a
working discussion forum for Ella be easily set up? yes (YaBB). Would it
be worthwhile? I dont know. It may be 'just another thing' that people have
to check. It may be better to wait until it is implemented within Alexis.
* April 5, 2003. Saturday
Questions: what is IRC how does it work? what are the implications
of using GPL software in a commercial venture? Some parts of the code
will have to be given away. To do: The Tag Editor Familiarization.
Get j-ftp. Wash up.
* April 6, 2003, Sunday
Sat in the sun. fixed another Robert Hotmail problem (the same problem
as befor)
to do: still tag-editor familiarization.
Off-site backups.
* April 7, 2003, Monday
Day off
* April 8, 2003, Tuesday
Another day off. Had a business meeting with Nick in
'pica pica'.
* April 9, 2003, Wednesday
Emailed Vanessa, made vim 'filter' aliases. For example, in order to turn this file
you are reading (assuming that you are reading the 'plain text' version, and that
you are reading it in a text editor called 'vim'), in order to turn this file into
HTML, all you have to do is type ',3' (that is: <comma> <three>) from within the
vim editor. You don't actually type those single-quote characters ('), they are just
there to separate the character sequence from the rest of this text. This character
sequence runs the script 'diary2html.sh' on the current file. If you type ':map'
you can see what other character sequences are available. Read interview with Bill
Joy at 'linux-mag.com'.
Notes for making a backup script for the Alexis Source code and Data
The script can use a cvs 'update' command to check out recent changes to the alexis
source tree from the (putative) server in the U.S. to the 'local server'. I say
putative because this U.S. server has not yet been established but possible will
be over the next few days. To test this functionality, I can run the script from
the cygwin shell, or another, from the 'ametlla' laptop. This method of using cvs
to achieve the backup allows us to do 'incremental' backups without having to code
that functionality ourselves. This incrementality greatly reduces the amount of data
that has to be transfered over the Internet. A possible disadvantage of this, is
that the versioning information that is contained in the CVS repository, will be lost.
Also, the MySql database needs to be backed up. This will be hopefully achieved by
using some MySql incremental backup facility. Also it is envisioned that the Alexis
Client could be easily modified (or extended) to act as a backup agent considering
that it will already incorporate functionality to allow for the caching of data and
for the incremental 'refreshing' of that data. Also the 'log' table of the MySql
database, which is the table which stores versioning and change information for the
Dictionary data used by Alexis, will need to be backup separately because the Alexis
client will not be able back this up.
also to do: create a symbolic link from ../alexis/docs/ to
/var/www/internal/alexis-docs/ or something similar. Or else create an Apache 'alias'
for this. Creating an alias may be difficult because the 'internal' directory already
exists, and because I am not sure if authorisation restrictions will apply to the
aliased directory or to the symbolic link.
* April 10, 2003, Thursday
Little things: I should modify the 'filter' scripts so that they recognise the
'query string', that is anything after the ? character as part of the url. At the
moment this is especially a problem for the 'mailto:' hyperlinks
Little things: Create a general 'ella' account for the 'internal' dir
Created a CVS module name 'alexis-docs' which points to the ella/alexis/documentation
directory withing the CVS repository. This module name is going to be used to
publish the documentation on the apache web server at the url
http://wwww.ella-associates.org/internal/alexis-docs/ .This can be achieved by
using the cvs checkout command when in the 'internal' dir in the following way
cvs checkout alexis-docs or if the /var/www/internal/alexis-docs dir already
exists use the 'update' command.
Imported the /var/www/utils/ and /var/www/alexis-info/docs/ directories into
the CVS repository. Set up the CVS repository with a definitive alexis code
tree (imported the code into the repository). This tree is now going to be
used for all development, which means that all alexis developers (i.e Nick Reddel)
must use CVS when developing. Thankyou.
[note: in reality cvs was never really used, neither for code development nor
for the documentation]
This import failed because the binary files were treated as text
by the CVS program. It is necessary to explicitly inform CVS which types of
files are binary.
The binary files extensions which we need to inform CVS about are as follows
gif jar png zargo zip exe jpg jpeg act ACT psd PSD pdf rar eps ai doc xls ppt fla swf
But what about '.class' files, for example?
[note: I would like to congratulate myself for recording this detailed information
since I forgot to get the appropriate cvs configuration file from the Barcelona
Debian Server]
The line below apparently tells CVS that 'exe' files are binary, but it may
be necessary to put multiple expansions in a file called 'cvswrappers' (yep). This info
is from the rather 'garden-pathish' cederqvist manual for CVS, which is the
main manual.
cvs import -I ! -W "*.exe -k 'b'" first-dir vendortag reltag
I placed the necessary lines in the 'cvswrappers' file, which is at
/var/lib/cvs/CVSROOT/cvswrappers (but this file should be edited by doing
a cvs checkout CVSROOT/cvswrappers command, editing the file and then
doing a 'commit'. This appears to have worked and the source tree appears to
now be happy lying in the repository. Nick's original ftp'd source tree
is at /home/matthew/alexisSource and a 'test' checkout of the code is
at /home/matthew/alexis.
In order to update the web publishing of Nick's documentation you do the following
cd /var/www/internal; cvs checkout alexis-docs
This appears to do the trick even if the directory 'alexis-docs' already exists.
Also should have a look at a thing called webDAV which may be a type of cvs
for non programmers. (No it is a protocol based on http which allows the
collaborative editing of documents)
Crazy ideas: create an Alexis and also Ella faq-o-matic
(http://faqomatic.sourceforge.net ) and an Ella and Alexis YaBB forums
It would also be useful to have 'discussion' pages where people
could contribute by clicking on a hyperlink.
A windows NT line to get a password protected page using wget
D:\mjb\utils>wget --http-user matthew --http-passwd bacibaci www.ella-associates
.org/internal/ella-internal-docs.tar.gz
A bash line to backup the /var/www/internal directory excluding the large
'all-chak' documents and Nick's documentation directory which is also quite large.
cd /var/www/internal/; rm -f ella-internal-docs.tar.gz; tar -czvf ella-internal-docs.tar.gz [^a]* | less
A bash line to backup the docs directory excluding the large 'netbeans-userguide' files.
The less pipe is not really necessary but allows the user to see what files are
being 'tar-balled'. This relies on the fact that the only files in the directory beginning
with 'n' are the netbeans user guides.
cd /var/www/alexis-info/docs; rm -f ella-web-docs.tar.gz; tar -czvf ella-web-docs.tar.gz [^n]* | less
Started a script which will add comments or other types of information
to a text or html file.
* 11 April 2003, Friday
Need to change my plane ticket.
Overcast day. Backup script is still to do. Created a prototype document
commenting script, which allows a reader of a document to add comments or
other text to it. It is located at /usr/lib/cgi-bin/add-comment. Also
I created a text2html script (plaintext2html-forum.sh) which is designed to
be used in conjunction with this cgi script. The filter script creates the
input form and the necessary links to the cgi script. These scripts are to
be rewritten in Java Servlets or JSP or PHP and are to use a MySql storage
system and to support threading.
I have set up the filter scripts so that the main repository is at
/var/www/utils/ but most of them are 'symbolically' linked to
the /usr/local/bin/ directory. This puts them on the 'path' and allows
them to be used from any directory.
Created an ftp guest user 'nick'
Added a line in the /etc/vim/vimrc file to map the ,1 key sequence to
running the forum generating script.
map ,5 :exec "!plaintext2html-forum.sh " . expand("%:p") . " > " . expand("%:p:r") . ".html"<CR>
Essentially finished the prototype version of a 'forum' script which
allows a user to add a comment to a text file. This now needs to be translated
into JSP, given a MySql database and made to support threading.
* 12 April 2003, Saturday
Didn't really work. Visitors (kerry lewis) arrived. Thought about the potential
of text messages. When to la quadra restaurant
* 13 April 2003, Sunday
Read Nicks 'project overview' document. Revised the main documentation
page. I can send an email with 'telnet localhost 25' on the local server
* 14 April 2003, Monday
Struggled with 'netcat' and sending an email to the 'local server' james
mail server using a bash script. A detailed description of this process is
available in the file 'smtp-torture.txt'
* 15 April, Tuesday
Note that the output of 'linkdoc2html-forum.sh' does not display
in Netscape 4.61
On Win NT, 2000 or whatever you would like to call it, the following
command line from within the 'sh' shell, when run from d:\mjb\ella-backup-site\
will create a tarred and gzipped backup of the utils subdirectory. The backup
is given a datetag for the current day. If more than one backup per day is
desired this would have to be altered by adding an Hour tag. Run 'date --help'
to see what format tags are available for 'date'. Note on WinNT there is already
a 'date' program thats why I renamed the unix one to ddate.
DIDO# datetag=$(ddate "+%d%B%Y"); tar -cvf utils.$datetag.tar utils/*; gzip utils.$datetag.tar
created a backup script to get the stuff I have been doing off the local server
and onto the Amettlla laptop. It is at d:\mjb\ella-site-backup\backup-ella-site.
Apologies for the name. It also archives old backup copies. It needs a bash shell
to run. I attempted to schedule it in Windows 2000 with
at 19:45 /every:M,T,W,Th,F,S,Su sh backup-ella-site
Apparently 'at' is the MS Windows 2000/NT scheduling command (did they hire someone
with a unix background?). The only real doubt I have is 1. Is the NT scheduling
service running? and 2. will the at command pickup the 'parameter' to the 'sh'
command. Did not work.
But the follwoing did work
at 05:05 /every:M,T,W,Th,F,S,Su cmd /c d:\mjb\ella-site-backup\start-ella-backup.bat
where 'start-ella-backup.bat' contains
d:
cd \mjb\ella-site-backup
echo "started ella backup" >> start-backup.txt
sh backup-ella-site
So the backup is scheduled for 5.05 am every morning. These techniques should be
easily applicable to the Alexis Repository Backup. The only problem with this is that
the laptop in Ametlla may not be switched on a 5.05 am. It probably wont.
Added warcontent and jarcontent to the 'cvswrappers' file at
/var/lib/cvs/CVSROOT/cvswrappers
Started a backup script for the alexis repository at d:/mjb/ella-site-backup/backup-alexis-repository
or something similar.
Started a php page which will send an email. Actually I think this should go in
the /var/www/index.htm which would become a php file so that a 'confirmation message'
could be displayed to the user.
It may be worthwhile setting up 'sendmail' instead of 'bashmail'. Bashmail is a simple
'bash' unix script which I wrote which sends an email through the SMTP server on the
localhost.
* April 16, Wednesday
Improved the bashmail.sh script so that the email is sent quickly (put a 'sleep' line in
the mail generating script). Some quick php syntax research.
Finished the 'ella-associates.org' e-learning contact form
* April 17, thursday
Did not work
* April 18, Friday
Fixed the 'checkhotmail' java script. Began to configure the U.S. Server.
* April 19, Saturday
Stuff to do: repository backup script, on US server install java, tomcat, transfer
files, change apache document root. Roberts script.
* April 20, Sunday
Some investigation of how to validate against a schema using xerces. The answer, may be,
is Document.normalizeDocument and the page explaining it is at
` apache.org/xml-xerces/java/build2.3/docs/faq-dom.html#faq-6
but really this is not what we want at the moment. We want to know what are all the
valid elements which could possibly be inserted in the current context. Apparently to
do this we should use something like PSIV. Look at the faq in the Xerces documentation
for how to do this.
* April 21, 2003, Monday
Didn't work. Did some house cleaning prior to the arrival of Garry. It was very sunny.
Went to dinner at la cuadra. Drank whiskey and wooden-barrel wine.
* April 22, 2003, Tuesday
Still have to link the 'Tomcat' java application server with Apache. The documents for
Tomcat dont really look that good.
* April 23 to 26, 2003
Haven't done anything. Distracted
* April 27, 2003, Sunday
Finished the 'checkhotmail' web script. This is at
http://www.ella-associates.org/cgi-bin/check-hotmail
The user can check if Hotmail Email Addresses are valid or not.
* April 28, 2003, Monday
Worked on a grapevine band editing capability. Found some interesting sed resources
Wrote an HTMLentitizer script to transform special characters into HTML entities
but then discovered this has already been done (better) at sed.sourceforge.net
[Actually I strongly suspected this would be the case before I started writing the
script, but I was unable to find the script. As it turns out, it would have been
virtually impossible to find the relevant sed script using normal Google searches.
I only came across by accident when investigating other capabilities of sed]
Although the writing of this sed HTMLentitizer script may be seen as a bit of a
waste of time, I did learn a lot about transforming 'control characters' in the
process of writing this.
* April 29, 2003, Tuesday
Not a great deal. Obtained and integrated a style-sheet for the alexis documentation
* April 30, 2003, Wednesday
Grappling with US server problems. Attempting migration to the US server.
* May 1, 2003, Thursday
The problems with the US server: Intermittently it is not possible to log on.
Also, once logged on the connection is 'aborted' quite frequently. This may be
a problem with the way the the SSH server is configured. This problem is currently
causing the US server to be unusable. The connection 'freezes' after a minute or
less.
Also the 'less' pager is displaying hyphens as a's with hats
The solution to this problem may be the '-7' or --ascii switch to man
According to nick the solution to this is to set the character set in 'putty' to
utf-8.
for the US server I may try killall -HUP sshd in order to restart it. also will try
telnet to see if that helps.
* May 2, 2003, Friday
* May 3, 2003, Saturday
Noticed some very odd stuff in the '/root' directory. Don't know what it is.
A little bit of 'proftpd' familiarization. Proftpd is probably a simpler ftp daemon or
server to use than wu-ftpd (which is what the Debian Linux Server is Barcelona uses).
In order to 'chroot' a user into his or her home directory, use the 'DefaultRoot'
directive in the proftpd.conf file. These directives a very similar to Apache style
directives.
The odd stuff in the /root directory on the Barcelona Server was just the
result of an in cautious right-click, rather than some dire hacking
intrusion. Cleaned this up.
* May 4, 2003, Sunday
Sat at the corner table at the La Cuadra restaurant as the sun went down,
and was joined by various people, including Sara and Jordi and Daniel.
Drank some coffee and listened to some songs by Daniel.
Grappled with the U.S. Server configuration. AT the moment I am looking at
how to set up the 'James' mail server. This is possibly very simple but I
tend to need to understand everything before I start. Also I looked at
sendmail but was repulsed by its arcane and ridiculous configuration. I
am happy to ditch it and start again with James (although sendmail is
already installed on the Norte-Americana Server).
Things still to do: copy all of vanessa's stuff over to the US server
set up 'webmin', set-up the 'cgi-bin' directory, notify Josephine of
stuff, notify everybody else, fix the DNS rollo, :w
fix this anoying 'vim' wordwrap thingo, go for a swim, install james,
fix the character-set problem with putty and the US server.
* May 5, 2003, Monday
Another James day. To set system wide environment variables, put them
in /etc/profile or the 'init' script. James was eventually persuaded to
start and run by stopping the existing Mail Server configuration which
existed on the United States Red Hat LInux server. This system involved
an application called 'sendmail' and its various related programs. As I
was able to understand it, there was a master program or 'daemon' which
controlled and 'spawned' the other applications. It was possible to stop
this master daemon, which was identifiable in the process list as
'master', by typing the command 'postfix stop'
Also it was necessary to stop the default pop3 server as well as the SSL
enabled pop3 server. These programs were controlled by the 'xinetd' daemon
which is sometimes referred to as a 'super-server'. By finding the correct
configuration file and setting 'disabled=true' in the ipop3.conf sub-
configuration file, it was possible to stop these demons from causing any
further trouble.
If these defaults daemons are already running it is not possible to start
the James Mail Server. Sadly James gives you a large and not overly
helpful 'stack-trace' instead of a simple 'port taken' message. Although
in the large stack trace there is a 'port taken' message somewhere if you
look.
* may 6, 2003, Tuesday
stuff. Worked on the problem of connecting the Apache Web Server to the
Tomcat Application Server. Both of these applications are developed under
the Apache Software licence at http://www.apache.org The process of
joining the 2 applications involves using the mod_jk module which is
refered to as an 'adapter' in the documentation. The on-line internet
based documentation for Tomcat is somewhat difficult to follow, probably
because of the rapidity with which the Tomcat application has been
developed in the last few years. One example is that the documentation for
version 4 of Tomcat refers to 'startup' scripts which are called
'startup.sh' and 'startup.bat' but these scripts do not appear to actually
exist anymore.
Some things which probably need to be done, still, are: To test Nick
Reddel's Java based XML editor; To look at the Amaya application which is
something that has been developed under the auspices of the w3c and which
apparently has editing capabilities as well as being an Internet Browser.
As Nick Reddel has put it, this is possibly the application which is most
similar to 'Alexis' at the current time. (There is a commercial product
called XmlMind which is also slightly similar).
N. Reddel would like FTP accounts for the following people
Nick, Matthew, robert, mustafa, marcus, jo, vanessa, monkey-rhythm,
poseidonia
(all with ~ - type public html dirs)
Other things: How can web-applications in the same instance of Tomcat send
data and information to one another
josephine/josephine
monkey-rhythm/m0nkey
* may 7, 2003, Wednesday
The apache web-server appears to need 'execute' permission on the
directory for the guest users. That is, the directory /home/username
seems to need to have its 'execute bit' set for 'others' in order for
apache to be able to display the contained web-pages.
The /home/username/public_html directory needs both the 'read' and the
'execute' bits set (I presume for 'other' since apache is running as the
user nobody)
It would probably be a good idea to put a skeleton 'public_html' directory
in the 'useradd' command skeleton directory, with the correct permissions
so that as soon as the new user is added the public_html directory will
be available
The checkhotmail script is not working properly, possibly because of file
permissions for the 'temporary' file [The problem is now fixed- it had to
do with the location of the 'java' executable and the usual Unix
permissions problems]. Or permissions for the directory.
The 'filter scripts' (in /var/www/utils/) are not all working properly
because of some problem with sed. plaintext2html.sh works but not
plaintext2html-forum.sh [PROBLEM NOW FIXED, it had to do with the
'{,4}' range regular expression syntax]
There is a problem at the moment that the server is rejecting requests
to ports other than the very common ones (80 etc). For example it
is possible to access the 'webmin' server which is called miniserv.pl
using the comand lynx http://localhost:10000/ but if the IP address is
used from outside of the American Server, it rejects the request with
a message such as 'connection refused'.
Otherwise Webmin seems to be nicely installed and does what its supposed
to do. Currently it uses the system root password, which will be changed
prontissimo
Created various user accounts.
There was a problem with the 'James' server dying. This was caused by it
not being properly started using 'runlevels'. Instead I was simply
starting it in a normal 'putty' session. It is necessary to put scripts in
/etc/rc.d/ ... which get executed when the run-level is changed by the
operating system or by the user. The operating system changes the runlevel
when boot the system and when shutting it down, but the runlevel can also
be changed manually by a root user. This means that the process (such as
a James mail server) is not owned by whoever happened to start the
process, but it owned by the runlevel, in a sense. If this is not done,
then when the user who owns the process logs out, the process is killed
although I am not sure if this has to be the case.
* 8 May 2003, Thursday
The following problems with the American Server remain: Something is
killing the James Mail Server daemon, The Server is rejecting requests
on most ports except the common ones, the Java Servlets are returning
'no data' messages when they are accessed
The hosts.allow and the hosts.deny need to be edited to allow
new services. By commenting out the line ALL:ALL in hosts.deny
file it should be possible to allow complete access. However, nevertheless
the American server is still rejecting requests to
http://63.105.73.195:10000/
So there is some additional problem apart from the hosts.deny file
Also the 'portsentry' program does not seem to be the problem because
even when it is killed, the connection to webmin as above is still
refused
PROBLEM IS NOW FIXED: caused by iptables.
The above problem was caused by a thing called 'iptables' which is
a piece of software which is part of the linux kernel. Iptables is
something that is often seen in connection with 'firewalls' and are
a method of preventing 'denial of service' attacts against a computer
as well as other hacking attempts.
Nick solved this iptables problem by 'zapping' the iptables configuration.
In other words the configuration has not been properly re-organised, it
has simply been disabled.
Here is the email from Nick Reddel about how the problem was solved
The prob was that james had the wrong java_home (had to put it directly in
phoenix.sh, cos environment vars don´t stick when running from a
non'shell environment)
and then tomcat wouldn´t start. changed k20tomcat4 to s20tomcat4 in
/etc/rc3.d (this may be an issue for a later full reboot)(probably not)
ummm...and i seem to have zapped the boot logger (which was logging to
boot.log)...cos i deleted boot.log to clear it, then touched it, but it
don't like me.
alas
but all is well on the system front
yes, now I'll read the boot.log manual
n
Tomcat is still not 'joined' to the Apache Web Server but the solution
to the problem my simply be that Tomcat needs to be restarted, which is
something which I never did after installing the 'mod_jk' module.
* may 10-14
FA
* may 15,2003
Fixed the checkhotmail cgi script on the US server.
Set the US Redhat Server to Barcelona Time, since the majority of users
are in Barcelona. Fixed the 'add-comment' scripts. Had a short nightmare
trying to start apache with the script in /etc/rc.d/init.d/httpd
Dont do this because it tells you that you are using 1.3 directives and
it wont start the server. Use /usr/sbin/apachectl instead
I established a reasonable 'error-document' for the apache server on the
American Host. These error documents now point back to the main index page
for Ella Associates, that is 'resources.html'.
* may 16, 2003
The 'public_html' directories for each of the users home directories need
to be 'writable' by the 'other' user. In other words they need to have
chmod o+w public_html
done to them, if any comment script is to work in those directories.
I stopped access to the 'guest' ftp accounts on the Barcelona Debian Linux
Server, in order to stop people from accidentally using that Server
instead of the new American RedHat Linux Server. I fixed the script which
sends and email from the 'e-learning' home page (the 'netcat' utility
needed to be installed on the American Server). I updated the
'resources.html' file to provide links to the Ella Associates contributors
home web-pages on the American Server, as well as links to the
'grape-vine' page.
* may 17, 2003, saturday
Some webdav familiarization. Tomcat investigation. The Tomcat examples
are not installed correctly. But the Servlet examples are working.
* may 18, 2003, Sunday
Some Tomcat reading.
* may 19, 2003, Monday
Still on Tomcat. Am going to copy over the '.bashrc' file for the root
user from the Barcelona Debian Server. Also, I should add 'aliases' for
all the important 'log' files, such as Apache, Tomcat, James, etc etc.
This will mean that I wont have to remember the directory locations for
all the log files. I could also do this for the 'home' directories for
the various applications. Yep.
I have created some of the aliases mentioned above and put them in the
file /etc/bashrc So they should be available to all users, rather than
just the root user, as would have been the case if I had put them in
/root/.bashrc
In order to see all the available aliases type 'alias'
The aliases are in the general form
applicationName.functionName
for example
tomcat.log will display the tomcat log file
The lynx help files on the American Server need to be fixed.
Tomcat is successfully serving JSP files to the localhost as well.
For the Tomcat Apache problem, the permission of the mod_jk.so file
may need to be changed. NO, probably not the problem.
Or the the actual mod_jk.so binary need to
be changed to an SSL enabled version (since Apache is running with
SSL).
I have tried this, but didn't work
Or the 'autoconf' mode for tomcat should be used.
Or the mod_jk.conf file is mounting stuff incorrectly.
Or the tomcat 'server.xml' file may need to be changed in some way
Probably not since it already has the 'ajp13' connector defined
The only 'log' file which appears to display any kind of error message
is the apache 'error_log' which displays a very cryptic error message
When I make a request to a Servlet or JSP using the mod_jk 'mapping' into
the normal Apache 'document space', the apache error log records the
following
[Mon May 19 14:43:18 2003] [notice] child pid 14393 exit signal Segmentation fault (11)
[Mon May 19 14:43:19 2003] [notice] child pid 16725 exit signal Segmentation fault (11)
[Mon May 19 14:43:19 2003] [notice] child pid 14403 exit signal Segmentation fault (11)
The mod_jk log file does not appear to contain any error messages.
The URL below is a pretty helpful article about this mod_jk/ apache problem
which I am having. It suggests that a connector needs to be added to the
Tomcat server.xml file. But that connector is already there.
http://www.experts-exchange.com/Web/Web_Servers/Apache/Q_20411439.html#7553532
This article also at some point recommends the solution which was eventually
used at Ella Associates: recompiling the mod_jk.so file
* May 20, 2003
* May 21, 2003, Wednesday
Pottering
* May 22, 2003, Thursday
A problem with the American WebServer. Pings to www.ella-associates.org
work. Telnet www.ella-associates.org 80
This may be a problem with the Telefonica DNS Server. The Apache Web Server
located on the American RedHat Server appears to be functioning normally
and requests that are made from within the Server itself, using a 'Putty'
command session, using the command
lynx localhost
This command displays a textual version of the Default Web Page for
Ella Associates. Also, when the Default Web Page is requested using
the I.P. Address of the American Redhat Server, the correct web page is
displayed.
AT 4:00 PM the Web Server Problem is still occuring.
At the moment there is some 'default' setting for the VIM text editor on
the American Server which auto-matically wraps or folds the text lines once
they exceed a certain number of characters, which number is approximately
about 80.
The difficulty involving the representation of certain special characters
on the American Server, through the medium of a 'Putty' login or command
session, currently continues.
In the pueblo of Ametlla de Mar, a festival of music is being organised to
occur on the 15th of August 2003. A number of different musical groups will
be playing. To provide information about this event a website will be
established which may contain information about the Bands, including
examples of their music as well as other details about the event.
I will be contributing towards this web-site.
The Hub is a newsletter orientated towards the locality of 'Baix Ebre'
which, I believe, is the region in which the town of Ametlla lies. The Hub
appears to be written mainly for an English-speaking audience, although it
is possible that it is also translated into Spanish or Catalan.
Currently it seems that the newsletter is to be published only on the
Internet after a period in which it was also distributed in a 'hard-copy'
form. This periodical probably should be notified about the Festival of
Music which will occur on the 15th of August, 2003.
The web-site which is located at http://www.ella-associates.org and
http://www.ella-associates.org/alexis-info/docs/resources.html is currently
hosted in the United States of America. It would seem useful to mirror this
site onto the Debian Linux
Note:
N.Reddel has solved the Tomcat/ Apache integration problem by
recompiling the 'mod_jk.so' file on the American Server. Once the file
has been manually compiled it is 4 times larger than the file which is
provided on the www.apache.org site, in the release directory of the
'mod_jk' module.
It is still required to set up 'Bugzilla' and 'IssueZilla' and 'Cocoon'
as well as to move the 'crontab' file over from the Barcelona Server to the
American Server in order to enable the periodic backups of important
directories and files.
The correct format for a 'multi-lingual' web page is as follows
pagename.pagetype.languagecode
For example, a page in Spanish should be called
index.html.es
Microsoft Internet Explorer also seems to understand a format which is
exemplified by the resource name index.es.html but from my somewhat limited
knowledge this resource naming format is not in accordance with the
'standard', whatever that may actually be. This standard is probably an RFC
1am in the Morning: the DNS problem (assuming that it is a DNS problem) is
still occuring. From an Ametlla De Mar laptop computer which is running
Microsoft Windows 2000, doing a
nslookup www.ella-associates.org
returns the following information
D:\mjb\ella-site-backup\utils>nslookup www.ella-associates.org
DNS request timed out.
timeout was 2 seconds.
Can't find server name for address 80.58.0.33: Timed out
Server: 97.Red-80-58-32.pooles.rima-tde.net
Address: 80.58.32.97
Name: ella-associates.org
Address: 63.105.73.195
Aliases: www.ella-associates.org
From this result, we can possibly deduce that once of the Spanish
'Telefonica' DNS Servers is 'down' or not functioning at the moment, unless
it is a configuration problem on the Barcelona Debian Linux Server.
It shold be remembered that at the moment the Barcelona located Debian
Linux Server is the 'authoritative' Domain Name Server for the Domain
ella-associates.org.
To simplify, when requests are currently made (1am today) to an Internet
Address such as http://www.ella-associates.org/alexis-info/ an error
message is displayed by the Web Browser such as 'the document contains no
data'. This error message, from my experience, is generally returned by
some types of Browser when there does not exist any Serving Application
listening on the port required. For example, if there is no Web Server
application 'listening' on the port 80 when a browser makes a request
for an Internet Resource then the browser (notably Netscape), may return
the message 'the document contains no data'.
On the other hand, if a request is made to a URL such as
http://ella-associates.org/alexis-info/ then the correct web page is
returned. From my limited and rather superficial knowledge of the DNS
system I would guess that this problem is caused by a configuration problem
on one of our Linux Servers, rather than by a problem with the Telefonica
DNs Server. Please note that 'Telefonica' is the main Spanish PSTN Service
provider in Spain, equivalent to digamos, Telecom in Australia.
In the opinion of N.Reddel (who seems to know significantly more than
myself about these sorts of subjects), this IS a Telefonica problem
This problem is also 'killing' the comment scripts on the American Linux
Server because the HTML Form submit button appears to point to
'www.ella-associates.org' to some subdirectory of this domain name.
* May 23, 2003, Friday
The problem with the DNS seems to have fixed itself. For example, when
a request is now made to a resource such as
http://www.ella-associates.org/alexis-info/ The correct page is
returned instead of the 'document contains no data' message as
before.
However, when doing an 'nslookup' from an Ametlla de Mar laptop
as before, the same sort of results appear to be returned which
possibly indicates that the Telefonica Server is still no
available. For example, the following is an extraction from a console
session on a Windows 2000 computer.
D:\mjb\ella-site-backup\utils>nslookup www.ella-associates.org
DNS request timed out.
timeout was 2 seconds.
Can't find server name for address 80.58.0.33: Timed out
Server: 97.Red-80-58-32.pooles.rima-tde.net
Address: 80.58.32.97
Name: ella-associates.org
Address: 63.105.73.195
Aliases: www.ella-associates.org
The 'linkdoc2html-forum.sh' script does not currently write a link to
the Alexis Documentation home page. This was because originally the
Documentation Home Page was a link document and for this reason did not
need a link to itself.
Also the scripts don't handle resource names such as index.txt.en very
well.
* May 24, 2003, Saturday
Installed the Cocoon document server and renderer on the Ella-Associates
Server. And now for 'bugzilla' http://www.bugzilla.org/
* May 25, 2003, Sunday
Trying to install Bugzilla. This is not entirely straight forward since
a large number of perl module dependencies are necessary.
administrator account for bugzilla:
user: [email protected]
The database user for bugzilla is
name bugs
Bugzilla is mainly working but cant send email notifications of
Bugs because it requires Sendmail to be properly configured. However
the James mail server is being used instead of sendmail although
it is possible that the two might be able to co-exist
Copy a file called 'slide.def' from nicks E drive. Grant mysql permission
only from the Ametlla IP address. Look at what is currently on the
Barcelona Debian Server.
Installing the FAQ-o-Matic
Temporary Password
There is a problem here in that in order to properly install the
faq-o-matic it is necessary to have a 'mailer' properly installed on
the host machine (something like sendmail for instance) which accepts the
-s subject switch. However, our mailer on the US Server is James which does
not seem to have a command line mailing interface.
It may be possible to get around this either by directly editing
the faqomatic configuration files, or by using my dodge 'bashmail' script
and modifying it to accept the -s switch. I do not know if this 'hack' will
work. In general the installation process seems unnecessarily complicated.
Did the initial installation process for the apache 'slide' webdav server.
I still need to enable the JDBC data store facility and to properly edit
the configuration files
* May 26, 2003, Monday
Doing a little bit of pottering around with the Ametlla De Mar music
festival web-site. This should probably be primarily in Spanish.
Still having a serious problem with 'character sets' on the American Server
For example when I type 'a acute', in the VIM editor, this is not stored
as 'a acute' but as something like 'Capital A tilde, Ordenal O [like a
degree sign]'. I still dont know if this is the 'putty' application or the
character set used on the American Server.
This is adversely affecting my ability to create a spanish version of the
L'Ametlla de Mar Music Festival web pages, since I am unable to get
the accented characters to appear decently.
Trouble shooting ideas for the American Server character set problem:
I could write the file on my local Microsoft Windows
2000laptop and ftp the file to the American Server
Interestingly, the 'vea esta pagina en blah blah' link which occurs
at the top of the generated HTML pages, occurs correctly with
accented characters, which is because I believe that I cut and
paste this text from a Google Automatic Translation page.
I could temporarly enable telnet, then telnet into the american
server and create a file with accented characters, if these are not
'mangled' then the prob is probably putty, I suppose
* May 27, 2003, Tuesday
Looking at slide JDBC stores
MySQL Data files for slide are at
c:\mysql\data\slide\ On Nicks laptop.
The slide.def file was at
e:\slide\server\conf\slide\slide.def
According to Nick, the process in getting the slide server to use
a JDBC store is just; copy the mysql tables from the location
mentioned immediately above to the American Server. Then change
the ownership of the files and permissions to whoever needs them,
this step I am not entirely clear about.
Then put the slide.def file in its correct location on the
american server and voila! con suerte
* Wed
Nope
* May 29, 2003, Thursday
Understanding Cocoon, and trying to fix slide.
Understood sitemaps and generators and serializers. The Cocoon
samples and docs are very good.
* May 30, 2003, Friday
A question about getting slide to work with tomcat and mysql
http://faq.globalvision.com.au/fom-serve/cache/88.html
The above URL contains very good step by step instructions for
setting up slide with mysql, tomcat and creating a web folder to
see the slide directory.
The Apache slide documentation is generally fairly poor and tends
to assume that you are a developer actually working on the code.
Or something
When Tomcat is stopped and started it will often create a
directory called 'contentstore' in the current directory, that is,
the directory from which tomcat is being started and stopped.
This is caused by the 'slide' webapplication which uses that directory
to store webdav content.
Slide then uses this newly created content store directory to
store all uploaded files, etc, which is bad
Sometimes, in addition, Tomcat needs to be started twice in order
for the 'slide' application to be accessible.
These problems should be solvable by editing the files
/var/tomcat4/webapps/slide/Domain.xml and
/var/tomcat4/webapps/slide/WEB-INF/site.xml
These are the only two configuration files which affect the slide
web application. The configuration files in the 'slide' installation
directory appear to have no effect, logically enough
The following error generated when stopping tomcat (with say, the
alias tomcat.stop) can just mean that tomcat is not currently
running.
Catalina.stop: java.net.ConnectException: Connection refused
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:305)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:171)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:158)
at java.net.Socket.connect(Socket.java:426)
at java.net.Socket.connect(Socket.java:376)
at java.net.Socket.<init>(Socket.java:291)
at java.net.Socket.<init>(Socket.java:119)
at org.apache.catalina.startup.Catalina.stop(Unknown Source)
at org.apache.catalina.startup.Catalina.execute(Unknown Source)
at org.apache.catalina.startup.Catalina.process(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.apache.catalina.startup.Bootstrap.main(Unknown Source)
The contentstore directory appears to be created the first
time that the slide default page, or url is accessed, by only the first
time after tomcat has been started.
There is probably a bug in Explorer 5.0 and web folders, it doesn't appear
to 'authenticate'. At the bottom of the Domain.xml file is a line wich
allows guest privileges this should be uncommented until authentication is
sorted out.
Files which are copied into the contentstore/files directory after
slide has started, do not seem to appear in the directory listings.
Files which are already in the contentstore/files dir when slide
starts, are ALSO not 'seen' or recognised. Dont know why
* may 31, 2003, saturday
Washed clothes, tidied house.
7:30 pm, a little work
The main configuration file for jakarta-slide running as a
Tomcat webapplication is
/var/tomcat4/webapps/slide/Domain.xml the other is
/var/tomcat4/webapps/slide/WEB-INF/web.xml
The Domain.xml file seems to contains a sort of 'amalgam' of three
configuration files which are used when slide is running as a
'standalone' application.
+++++Things to try with the Jakarta Slide Server. I am currently still
having problems with the material (ie files) in the web-folder
'disappearing' when tomcat is stopped and started.
remove the 'slide-webdav' webapplication which I installed in a
moment of ignorance and blind hope
NOPE, doesn't seem to have improved anything
Do a complete re-install of slide, making sure to preserve Nicks
'slide.def' file. There doesn't seem much point to this because I
haven't modified many files and have been careful to preserve
the default configuration files
Set the 'resetsomething' parameter to true in the Domain.xml file
then restart tomcat.
Switch to a JDBC store using nicks slide.def file and inserting the
relavent bit in Domain.xml Also examine the 'permissions' section of
the 'Domain.xml' file and remove or comment out the bits which will
confict with the permissions or ACL information in the JDBC database
store. By doing this it should be possible to determine if slide is
actually getting file and permission information from the Database or
not. This process of going one step further in using JDBC stores may
seem to be unnecessarily complicating things but I have sort of run
out of ideas for the 'simple' file store.
Also, if permission information IS being drawn from the database, then
try emptying the 'objects' table,
* 1 june 2003, sunday
2:15pm Some work
I need to get a mysql client for windows so that I will be able to
check that the slide database is accessible
Or the alternative is to 'putty' in to the Barca Server and check
from there.
The requests to the Mysql server on the United States Server
are causing the 'iptables' program to block all requests to the
server. If this occurs the solution is to putty into the Barcelona
Debian Linux Server and then use the 'ssh' program to get a
remote console on the american server. You will need to know
the root password for the American Server. Once you have a
'console' then issue the command
iptables --flush
This will remove the rules from the IP tables which block requests to
the Server from being replied to.
Instead of trying to learn about the 'iptables' program, I am going to
do a complete fresh install of slide and attempt to get it working in
standalone mode, with a simple file store.
In order to do this, I am going to use ... (?)
* 2 june 2003, monday
Electricity to the house in Ametlla not working until 4pm approx.
Fuse was the problem. The fuse was broken in a way that was not usual. That is
the fuse was not 'blown' in the normal way, but was broken in some other way
which caused sparks to come out of the fuse box
* 3 june 2003, tuesday
looking at iptables still. I think the problem was not really an
iptables problem. The problem which I am referring to is the
American Server blocking attempts to connect to it from the Laptop
computers located in Ametlla de Mar. I believe that this was actually
caused by a program called 'portsentry', although I am still not
entirely sure. I have temporarily stopped this portsentry program
until I have time to understand it more fully. I believe that the
portsentry program dynamically adds entries to the 'filter' table
of iptables. It does this in order to block any ip addresses which
it considers to be 'attacking' the American Server. However,
any request to a port such as 3306 (the MySql port) was also being
regarded as 'aggresive' or hostile.
* 4 june 2003, Wednesday
I am doing some testing and documentation of the Alpha version of
Alexis. To this end, there are several related tasks which may be
considered useful.
Write a script which backs up the MySql database in which all the
documents are being stored. With luck I should be able to do this
with an ftp script (if my Microsoft Windows ftp program allows me
to 'shell out' during a scripted session, using something like
ftp>!tar -czvf /mysql/data/dir/
Also, I could write a script on the American Server which 'pushes' the
data onto the Barcelona Debian Server.
Write scripts which convert my current documents into a very simple
kind of DocBook. I could do this by modifying the scripts
plaintext2html-forum.sh and linkdoc2html-forum.sh
Work out how to change the schema and style-sheet for Alexis
Learn a little about the normal usage of Docbook tags, so that I can
use them effectively.
Think about the command line client for slide, so that I can script
additions to the slide repository (MySql Database). Will this
client run on windows?
* 5 june 2003, Thursday
Made a script that turns a plain text file into DocBook XML.
Basically all the script does is mark up 'section headings' which are
all capital lines, mark up paragraphs, which are seperated by
blank lines, mark-up urls which start with http:// or something
similar and other various simple things like this.
* 6 june 2003, Friday
Having a look at the slide command line client. The start-up script
for the client does not work on the American Server, but the
client can be got to run by executing each of the lines within the
script individually. Not sure why the script won''t run. It appears
to be acting under a different command interpreter (ie not sh and
not bash)
* 7 june 2003, Saturday
Some testing of Alexis and documentation.
* 8 june 2003, Sunday
nothing
* 9 June 2003, Monday
Searching for a better Docbook XML stylesheet. Found one with the
XMLmind editor. Thinking about wikis again
* 10 june 2003, Tuesday
Stuff to do: get a Xalan transform for DocBook:yep, get cocoon working with the
slide version of tomcat:sort-of, make the Docbook css stylesheet. using the
various files with the XML mind editor:yep, get a better WebDav client: something
that can be scripted to put a file up on the webdav server.
Below is the link to the Docbook XSL style sheets, to be used with
Saxon, or with xsltproc
http://sourceforge.net/project/showfiles.php?group_id=21935
In the end I was not able to install a scriptable webdav client:
see the document http://www.ella-associates.org/alexis-info/docs/webdav-stuff.txt
for the results of my endevours to find something suitable
I uploaded the new DocBook Style sheet, which appears, from a
very cursory glance, to be working.
* 11 june 2003, Wednesday
Am looking at the cocoon configuration problem.
To start and stop tomcat webapplications without starting and stopping
the server at a user with role 'manager' to the 'tomcat-users.xml' file
then stop and start Tomcat, then access the url
http://ella-associates.org:8080/tomcat-manager/list
And log on with the user name and password which you have just entered
This will provide a basic interface to starting and stopping
webapplications as well as installing new ones.
* 12 june 2003, Thursday
Will work on making a backup of the mysql (slide) database.
Importing all existing documentation into Alexis (slide), and
establishing 'cocoon' transformations. The XSL stylsheets appear to be
'carking it' when, for example, the alexis-faq.xml document becomes
complex. That is, when sections were added to the document each
containing a qandaset, the stylesheets failed to render the document into
HTML. Only the table of contents was produced and nothing else.
* 13 june 2003, Friday,
Need to move the DNS configuration onto the American Server.
* 14 june 2003, Saturday,
Nothing
* 15 june 2003, sunday
Deal with Roberts email problem. Do the database backup. Deal with the
DNS problem. Wrote a backup script for the mysql slide database, and
scheduled it for every 2 hours, since the script runs extremely quickly
and the database is not big. About 80 unique versions will be kept
* 16 june 2003, monday
Local Server has gone down. Working on DNS Migration.
* 17 june 2003, tuesday
Achieved DNS migration. The keys to using DNS are doing 'rndc reload'
when you change the configuration files and do 'rndc flush' to get
rid of the DNS 'cache'. There are only two important configuration files
namely 'named.conf' and the configuration file for each 'zone' which
you control. Actually, this isn't quite true but the other configuration
files you can pretty much leave as they are. To test a DNS configuration
use 'dig' with the '+norec' option which stops dig from going ahead and
doing all the work by itself. so just do something like this
dig +norec hostname
There are important things to remember about DNS resolving. Remember
'who' is doing the resolving. That is, which name server is doing the
lookup. If you dont tell 'dig' which nameserver to use or to query,
dig will use the one in the file 'resolv.conf' To test a DNS configuration
use 'dig' with the '+norec' option which stops dig from going ahead and
doing all the work by itself. so just do something like this
dig +norec hostname
There are important things to remember about DNS resolving. Remember
'who' is doing the resolving. That is, which name server is doing the
lookup. If you dont tell 'dig' which nameserver to use or to query,
dig will use the one in the file 'resolv.conf'. The thing is, do
nameservers resolve anthing or do they just refer you to other name
servers if they dont know the answer. I dont know.
I am still slightly confused. I think it is that the first name server which
you make a request to actually does all the hard work of querying other
name servers.
* 18 june 2003, wednesday
stuff
* 19 june 2003, Thursday
stuff
* 20 june 2003, Friday
Emailed Erik seth about programming. Erikseth is going to try to hellp us in the
Alexis programming effort
* 21 june 2003, Saturday
Worked on the Posidonia web-site. This is available currently at the URL
http://poseidonia.ella-associates.org/
* 22-26 june 2003
Not much
* 27 june 2003, Thursday
Poseidonia web site
* 28-31 june 2003,
Posidonia web site. Made the web-site into a wiki so that it can be contributed to
by anyone. This may be useful in eliminating language errors in the web-pages. This
is achieved by a script called text2html-collab.sh This script essentially creates a
'wiki' system but it is different from the normal wiki systems which are available
on the net in that the 'editing' box is on the same page as the web-page and also
the wiki 'language' is simpler than other wiki languages. This Posidonia Wiki
has problems because the Browser does not refresh the web-page so that it looks
to the web-visitor as if they havent done anything at all.
* 2-6 june 2003
Made indexes for the Alexis documentation using cgi transforms of the
slide web-diretories. These transforms point the links to the required
place which is http://www.ella-associates.org/cocoon/livedocs/ etc The cgi
scripts are currently located in the /var/www/alexis-info/ directory. I
am not entirely sure why I put them there. I then reduced the 'indexing'
scripts to one script which does the work for all the directories and I
have included aliases in the apache configuration file so that if new
directories are added to the documentation they will be automatically
'rendered' as they should
I also rearranged the Alexis documentation into directories such as
programming, alexis-overview, alexis-help, e-learning, business, general.
The first 5 of these were suggested by nick but I added the last for the things
which didn't fit into any of these.
Enhanced and generalized the Posidonia web-site wiki.
* mid june 2003
Helping Marc Vila i Leon with various posters and flyers for posidonia
* 15 june 2003
Helping Erik-seth with his introduction to programming
Yesterday it was reported by Juanma (Joan Manual) a resident of L'Ametlla de Mar that the Posidonia
music festival website was not available when he tried to access it. I will
investigate. Yes, all requests to the ella-associates.org web site seem to be failing
now. Not good. Could this be related to the Cocoon aliases which I created.
Yes I think so, because I didn't put the aliases in the virtual host. Because the
American RedHat Apache web-server now has multiple virtual hosts, many directives
should be put in between the <virtualhost> tags. To do this you can use the
alias which I have defined; namely type apache.conf and go to near the end of
the file.
directories.
Yes this was the problem. I really need some kind of daemon script to alert me
when one of the web-sites goes down. Strangely when I removed the aliases
requests to http://poseidonia.ella-associates.org/ still did not work for a
little while.
The source tree for Alexis is at
\\Lap\e\cvs\ella.org\alexis\src
on nicks computer
* 16 june 2003
Vim is still 'auto-wrapping' lines on some occasions. This is despite my
attempting to fix the problem many times. The solution may just be to set the
textwidth to 0.
One of my tasks is to set up cvs on the American Redhat Server. This involves
making sure that the text files have the correct line endings (since I ftp'd
them from my Almetlla laptop as a .tar.gz binary file)
informing cvs what are binary files (see april 10), importing the code tree, setting up cvs 'modules'
and restricting access to the 'pserver'
* 17 july 2003
Teaching Eriketh about java programming and XML. We covered baic concept about classes
Today the American Redhat Server stopped functioning. The server was unavailable
for about 24 hours. Eventually when the problem was solved the technical support
man at the server hosting company informed us that the problem was due to
'iptables' This is a 'firewall' program which is designed to improve the
security of the American server. Ryan Flowers the technical support man
indicated in his email that he has now disabled this program. He suggests in his
email that we can see what he has done by running the 'setup' command.
Nick is keen to put in place a 'physical reboot' device which would allow us to
start the Redhat box if something goes drastically wrong.
When the Linux Box restarted we had difficulty starting the James Mail Server.
Nicks method of doing this is to use the 'telinit' command to change the
'run-levels' on the computer. Apparently the point of this is to start the
Mail Server as a true 'service' rather than as a process that is owned by a
particular user. Specifically Nick executes
telinit 4
telinit 3
Although it may be the other way around. The problem with this was that the
sendmail daemon also has scripts in the /etc/rc.d directories and so when
the commands above were executed the 'sendmail' daemon actually 'grabbed' the
relevant email ports (eg: 25) before the James mail server. This meant that
James was unable to start properly. This occured because sendmail is written
in C whereas James is written in Java and therefore the sendmail daemon
carries out its work slighly faster than the James Daemon
* 18 july 2003
teaching Erikseth about Java programming. Nick was keen for us to write a
'document change notification' program, but in the end we did not have time
The mentioned program would send an email to somebody when a document was
changed through Alexis. The idea behind this is to notify editors of documents
when a document they have been working on has been modified by somebody else.
The program should be reasonable simple to write since in doesn't need to
be integrated with the rest of the Alexis code. It would need the following
components: jdbc to access the slide database and decide which documents have
changed; javamail section to send an email to those people who have an interest
in the specified documents. Nick would like all the database calls to be in
one Java class in case the database schema changes.
Originally Nick wanted this Document-change-notification element to be written
as a JSP running in the Tomcat Servlet container. This would allow the program
to have a web-interface or perhaps just some kind of web 'activation' component.
* 19 july 2003
I am in Barcelona. My keyboard on the laptop is now not working very well. This is
due to the fact that during the journey from L'Ametlla de Mar to Barcelona the
various bits of 'stuff' (dirt) which is floating around under the keyboard have
moved around and lodged in different places. In particular the s and x keys
are particularly bad. This can normally be rectified by blowing forcefully in
the direction of the problematic key with your mouth close to the keyboard.
Did not do much work. Saw paul andersen in Barcelona. Keyboard problems fixed by
blowing method
* 20 july 2003
Need to complete cvs setup. Do we really need backups every two hours for the
slide mysql database? Changed slide database backups from every 2 hours to
every 6 hours The Local Server (based in calle Blanqueria) is now permanently
switched off.
Went to picnic with Robert
* 21 july 2003
Need to look at why Cocoon is not dealing with European characters properly.
This was noticed by Nick Reddel. Also, need to edit Nicks business proposal
Had a meeting with robert about Alexis stuff. Wrote to American Server people
about putting in a 'physical reboot' function which would allow us to actually
turn the American Redhat Linux Server on and off and therefore debug problems
with the Server if they arise
The name of the main technical support guy on the amhosting server is
Ryan Flowers and his email address is [email protected]
* 22 july 2003
The American Server went down again today. The American Server company
claim that it is our software which is the problem. The James mail
server has not started properly
* 23 july 2003
To start james do the following
Make sure that James is stopped by doing
/usr/bin/james-2.1.1/bin/phoenix.sh stop
Kill all sendmail, poprelayd and maybe 'master' processes by doing
ps -el to find out the process numbers and then do
kill number to get rid of each process
Now do: telinit 4 and then telinit 3 in that order to start james as a service.
Now check that james is running by doing
james.manage or telnet localhost 4555 to connect to the 'manager' application
Below is a line which could be used to automatically kill sendmail type of things automatically
for n in $(ps -el | awk -F" " '/sendmail/{print $4}'); do kill $n; done
* 24 july 2003
I created a script at /var/www/utils/start-james.sh which does all the above mentioned processes
of stopping james and changing runlevels, etc. We have been experiencing ongoing
problems with the American Redhat server. On one ocasion today Nick issued the
command iptables --flush in order to attempt to allow access to the
Apache Slide Server which carries out most of its transactions on port 8081, but this
command actually killed the Server, or at least the Server died simultaneously with this
action.
However, the 'physical reboot' feature has now been implemented on the American Amhosting
Redhat Server and by telneting to a particular port of a particular ip address it is
possible to restart the Server. This was used to over come the problem
iptables -A INPUT --protocol tcp --sport 1024: --dport 8079:8084 -j ACCEPT
The line above is, according to Nick, the correct line to flush the 'iptables' on the
Server and thus allow necessary access to the Server.
And here are the details for accessing the reboot feature
Email from Ryan Flowers of Amhosting, Reno America
I added this and billed your card for the $59.90 required for it
The reboot switch is internet accessible and turns off/on the power switch
To login, telnet to 63.105.72.10, username nick, password . This will add 9.95/mo to your account
According to Nick this works without a problem
The afternoon and Evening:
I worked with Robert Grundy looking at rival xml editors to Alexis and talking about what
features make Alexis important. Robert made a number of, in my opinion, very valid comments
about the Alexis interface.
That all 'functions' or actions should be available from the menus. For example the list of
valid tags for insertion into an XML doc should be available from a menu.
That it is tricky to get the cursor in between an empty tag set. Really the 'triangles' of the
xml tag markers should not be 'sensitised'. That is, if you click on the triangles then the
cursor should be put in between the tags rather than selecting the set of tags. Also: it would
be nice if the mouse pointer would change its shape or symbol in the sensitised areas. That is,
if, by clicking, you are going to select the set of tags then the mouse pointer should be a
certain symbol but if by clicking you are going to put the cursor between the tags then the
mouse pointer should be a different symbol. This is in accordance with the Xmetal editor
That the XMetal editor starts up in less than a second whereas Alexis takes about half a minute.
That when you gain a lock on a file by double clicking on the file name and then editing that
document, then really you have 'checked out' that document, and for that reason you should have
the option of 'checking back in' the document manually, rather than just be saving it. This
point may be dubious. However the menu item for 'check in' is disabled when the document has
been automatically checked out.
That if you are not connected when you try to login in using Alexis then the operation system
connect function (window) should be triggered. This may be tricky to implement.
That you should be able to edit the schema document, if you have sufficient authority.
End Of ROBERTS SUGGESTIONS
Nick has suggested: move cgi scripts to Java, write the email notification thingy,
work with Robert. Also I should stop James being used as an open Relay for spam. This is
probably super super important but I am going to bed. Email sent from james account to the
outside world DO NOT arrive. not good
The Server appears to have stablilised.
* 25 july 2003
mail server still down to stop it being used as an open relay for spam.
I could write a script which looks at the james mail spool (outgoing queued mail) and
if it becomes very large this script could send an email to alert somebody that the
mail server is being spammed.
I wrote this script and called it /var/www/utils/check-spam.sh . It can be run from anywhere
with check-spam.sh but the point of it is for it to be scheduled using crontab to
automatically send an email if the server is being spammed. I have scheduled it in this
way. Currently it is every five minutes but I will reduce this soon
Also a script which checks the webservers
This James Mail Server spamming problem may be fixed but I am not entirely sure about
this. I changed the configuration by adding the extra authentication check that
the user should have the correct email address.
Also I wrote a lot of aliases for the james server to make it easier to see what is
going on in the log files. Type alias | grep james to see these aliases
From looking at the logs, somebody seems to have the 'webmaster' password. Although
they might have others.
Found a thing called jvmail which is a java console smtp client. Could solve all
my email problems. I couldn't get the latest version of JVMail to work, but an
older version does.:q
Here is the command line which seems to do the trick
java JVMail -nmx1.mail.yahoo.com [email protected] [email protected] -s"test jvmail 1.1" -b"Message Body"
* 26 july 2003
Early this morning I was investigation simple command line smtp clients. and I found
certain things but I did not have great success.
For example
jvmail is a java smtp email client which appears from the website to have all the
required features. But I couldnt get the latest version to work, since it uses a
jar file and all my attempts to run the jar file using things such as
java -jar JVMail.jar all failed. Also it appears that the 'manifest file' in the
jar did not contain the necessary configuration lines
There is also a c program called something like SMTPclient on the internet. But it
doesn't contain any compiling instructions. I should be able to overcome this but it
was a bit intimidating
Also there would be various perl scripts progs to do this but I cant seem to get
my perl ::net packages working.
All in all the conclusion was to use the JavaMail api to do this myself. This would
make sense because I also have to write the 'alexis notification' program which would
have to use the javamail api.
The James Mail Server does not appear to be being spammed anymore. The spammer is still
attempting to connect from 218.70.141.188 but cannot because I changed the webmaster
user account password using the James 'remote manager'
* 27 july 2003
The James Mail Server has gone down again. I am investigating the cause.
Server restarted. I dont know why it stopped. Maybe sendmail again
* 28 -31 july 2003
pottering around with various things, not much really.
saw michael the cellist about a room in his flat.
* 1 august 2003
Probably should start on the email notification script.
The ftpserver on the American Server does not appear to be running
Beginning to look at the 'javamail' api. I think that I am going to have to install
this. See the file http://www.ella-associates.org/alexis-info/docs/javamail-stuff.html
* 2 august 2003
Understanding JavaMail and the Slide Webdav java apis
I wrote a program at http://www.ella-associates.org/utils/TestJavaMail.java which successfully
uses the javamail application programming interface (api) to send an email using
normal smtp authentication
work time: 2 hours [3]
* 3 august 2003
Wrote a set of java programs GetDavFile.java PutDavFile.java and also vimdav.sh
The vimdav.sh script allows you to edit a file on the webdav server using the 'vim'
text editor. This script uses the two previous java programs. These can all be found
at http://www.ella-associates.org/utils/
vimdav.sh /slide/files/documents/alexis_docs/the-sand-box.xml
The script has been invoked successfully with the invokation above. I am really quite
proud of this since it makes making small changes to an xml document much simpler
work time 3 hours [6]
* 4 august 2003
Installed catdoc which seems to work well for converting Microsoft word docs to plain
text, and also installed word2x which is supposed to do the same thing. But not for me
Started to transform Roberts Glossary using the following command line
catdoc robert-glossary.doc | expand | \
sed -e "s/^[ ]*//g" -e "s/[ ]*$//g" | \
sed "/[ ]\{2,\}[^ ]/ s#\(.*\)[ ]\{2,\}\(.*\)#<a>\1</a><b>\2</b>#g" | less
Had a 'business' meeting with Nick Reddel, Marcus and some girl. This meeting in reality
was actually a series of beers and carajillos.
The line below is nicks solution to the portsentry application adding lines to the
iptables on the RedHat American Server which block us from using ports such as the webdav port
I am somewhat dubious about this solution. Firstly it doesn't address the problem of portsentry
Secondly if there is a reject rule in the iptables database already, is an accept rule going to
do any good
iptables -A INPUT --protocol tcp --sport 1024: --dport 8079:8084 -j ACCEPT
However, this line above did just work for me tonight. I executed it and now I am able to
get webdav access again. I wonder if my javawebdav scripts will work now. YES YES, my webdav java
programs now work on Microsoft Windows as well as Redhat Linux. To make this happen I had to
copy the necessary jar files to my MS Windows computer and then reference them with the
classpath variable. I did not need to change the jar files in any way
Here is what the windows invocation looked like.
D:\davscript>java -classpath d:\davscript\commons-httpclient.jar;.;d:\davscript\webdav.jar GetDavFil
e files
work time 3 hours [9]
* 5 august 2003
Had another business meeting with N Reddel and Marcus in the Galician bar which is near the Arc de
Triomf. In this meeting Nick suggested the following work for me.
finish the email notification of document changes withing Alexis.
Mock up a structure for the Alexis (product) website using docbook XML. This would require an
enumeration of the various categories and pages which would be necessary on the website.
some suggested pages were: a comparison of Alexis with various other XML editors. A technical
resources page, what ever that means, support, javadoc, general documentation.
Look at Nicks mock up of the Alexis gui interface and make comments.
Work on the server extensions to slide and webdav. This would be a major programming task but
would, in my opinion be much less difficult than learning about the Netbeans architecture
Tasks which I thought of myself were to do some general research on lexicographical companies in
Spain. Also had the idea of doing some general 'training' in these companies which would essentially
involve suggesting alternatives to Microsoft word
As discussed, thee distinguishing feature of Alexis in document management, including change tracking
and versioning etc. Also a messaging system and a 'plug-in' schema semantic help
[10]
* 6 august 2003
Nick has suggested that we need to solve a problem with the Cocoon Server in that it is not performing
transforms on documents which have European accented characters in them. It is possible according to
my small knowledge of this topic that the accented characters need to be 'entitized'.
I have been able to insert some accented characters into some documents and been able to view
transformations of those documents. I had a dicussion with Nick Reddel about my continued employment
with Ella Associates. The following things were discussed:
my disagreement about some of the processes through which Alexis is being developed. In particular
the idea that my putting intensive effort into Alexis in the next 3-4 months we can in some way
hit a 'jackpot'. My preference is for a sustainable development process, building the application
up in useful stages, each of which 'stage' would represent a useable product which could be shown
to companies. It is possibly to late to suggest this since Alexis has not been begun in this way
an therefor its 'architecture' probably does not really lend itself to this. Nicks argument is that
he does not have sufficient time or money to develope the application in this way. This argument
is based on the idea that Alexis has a limited 'shelf life', or that if the application is not
got out in a reasonable amount of time (within 3 months seems to be a very approximate figure), then
there will be no market for the application. I disagree with this. I believe Alexis has a much longer
useful working life than this.
Nick mentioned the fact that he needs to be given estimated times for when certain tasks will be completed by
me.
* 7 August 2003
The file containing the problem accented characters was locked (in webdav) by nick. I am developing
a webdav debugging java program ( http://www.ella-associates.org/utils/DebugDavFile.java ) which is
designed to give information about WebDav resources located on the Apache Slide Server
work time 1 hour [11]
* 8 august 2003
reformatting shell scripts to java in vim
% s/elif/} else if/gc
% s/;[ ]*then[ ]*$/ {/gc
Format boolean variable comparisons
% s/if \[[ ]*"\$\(b[^"]\+\)"[ ]*\(!\?\)=[ ]*"true"[ ]*\];\?/if (\2\1)/gc
Turn null script parameter checks into argument length tests
:% s/if \[[ ]*"\$\([0-9]\+\)"[ ]*=[ ]*""[ ]*\];\?/if (args.length == \[\*\*\1 - 1\*\*\])/gc
Turn non null parameter tests into java comparisons
:% s/if \[[ ]*"\$\([0-9]\+\)"[ ]*=[ ]*"\([^$"]*\)"[ ]*\];\?/if (args[\1].equalsIgnoreCase("\2"))/gc
Format string variable comparisons handling negative tests as well
:% s/if \[[ ]*"\$\([^"]\+\)"[ ]*\(!\)\?=[ ]*"\([^$"]*\)"[ ]*\];\?/if (\2\1.equalsIgnoreCase("\3"))/gc
And handle comparisons of two string variables
:% s/if \[[ ]*"\$\([^"]\+\)"[ ]*\(!\)\?=[ ]*"$\([^"]\+\)"[ ]*\];\?/if (\2\1.equalsIgnoreCase(\3))/gc
End of if statements to c/java curly braces
:% s/\<fi\>/} //-- if/gc
Add a curly brace to isolated else statements
:% s/\<else\>[ ]*$/} else {/gc
Make thens into curly braces
:% s/\<then\>/{/gc
Make embedded variables into string concatenations
:g/echo[ ]\+"/ s/\$\([^0-9][a-zA-Z-]*\)/" + \1 + "/gc
Make echo statements into string buffer append statements
:% s/\<echo[ ]\+\(["']\)\(.*\)\1[ ]*$/sbResult.append("\2 \\n");/gc
Improved to the script http://www.ella-associates.org/utils/DebugDavFile.java
Ideas for the email notification script. Add a transform to the live docs transformation
script which will add a submit button and check boxes to the file names. This will then
point to a cgi script which will register an interest in that document. So we would have
a list of check boxes, one each next to each file name and one text box for email address
and one submit button.
I wrote the user interface for the Notification Thingy. It is at
http://www.ella-associates.org/cocoon/livedocs/
The interface uses a stream transform on the standard Apache Slide Servlet. This
transform provides links to the live versions of various documents within the Alexis
Slide server and also provides a way for users to request automatic notifications of
document changes. There is also a cgi script which does the job of storing the data
and displaying a few messsages. The data which this script stores will then be used
by a Java Program which is scheduled as a 'cron job' to run probably every 3 or 4 hours.
This java program will handle the job of checking if documents have changed and sending
emails if they have
Estimated working time for day: 9 hours [20]
* 9 august 2003
Continuing with alexis change notification thing
Started work: about noon
Got a basic skeleton written for the Java notification program
3pm distracted by Jonathon: drank one canya and a small amount of sangria in the
bar near the locale
got a system which appears to work for email notifications. Need to polish
Worked for a while at night ironing out bugs in the notification system. Set the
notification interval to 12 hours
work time 8 hours [28]
* 10 august 2003
11:30 am
edited named.conf and made a hosts file for monkey rhythm so that the American Server
can serve the website. Also I set up an Apache virtual host so that the Apache Server
will server the domain 'www.monkeyrhythm.com' .
I am now going to look at the Cocoon accented character problem. The
strange codes such as <95> and <92> in the createca document do not seem to
be the problem. The cocoon logs show that a UTFDataFormatException is
being thrown by this document.
fixed the createca document accent problem
Alexis actually uses the iso-8859-1 character set. but has the meta
directive <?xml version="1.0" encoding="UTF-8"?>
At the moment, accented characters entered into Alexis will look alright in
Alexis but will cause cocoon to crash because it is expecting utf-8
accented characters, not iso-8859-1 accented characters
The simplest solution to this is to change the meta directive of the
document to <?xml version="1.0" encoding="ISO-8859-1"?> This needs to be
done for every document which may contain (iso) accented characters The
only way I know how to do this in Alexis is through the 'source editor'. Is
there another way? I suppose this should be the default for all documents
until alexis displays the utf character set properly.
Another solution is to make Alexis display utf properly and to convert our
existing documents to utf This can be done on the server with the command
iconv -f iso-8859-1 -t utf-8 -f newfilename oldfilename
For the createca document I changed the meta directive and it now displays
correctly in cocoon and alexis
Added pdf and plain text transforms to the Cocoon sitemap.xmap file.
This means that live renderings of Alexis files can be seen in plain text and
in pdf formats. The relevant sitemap is the one in the cocoon home directory.
There is an alias cocoon.alexis.sitemap to access this file
Discovered a thing called apache forrest http://xml.apache.org/forrest/ which
is built apon apache cocoon and which seems to include a nice looking wiki
called JSPWiki. http://www.jspwiki.org/Wiki.jsp Forrest may not be a mature
product
work time 7 hours [35]
* 11 august 2003
1am: 2 hours in early morning
2:30 pm investigating a website system for the Alexis project. Am thinking
about using the Apache Forrest system. I set up the forrest system on the
ella associates server. It can be seen at
http://www.ella-associates.org/alexis/
Worked to 6pm
time 5 hours
worked from 10:30 to 3:30
Nick way to open a hole in the iptables for the Mysql database
iptables -A INPUT --protocol tcp --sport 1024: --dport 3306 -j ACCEPT
so i can look at mysql from 'ere
Nick has requested that the cvs be set up
I got external document feeds working in Forrest.
Firstly, copy the sitemap.xmap file from the build/tmp directory of forrest
look at the document http://xml.apache.org/forrest/your-project.html#integrating_rss
now copy that code in the 'internal URI space' section of the sitemap.xmap file.
Change the transform to 'docbook2document.xsl' etc.
Place 'body-' before the apparent file name, this is to allow the skinning process to
work. The only problem now is that the docbook2document.xsl style sheet doesn't seem to
work very well. But this can be solved using the normal docbook style sheets. See the
forrest faq document for how to do this .
total hours=12 [47]
* 12 august 2003
worked 11:30 to 3:30
trying to fix things in the Alexis website at http://www.ella-associates.org/alexis/
but with only partial success. I think that I need to download a more recent version
of forrest using CVS. I will also set up the Ella CVS system today. Have to meet
michael the cellist to help him with dome midi file problem at 6pm
total hours: 4 [51]
* 13 august 2003
I saw michael today and discussed moving in an making copies of CDs.
started work 4:30
writing emails
Trying to get the cvs wrappers list of the local computer, but using PQBoot managed to
make the computer unbootable. It was booting automatically into windows, and now
is always trying to boot from ide0 no matter what the cmos settings are.
I made a list of binary filename extensions for the cvswrappers file by using the following line
lynx -dump http://www.ace.net.nz/tech/TechFileFormat.html | \
expand | sed "s/^[ ]*//g" | \
sed -n "/^[A-Z0-9]\+ /{G;s/\([A-Z0-9]\+\) \([^\n]*\)\(\n\)/#-- \2\3*.\1 -k 'b'/gp;}" | \
tr '[A-Z]' '[a-z]' > cvs-binary-list.txt
And I made a set of uniq file extensions and what they mean in the alexis source tree with
for f in $(find . -iname "*.*" | sed -n "/\.[A-Za-z0-9]*$/p");
do
file $f;
done | sed "s/^.*\(\.[a-zA-Z]\+:.*\)$/\1/gi" | sort | uniq | less
worked till 8:30 pm
total hours 4 [55]
* 14 august 2003
There is a fiesta in the local tonight which will probably stop me doing any work.
El cumpleanyos de Victoria
* 15 august 2003
nothing
* 16 auguste 2003
some cocoon forrest work. Cocoon is giving 'out of memory' errors. Changing the
main sitemap back to the old version did not work
1hour [56]
* 17 august 2003
4:00pm a bit of cvs. thoughts: in my opinion there really should be email notifcation
scripts for the following; cocoon, slide, james, Apache (including websites), dns
mysql, forrest, and the actual American Box itself. Also notification scripts for
disk usage on the American Redhat server
Further struggles with cvs. I created a hole in the filewall with this
iptables -A INPUT --protocol tcp --dport 2401 -j ACCEPT
And I made an xinetd file called /etc/xinetd.d/cvspserver based on info in the cvs faq
then I restarted the xinetd server with /etc/init.d/xinetd restart Then I tried to
make an anonymous user in the 'passwd' file, now I just have to make some real users
using a perl trick also in the FAQ, and things probably should work. Also see my comments
about setting up CVS the previous time- some months ago. (Search for cvs in this document)
Then I needed to update the permissions on the cvs repository with something like
cd $CVSROOT/alexis; chmod -R a+w *
The R switch makes this a recursive change. However we now have a different problem, namely
that it doing a checkout from the cvs repository a 'disk quota' was exceeded. I am not sure
where this disk quota is coming from. Is it part of the 'ftp' system (which has nothing to
do with CVS) or part of the Unix system, or is it a problem with my local computer.
worked till 7pm
worked 2am-3:30
total 4:30 hours [60]
* 18 august 2003
1:00pm some work. I still need to finish off the cvs setup
* 19 - 21 august 2003
I havent been doing work for ella associates during these days. I fixed some 'midi'
files for Michael and met various people.
for f in *.sh; do g=$(echo $f | sed "s/\.sh/\.txt/g"); upload-geocities.sh $g eg/unix-shell ; done | less
a snippet to upload stuff to a geocities account
use the 'top' utility to look at whats happening on the server. This utility shows
which applications/ daemons are using up the most processing time.
To start and stop tomcat webapplications without starting and stopping
the server at a user with role 'manager' to the 'tomcat-users.xml' file
then stop and start Tomcat, then access the url
http://ella-associates.org:8080/tomcat-manager/list
And log on with the user name and password (10 august 2003: ) which you have just entered
This will provide a basic interface to starting and stopping
webapplications as well as installing new ones.
http://www.keysolutions.com/ServletFAQ.nsf/0/99abc10529dab92a8525686700737ea2?OpenDocument
An example Java Servlet
If you want to reload a .bashrc file or something like that type
source .bashrc
or something similar
wget http://www.ella-associates.org/utils/ -nd -np -r -l1 -Rgz
This is a command line to get all the files in a remote 'web directory'
based on the index listing given by the web server. This example
rejects files with the gz file name extension
You can set the system clock with something like
date --set='+4 hours'
which will advance the clock four hours. Type 'info date' for more information
The command below gives a 'nice' full text date format with weekday name
and AM/PM time (no seconds)
date "+%A, %d %B %Y, %I:%M %p"
www.insecure.org
The home site for the nmap port scanner and a good general security site.
important directories
/var/www/ the root for documents served by the Apache Web Server.
A Command Line to send an email through James using 'Blat' (on
Windows). This command sends whatever message the user types at
the command prompt (ending with ^Z) and displays the responses of
the server.
blat - -t [email protected] -plain -u matthew -pw bacibaci -debug
A simple example of an nroff (groff) document is available in the 'vi.advanced' vi
tutorial document. Also included is the command line to process the nroff
document.
to connect to the Alexis Server on the 'local computer' use the url string
www.ella-associates.org/alexis/alexisServer
to restart the inetd server, (according to a cvs mailing-list archive) you
can type 'kill -HUP <inetd process id>' where the stuff in angle bracket is
not taken literally. Get the process id by typing 'ps -el'
The 'ispell' program can do a spell check of html documents (and xml) using the -h switch.
to get basic linux help type 'info Cookbook'
to view mysql docs at the console type 'mysqldoc'
XWindows KDE Font size configured in
main menu --> control center --> look and feel --> fonts
For setting up the DNS Bind package, see the
Linux Network Administrators Guide.
For a discussion about 'vi' and 'vim' the Unix text editors please see the
document http://ella-associates.org/alexis-info/docs/important-tools.html
Also included is a list of vi commands.
A url to send text messages
begira.com
The account is u=nitsk p=ouiji
But I have only managed to get the message 'mensaje no enviado. Codigo 3.'
I dont know what 'codigo 3' stands for.
There is some suggestion that it is possible to send SMS messages through a thing
called ICQ.
http://www.mobilesms.com/whatis.asp
A site explaining SMS from a technical and other point of view
Some classical music sites
A schubert impromptu, chopin etc
http://www.iuma.com/IUMA/Bands/beatasza/
Nice quality, interpretation?
http://www.classicalarchives.com/artists/indexa.html#s
big collection, mainly midis, dubious quality
to restart the local server:
resue root=/dev/hdb4
and then run fsk if there is some hard disk problem