Dustin Stevens-Baier
Comp 569
Assignment #7
I choose to take the domain I did in the problem number 6 with pitch recognition and make some changes to it in an effort to get better results. I have the same three inputs the horizontal location of the pitch, this was basically how confident the batter is that the horizontal location will be a good pitch to hit. The vertical location of the pitch, again how confident the batter is that the vertical location will be a good pitch to hit. Finally, there was the pitch recognition, this was how confident the batter was that they recognized the type of pitch (fastball, curve ball, etc.). then there were eight nodes in the hidden layer (I also tried with 10). The last layer was the output nodes which take the hidden layers into account both the hidden layer nodes into account.
I decided to limit the types of output to Swing or Take the recommended action to take. This is defined as a 1,0 for a swing and a 0, 1 for a take.
The training set that I used was 10,000 pitches that where run through for 1,000 iterations or epochs ( I also tried 2,000). I used the 10,000 because I wanted to get a large enough training set to avoid the possibility of predicting Takes to frequently. When running through the training set I start by creating a new pitch which takes a random number between 0 and 1 for x position, y position, and recognition. This is used to set the activations for the input nodes. thes are run through the network one at a time. Then I start the feedfoward and calculate the hidden nodes and then the output nodes. Then the target is set and from there I do the backpropogation, finding the output delta and then the hidden delta. This is then used to adjust the weights of input to hidden and hidden to output. Then finally I update the bias values.
Selecting a stepsize also called gain in the code, I choose .1 this value seemed to produce reliable results. I have also used .2 which wasn't as good the effective ness seemed to drop from around 90% to 83%-86%. I did not use a momentum turn, I did not think that such a simple network would gain much values from this.
When running test runs I tried some with an EPOCH value of 100 this was very erratic and produced a range of results from 65% to 90%. I then upped my value to 300 and the output was right around 90% and was much more consistent. I then tried 500 and noticed some slight improvement in consistency but it didn't seem that drastic. I used 1000 in the sample ouput to make sure that I was running through the system more than enough.
I used the following sigmoid function as it appeared to be a common function used according to class notes and wikipedia.
Sample output:
settings-
#define NUMINPUTS 3
#define NUMHIDDEN 10
#define NUMOUTPUTS 2
#define TRAINING_POINTS 10000
#define TESTING_POINTS 100
#define EPOCH 2000
#define THRESHOLD 0.01
This was one of the better results that i got. usually these settings where between 93% and 96%. If you look closely the ones that it has a problem with are ones which are around .15 x or y values, this is because this is the cutoff for swinging, if the value is less than .15 for xpos, ypos or recognition then you should take. Also if the value is greater than .85 in xpos or ypos then you should take. This means that the esges in the categories are when we are going to see the most mistakes for the testing run.
Settings-
#define NUMINPUTS 3
#define NUMHIDDEN 10
#define NUMOUTPUTS 2
#define TRAINING_POINTS 10000
#define TESTING_POINTS 100
#define EPOCH 1000
#define THRESHOLD 0.1
Again this is one of the better scores with this batch of settings the effectiveness has usually been around 86%.
The system seemed to handle the data that was not in the training set because the values that are in the training set are random and the values in the testing set random, so I am hoping that we have data that has not been dealt with before. The data however does fall within the same range so I cannot say that it would handle data outside these ranges.
Code:
References:
Neural Net Class Notes
Artificial Intelligence Class Notes
http://en.wikipedia.org/wiki/Feedforward_neural_network