Sources Pieces
Reading
interruption through realization of the work in two threads. The possibility to interrupt
the agent after she has begun reading the text and has not completed it is
realized by starting two threads. The first thread accomplishes the process of
reading. The second thread can interrupt the first one in accordance with the
users desire. When the first thread calls upon the functions for realization
of synchronized with the head and lip movements speech pronouncing, the second
one generates a message, visualized in the message box and informing the user
that the process of reading could be interrupted by clicking on the ΞΚ button
in this message box. If the user clicks on this button, the second thread
interrupts the work of the first one and the agent stops reading the text.
Private Sub
cmdRead_Click(ByVal eventSender As System.Object, ByVal
eventArgs As System.EventArgs) Handles cmdRead.Click
myThread1 = New
Thread(AddressOf Words_rtb)
myThread2 = New
Thread(AddressOf StopReading)
ReadQuietly = True
ReadFast = True
myThread1.Start()
myThread2.Start()
boolStopText = False
'Call
Words_rtb()
End Sub
Private Sub
StopReading()
MsgBox("Click
here to stop reading")
myThread1.Abort()
Timer1.Enabled = True
Timer1.Start()
NowReadingDoNotLookForMouseMove = False
boolStopText = True
End Sub
We have two basic head positions and all
possible eye and lip positions corresponding to them. Each of the images has
its own number. For each of the two head positions there are 10 images arranged
in the same sequence. Thus the number of each image for the second basic
position can be obtained by adding 10 (the number of the images for one of the
basic head positions) to the number of the same image for the first basic position.
Now it should be watched which of the two basic
head positions is the current one in order to visualize the images for the
realization of lip movements for this head position synchronized with the word
being pronounced, and also for the realization of the eye blinking once in a
while.
A Boolean variable keeps track of which basic
head movement is used any moment. It is true when the agents head is in the
first basic position and false when the head is in its second
basic position. For the realization of the lip movement in the first basic head
position the numbers of the pictures read from the relational database are
directly used.
When the Boolean variable is in a false position and
the second basic agents head pose is the current one, then to the number of
the picture to be visualized number 10 has to be added (the number of poses for
a basic position), and the same picture for the second agents head position
appears.
Private Sub
HeadLeftRight()
If numberHeadBaseMirror
>= 20 Then
numberHeadBaseMirror = 0
If
boolHLR = False Then
boolHLR = True
addHLR = 10
Else
boolHLR = False
addHLR = 0
End
If
End If
End Sub
For
every word call sub-routine Speech() and reading information from database
while the buffer not full. Sub-programme Speech()
extracts all the information about the word from the relational database
and stores it in a buffer (the absolute path to the wav file for each word is
concatenated this way).
Private Sub
Speach(ByRef word As
String)
Dim grdSrichka As Object
On Error GoTo
LoadErr
Dim
wsApp As DAO.Workspace
Dim
dbAccount As DAO.Database
Dim
srFile As DAO.Recordset
Dim
strSQL As String
Dim
imgNumber1 As Integer
Dim
imgNumber2 As Integer
Dim
imgNumber3 As Integer
Dim
imgNumber4 As Integer
wsApp =
DAODBEngine_definst.Workspaces(0)
dbAccount = wsApp.OpenDatabase(sFileName)
strSQL = "Select
* from " & sTableName
srFile =
dbAccount.OpenRecordset(strSQL,
DAO.RecordsetTypeEnum.dbOpenSnapshot)
srFile.MoveFirst()
Do While Not srFile.EOF
If word
= srFile.Fields("Word").Value Then
HeadLeftRight()
imgNumber1 = srFile.Fields("Picture number1").Value
imgNumber2 = srFile.Fields("Picture number2").Value
imgNumber3 = srFile.Fields("Picture number3").Value
imgNumber4 = srFile.Fields("Picture number4").Value
imgNumber1 = imgNumber1 +
addHLR
imgNumber2 = imgNumber2 +
addHLR
imgNumber3 = imgNumber3 +
addHLR
imgNumber4 = imgNumber4 +
addHLR
If
ReadQuietly And ReadFast Then
MasivDir(MasivBr,
1) = "C:\Izkustven_Intelekt\VB_Project_HeRead\he_read_VB_Dot_Net\wavfiles\wordsfastquietly\"
& srFile.Fields("Name and path to file").Value
ElseIf
ReadQuietly = False And
ReadFast = False Then
MasivDir(MasivBr,
1) = "C:\Izkustven_Intelekt\VB_Project_HeRead\he_read_VB_Dot_Net\wavfiles\wordsslowlyloudly\"
& srFile.Fields("Name and path to file").Value
ElseIf
ReadQuietly = False And
ReadFast Then
MasivDir(MasivBr,
1) = "C:\Izkustven_Intelekt\VB_Project_HeRead\he_read_VB_Dot_Net\wavfiles\wordsfastloudly\"
& srFile.Fields("Name and path to file").Value
ElseIf ReadQuietly And
ReadFast = False Then
MasivDir(MasivBr,
1) = "C:\Izkustven_Intelekt\VB_Project_HeRead\he_read_VB_Dot_Net\wavfiles\wordsslowlyquietly\"
& srFile.Fields("Name and path to file").Value
End
If
MasivPicture1(MasivBr) =
imgNumber1
MasivPicture2(MasivBr) =
imgNumber2
MasivPicture3(MasivBr) =
imgNumber3
MasivPicture4(MasivBr) =
imgNumber4
MasivBr = MasivBr + 1
numberHeadBaseMirror =
numberHeadBaseMirror
+ 1
Exit
Sub
End
If
srFile.MoveNext()
dbAccount.Close()
Exit Sub
LoadErr:
End Sub
When
buffer is full call sub-routine to prononce words from buffer. When the buffer is full or there are no more words
in the text, sub-programme Prononce() is called upon, which executes the pronunciation
itself and visualizes the images required for the realization of the
synchronized lip, head and eye movements of the agent.
Private Sub
Prononce()
Dim i As Integer
numberHeadBaseMirror =
stignahdoNomer
CounterEyesBlink =
stignahdoCounterEyesBlink
addHLR = stignahdoaddHLR
boolHLR = stignahdoboolHLR
For i = 0 To MasivBr - 1
HeadLeftRight()
If ReadFast
Then
gr.DrawImage(ImageList1.Images(MasivPicture1(i)),
10, 10)
System.Threading.Thread.Sleep(160)
gr.DrawImage(ImageList1.Images(MasivPicture2(i)),
10, 10)
System.Threading.Thread.Sleep(160)
gr.DrawImage(ImageList1.Images(MasivPicture3(i)),
10, 10)
System.Threading.Thread.Sleep(160)
gr.DrawImage(ImageList1.Images(MasivPicture4(i)),
10, 10)
MMCntrlWav.FileName = MasivDir(i,
1)
MMCntrlWav.Command = "Sound"
Else
gr.DrawImage(ImageList1.Images(MasivPicture1(i)),
10, 10)
System.Threading.Thread.Sleep(600)
gr.DrawImage(ImageList1.Images(MasivPicture2(i)),
10, 10)
System.Threading.Thread.Sleep(600)
gr.DrawImage(ImageList1.Images(MasivPicture3(i)),
10, 10)
System.Threading.Thread.Sleep(600)
gr.DrawImage(ImageList1.Images(MasivPicture4(i)),
10, 10)
MMCntrlWav.FileName = MasivDir(i,
1)
MMCntrlWav.Command = "Sound"
End If
numberHeadBaseMirror = numberHeadBaseMirror +
1
CounterEyesBlink = CounterEyesBlink +
1
If
broqchMigane = 6 Then
gr.DrawImage(ImageList1.Images(4 +
addHLR), 10, 10)
System.Threading.Thread.Sleep(50)
CounterEyesBlink = 0
gr.DrawImage(ImageList1.Images(0 +
addHLR), 10, 10)
End If
Next
stignahdoNomer = numberHeadBaseMirror
stignahdoCounterEyesBlink = CounterEyesBlink
stignahdoaddhlr = addHLR
stignahdoboolHLR = boolHLR
End Sub
Following the mouse movement with eyes and
shaking head when the mouse passes over the agents face. Agents eyes follow the cursor movements before
the reading has begun. On the purpose the event mouse move is processed and, in
accordance with the current coordinates of the mouse position, a collection of
images with various agents eye positions is visualized again, which creates
the impression of eye movement following the movement of the cursor.
When
the cursor goes over the agents face, he/she reacts by shaking his/her head. A
change between the two agents head positions is realized again.
Private Sub
frmFiles_MouseMove(ByVal sender As Object, ByVal e As
System.Windows.Forms.MouseEventArgs) Handles Me.MouseMove
CurrentX = e.X
CurrentY = e.Y
If
NowReadingDoNotLookForMouseMove = False Then
If
CurrentX > 150 And CurrentY < 80 Then
Label1.Text = CurrentX & " " & CurrentY
gr.DrawImage(ImageList1.Images(20), 10, 10)
ElseIf
CurrentX > 150 And CurrentY < 160 Then
Label1.Text = CurrentX & " " & CurrentY
gr.DrawImage(ImageList1.Images(18), 10, 10)
ElseIf
CurrentX > 150 And CurrentY > 160 Then
Label1.Text = CurrentX & " " & CurrentY
gr.DrawImage(ImageList1.Images(21), 10, 10)
ElseIf
CurrentX < 100 And CurrentY < 100 Then
Label1.Text = CurrentX & " " & CurrentY
gr.DrawImage(ImageList1.Images(20), 10, 10)
ElseIf
CurrentX < 100 And CurrentY < 160 Then
Label1.Text = CurrentX & " " & CurrentY
gr.DrawImage(ImageList1.Images(19), 10, 10)
ElseIf
CurrentX < 100 And CurrentY > 160 Then
Label1.Text = CurrentX & " " & CurrentY
gr.DrawImage(ImageList1.Images(21), 10, 10)
ElseIf
CurrentX > 50 And CurrentY > 50 And_
CurrentX
< 150 And CurrentY < 150 Then
Label1.Text = CurrentX & " " & CurrentY
gr.DrawImage(ImageList1.Images(9), 10, 10)
Zabavqne(20)
broqchMishka = 0
gr.DrawImage(ImageList1.Images(0), 10, 10)
End
If
End If
End Sub