The 'master' branch is the more interesting one as it uses the viseme information that Amazon Polly provides. For this to work I needed some linguistics help to interpret what those visemes mean.
In my DevCon session I jokingly stated that the session happened because of the help of both my daughters, one is in university studying Biology (fish, get it?) and the other is in university studying Linguistics. So I was all set 🙂 Thanks, Girls!
Back to those visemes, Figure 28 shows what Amazon sends you:It tells you the mouth position at any time as it changes during the audio. And we can determine the duration from the time elapsed between two entries.
But which of those weird-looking viseme characters indicate whether the mouth is open or closed? Obviously, the plastic fish doesn’t have the full mouth range as we have, it can’t purse its lips for instance. So, I just had to find the ones that mean that the mouth is open, half open, or closed. Figure 29 shows the visemes in master branch code.
Query the FileMaker Data API
You will note that line 272 in Figure 31 introduces a delay between starting the voice (the audio playback) and starting the thread for the mouth, that's a value set in billy.ini and set to what I found to be necessary to make sure the mouth action did not start earlier than the audio playback.
And with that I was pretty much done. Everything was working as I wanted. The new pinion gears arrived, and I replaced the old gears on all three of my Billy Basses (the original one and my two Billy Bass Bones), and now the heads turned the proper 90 degrees, making Billy look straight at you as it begins talking.
It turns out that the excellent (and free) Visual Studio Code editor on macOS also supports Python as shown in Figure 33.
All I had to do was clone my GitHub repository to my Mac, and away I went. From my Mac, I can SCP (Secure Copy Protocol) to copy the modified Python scripts over to the Raspberry Pi and use SSH in Terminal to run the main Python script to put the Raspberry Pi in its wait-and-read loop. Figure 34 shows the Python script running on the Raspberry Pi, logging every step of its actions.
And here is a final picture of my hotel room with the whole thing set up for last-minute testing:
Time for a little fun then. Early on I mentioned that the text we are sending to Amazon Polly uses SSML markup. Using SSML allows us to specify things like accents. Or tell Polly to speed up the text narration, like so:
Hi, my name is Andrew Lecates, when I get going, <prosody rate="x-fast">you really have to fasten your seatbelts, because there is a lot to cover and you are standing between me and my coffee, or red bull, or whatever the heck it is that makes me talk like this.</prosody><amazon:breath duration="x-long" volume="x-loud"/><break time="500ms"/>b</speak>
I had a lot of fun putting this particular DevCon presentation together, and there is something immensely satisfying to seeing devices collect sensor data and being able to things move from a FileMaker solution.
If nothing else I hope I have demonstrated that the FileMaker platform fits in very well in this IoT environment.
Writing the code to run on the Raspberry Pi was fairly straightforward, in both C# and Python. None of the demos that I used in my session are more than 300 lines of code, with plenty of whitespace, comments and logging. Don’t let unfamiliarity with those languages deter you. It's easier than you think.
Questions about anything in this series; leave a comment here or find me on community.filemaker.com