ChatterPi: a Raspberry Pi Audio Servo Controller

Introduction

ChatterPi is a software package that turns a Raspberry Pi into an audio servo controller. In other words, the Pi outputs commands to control a servo based on the volume of the audio input. The input can be either stored audio files (in either mono or stereo .wav format) or from an external source, such as a microphone or line level input. One of the uses is to drive animatronic props, such as a skull or a talking bird.

Background: A Brief History of Talking Skull Control

A common prop that still makes a good impact is a talking object, whether a skull or animal. Some lower cost commercial props use a motor and spring. Another approach is to pre-program a complete sequence to match the vocals, but this is very time consuming and if you want to change the vocals, or even just edit them slightly, you need to reprogram the entire sequence. For that reason, the use of an audio servo controller to drive a servomotor controlling the jaw is a very popular approach. There are several variations. One of the earliest use hardware to detect when the audio exceeded a threshold, and then began moving the jaw towards a fully open position, and when the audio went below the threshold, it would begin closing the jaw. “Scary Terry” Simmons may have been the first to develop an electronic hardware board to do this, and Cowlacious Designs has continued to improve and sell commercial versions, with many added features such as a built in audio player, various triggering options, and the ability to control LEDs as eyes.

Later, someone named Mike (no relation) combined an Arduino with a hardware volume level board to produce the Jawduino. This went from having just 2 levels to 4. The original project just took audio in and controlled the servo, but others added extensions to play stored mp3 files and/or randomly move additional servos (for example, http://batbuddy.org/resources/Halloweenstuff/TalkingSkull.php).

A few years ago, Steve Bjork from Haunt Hackers combined dedicated hardware with a propeller microcontroller to increase the number of levels to almost 256 and also to filter out low and high frequencies that don’t tend to result in jaw movement for spoken sound. The result is the Wee Little Talker. This commercial board also has an onboard mp3 player, can be triggered externally, control LED ‘eyes,” and adds a wide array of features including a voice feedback menu system.

It occurred to me that with current single board computer capabilities and powerful software libraries, it should be possible to incorporate most of the best features of all of these into a single, software-based system running on a Raspberry Pi. The result is ChatterPi. ChatterPi was developed from scratch using the Python language, but ideas for capabilities and features were freely borrowed from previous audio servo controller projects.

Features

ChatterPI is designed to be extremely powerful and flexible without requiring the user to modify any of the code (although advanced users can certainly do that as well). Table 1 compares the current capabilities of ChatterPi with other audio servo controllers. The full range of capabilities and options are described in the Operations subsection.

Table 1. Feature comparison of ChatterPi and other audio servo controllers

  Cowlacious Scary Terry Board Original Jawduino Wee Little Talker ChatterPi
Audio signal controls servo ü ü ü ü
External Trigger ü X ü ü
Timer Trigger X X X ü
File or ext. audio input ü external only ü ü
Control Levels 2 4 ~256 4
Bandwidth Filter X X ü ü
Plays “ambient” sound files X X ü Planned
Automatic “Gain Control” X X ü Planned
Trigger Out ü X X ü
Control LED “eyes” ü X ü ü
Settings Few (control knobs) Requires software edit Comprehensive voice menu Config file (GUI planned)

Demonstration Video

This video shows ChatterPi in action, using both a saved .wav file and microphone input. ChatterPi is controlling the jaw movement. The other skull movements are pre-programmed as a script running on a Pololu Maestro Servo Controller.

Using ChatterPi

This section describes how to set up and install both the hardware and software for using ChatterPi, and for using it.

Hardware

ChatterPi was developed and tested on a Pi 3 A+ and a Pi Zero W. It should work on any Pi. [Originally it ran too slowly on a Pi Zero. A section of code for analyzing audio volume used list processing and a loop. This code was replaced using Numpy, and it’s now fast enough to work on a Pi Zero.

In addition to the Raspberry Pi, you’ll need a USB sound card. This is needed for several reasons. First, if you plan to use an external sound source, you need a way to get audio into your Pi. Second, besides not producing very good sound, the audio out connector may share timing with the Pulse Width Modulation (PWM) code that is needed to drive the servo, creating conflicts.  Use of an inexpensive USB sound card solves both issues. I’ve used one from Adafruit that sells for less than $5 and works well (see https://www.adafruit.com/product/1475). You will need TRS (standard stereo) plugs or an adapter to go into the headphone and microphone jacks on the sound card. The card does not work with a TRRS (combination microphone / stereo headphone plug. You only need a microphone or other external sound source if you wish to use one. Otherwise you can use audio .wav files that you save on the Raspberry Pi. You still need the USB sound card for audio output, however.

That’s all you need for the audio servo controller. Of course, you’ll need a power supply and a servo that you want to control, such as a servo-equipped talking skull, and a passive infrared sensor (PIR) if you want to trigger your prop using one. I used this one (https://www.parallax.com/product/555-28027) from Parallax for development, as I had a spare one already. ChatterPi can also be set to trigger off of a repeating timer or to just turn on and run if you don’t want to use an external sensor.

Figure 1 shows a test bench setup for testing operation. The Red LED is attached to the “TRIGGER_OUT” pin for testing purposes. It can be moved or another LED and resistor attached to the “EYES_PIN” to test that feature. The TRIGGER_OUT pin goes high for 0.5 seconds when the controller is triggered. This can be used to trigger another prop or controller. The EYES_PIN stays high for as long as the audio plays.

Figure 1. A layout for fully testing the ChatterPi

The default PIN selection (which can be changed in the config.ini file) are:

  • Jaw servo: 18
  • PIR input trigger: 23
  • Trigger out: 16
  • Eyes: 25

Figure 2 is a photograph of my test setup. The placement of the wiring on the breadboard is slightly different because I was using it to test a variety of items and also a 3-wire servo controller wire, but the schematic connections are identical.

picture of the test setup

Figure 2. A picture of the test seup used for development

Software Overview

Knowledge or understanding of the software code is not required to operate or use the ChatterPi. The ChatterPi package consists of four Python 3 modules and one configuration file, as shown in Figure 3.

Software Configuration

Figure 3. The four python modules and one configuration file in ChatterPi

The configuration file, config.ini, holds all of the user selectable parameters, including which pins are used for which functions, whether the audio source is the microphone input or stored .wav files, which servo control mode should be used, and the servo threshold levels. The config.py program simply reads these values and makes them available in memory during runtime.

Most of the processing occurs in the main.py and audio.py modules. The main.py program handles triggering (either my time, an external trigger such as a PIR, or immediately upon startup, with the method specified in the config.ini file. It uses the GPIO Zero and PiGPIO libraries to monitor the triggering sensor and send output to the output trigger and led pins. PiGPIO is used as the GPIO layer underneath GPIO Zero because it uses DMA control for the Pulse Width Modulation (PWM) control used to the control the servo. Some other libraries, including the default one used by GPIO Zero, use software PWM, which is adequate for tasks such as controlling the brightness of LEDs, but not precise enough for servo control.

Unless the triggering mode is START, the file enters an infinite loop waiting for either a timer to expire (TIMER mode) or the external trigger to be generated (PIR mode). The wait functions meet the requirements and during development, interrupt driven approaches interfered with the audio output, probably due to timing conflicts. In TIMER mode, the timer is restarted after either the audio file finishes playing (if the source is FILES) or after a configurable pre-set time (if the source is MICROPHONE).

When triggered, an event handler is called that, depending upon the settings, sets off the TRIGGER_OUT to trigger another prop or device and turns on the LED eyes or other low power device. Then, if the audio source is FILES, it will select the next .wav file to be played and call audio.py, passing the name of the .wav file to be played. If the audio source is MICROPHONE, audio.py is called without passing a file name.  When the call to audio.py returns, the event handler turns the LED eyes off and returns.

Audio playback, audio analysis, and servo control are all performed by the audio.py module. It defines one class, AUDIO. When the audio.play function is called, it checks on whether the audio source is MICROPHONE or FILES and opens a PyAudio stream appropriately. The stream call runs in a separate thread (this is automatically handled by PyAudio). For each chunk of the input stream, a callback function is called. This callback function is where the audio stream volume is analyzed. The average volume for each chunk is calculated, and the servo is commanded to the appropriate position based on that average volume and the threshold levels that the user has specified in the config file. The wave library is used to read the wave files from storage, and the struct library is used to help deconstruct the wave data to calculate volume and to help to separately analyze the left and right channels for stereo files. The number of levels, the specific thresholds, and whether or not a bandpass filter is applied before calculating the volume is based on the STYLE setting set by the user in the config file. In addition to the official documentation, I found a slide presentation, Introduction to PyAudio, by Jean Cruypenynck to be very helpful.

If the STYLE is set to 2, then bandpassFilter.py is called to process the digital audio stream and return a modified stream with the bandpass filter applied. The program is very short and simple. It uses two functions from the scipy signal processing library to filter out audio input below 500 Hz and over 2500 Hz. No bandpass filter is applied for STYLE 0 or STYLE 1.

Software Installation and Setup

This section assumes you have a Raspberry Pi with Raspbian installed and either running with a keyboard and display or in headless mode remoted to another system. There are good instructions on the Internet already on how to get a Raspberry Pi up and running. You also need to have the Pi connected to the Internet to download software.

ChatterPi is written in Python 3, and Python 3 comes already installed with Raspbian. Hower, in addition to the ChatterPi modules, you’ll need to install and initialize several other libraries that are not included out of the box in Raspbian. These are:

  • GPIO Zero : This is already installed if you have loaded the full Raspbian operating system, which is recommended. This is the library ChatterPi uses to control the General-Purpose Input/Output (GPIO) pins on the Raspberry Pi. It is very user friendly for programming.
  • PiGPIO: this is the underlying pin control library that is used “underneath the covers” of Gpio Zero.
  • PortAudio: A library for audio playback and recording.
  • PyAudio: The wrapper that lets Python programs access PortAudio.
  • Scipy: A Python library for scientific and technical computing. ChatterPi uses its bandwidth filter function in some modes.

Here’s how to go about setting this up. On the Raspberry Pi, open a command line. It’s always a good idea to be sure all the packages on your system are up to date. Type the following:

$ sudo apt-get update

$ sudo apt-get upgrade

If you are running the full Raspbian installation, you can skip these steps. If you are running Raspbian Lite (untested), type:

$ sudo apt update

Then install the package for Python 3:

$ sudo apt install python3-gpiozero

To install PiGPIO, type:

     $ sudo apt install pigpio

$ sudo systemctl enable pigpiod

$ sudo apt-get install python3-pigpio

The first command downloads and install PiGPIO, while the second command ensures that it starts running in the background whenever the Pi starts. The third command installs the required python wrapper module.

To install PortAudio:

$ sudo apt-get install portaudio19-dev

To install PyAudio

$ sudo apt-get install python3-pyaudio

To install Scipy, use the command:

$ sudo apt-get install python3-scipy

Audio Device Setup and Volume Control

The easiest way to select the USB sound card for audio input and output is through the volume control on the PI’s desktop. Right click on the volume (speaker) icon on the upper right of the screen and select USB Audio Device for input and output. You can also use this control to adjust the volume.

To adjust the volume from the command line, type:

$ alsamixer

Do not set the microphone input or speaker output levels too high, as you may generate too much noise or feedback.

If your USB card does not show up, then that card does not have hardware support. Google “raspberry pi software volume control” to find out how to control the volume via the command line.

Last step: reboot your Pi so that all changes will take effect.

Congratulations, your ChatterPi is now all setup and ready for use!

Operation

QuickStart: Over time, you should explore and experiment with the many parameters can be changed in the config.ini file. However, the number of parameters can seem intimidating, so this section will get you up and running with a minimum of work. Initial default values have been set for all the parameters, the source is set to use audio files, and a sample audio file is included, so you do not need your own audio file or a microphone to get started. The trigger is set to TIMER mode, so an external sensor such as a PIR is not needed.

First, install the ChatterPi files in /home/pi/chatterpi. If you know the upper and lower limits for your servo, open the config.ini file for editing and set the SERVO_MIN and SERVO_MAX values (in microseconds) the values for your servo. Then save the file. Now go to the terminal window and go to the /home/pi/chatterpi directory. Then run ChatterPi:

$ cd /home/pi/chatterpi

$ python3 main.py

You’ll see a lot of warnings and errors concerning “ALSA” but don’t worry about them. Believe it or not, this is normal. Every 10 seconds, the audio will play and the servo will move in sync. Congratulations!

Control-C will stop the program.

Detailed Instructions: The ChatterPi files should be installed in /home/pi/chatterpi. Within the chatterpi directory there is a subdirectory called vocals. The sound files you wish to play should be placed in that folder. The files must be 16 bit .wav files (the most common type) and can be either mono or stereo files. You must use the naming convention v##.wav, where xx ranges from 01 to 10. The purpose is so that you can have the prop play different audio tracks, in a set order, one each time the prop is triggered. The first time the prop is triggered it will play the lowest numbered file, and cycle through the list, skipping any missing numbers, and starting over again with the first track after all have been played. If you only want to play one track, that’s fine, just name it v01.wav. If you need to edit your files, or you have mp3 or other format files that you need to convert to .wav to work with ChatterPi, the free and excellent Audacity software can do this and much more.

Use of Separate Tracks in Stereo Files: If you just have a mono or stereo sound file that you want to play and control the servo, you can skip this section. However there are some tricks that can be used with stereo .wav files to improve the fidelity of your jaw motion in some scenarios. The jaw motion is actually controlled exclusively by the right channel of stereo .wav files. The audio output sent to the speakers can be set using the OUTPUT_CHANNELS parameter to be either LEFT (only) or BOTH (both left and right channels. The reason for providing this option is that some haunters want to do something different with the right channel. One option is to record the desired complete audio output (the voice along, with music or other sounds) on the left channel (channel 0) and record a separate track of just sounds or signal tones, to drive the servo but not be played out through the speakers, on the right channel (channel 1). That way you can more precisely control the servo action and have it, for example, not open wide when a loud “ssssss” sound occurs. To do this, set the OUTPUT_CHANNELS parameter to LEFT, since you do not want to play the right channel through the speakers.  Another option is to provide the voice on the right channel, and any accompanying audio or other sounds on the left. In this case, set the OUTPUT CHANNELS to BOTH. The right channel, as always, will control the servo, but both the voice and accompanying music will be sent out to the speakers.

Configuration: The options and parameters for ChatterPi are set in the config.ini file, which should be edited and saved with your desired settings prior to running the program. ChatterPi comes with a default config.ini file. It is recommended that you save a backup copy under another name. That way, you can always revert back to a working configuration file. Config.ini is divided into 5 sections, which we will now walk through. A copy of the contents of the default config.ini file is provided in Appendix B. There are a lot of parameters you can change, in order to provide flexibility, however you do not have to set or change them all. Simply focus on the parameters of interest to you. As described in the QuickStart section, you can run ChatterPi with the default settings right out of the box.

[SERVO]

This section has four parameters. SERVO_MIN and SERVO_MAX are the minimum and maximum values to set your servo to, to avoid going beyond it or the prop’s limit. This is provided in microseconds (the time of the pulse signals).

MIN_ANGLE and MAX_ANGLE are the open and closed position for your servo, in degrees. If your jaw is closed when the servo is commanded to a high value and open when the servo is commanded to a minimum value, you should set the MAX_ANGLE to the value for closed and the MIN_ANGLE to the value for open.

[CONTROLLER]

This section selects the “style” to be used for controlling the servo and the threshold values for each style. There are 3 styles, numbered 0, 1, and 2. Style 0 operates like a Scary Terry style audio servo controller. It begins to open the jaw fully when the audio input volume exceeds the threshold, and begins completely closing the jaw when the volume is below that threshold.

Style 1 operates like the jawduino. The servo is commanded to open 1/3rd of the way if the first threshold is exceeded, 2/3rds if the 2nd threshold is exceeded, and completely if the third threshold is exceeded.

Style 2 functions like style 1, but first applies a bandpass filter to filter out low and high frequencies, since in speech, these do not tend to cause jaw movements.

The audio volume of each sample of the audio input is an integer that may range from 0 to 32767. The thresholds should similarly be set to integer values in this range.

[AUDIO]

This section specifies the source for the audio input (either FILES or MICROPHONE). Files must be 16 bit .wav files, either monophone or stereo. External input can be either from a microphone or line in from another source.

The OUTPUT_CHANNELS specifies, for stereo files, whether both channels should be sent to the speaker (BOTH) or just the left channel (LEFT). See the subsection above titled Use of Separate Tracks in Stereo Files for more information on why and how you may want to use this feature. If you have not done any special editing on your files, leave it set to BOTH.

When you are using .wav files the software can determine when it has reached the end of the file. If you are using external input by setting SOURCE to MICROPHONE, the software has no way to determine when an audio stream has ended. The MIC_TIME setting, in seconds, specifies how long to keep the audio stream running form the external input when ChatterPi is triggered. I’ve found that the input levels on my setup are lower when using a microphone so I have to lower my trigger levels.

The AMBIENT parameter is not currently used. It will be used in a future release.

There is one other parameter in this section, the BUFFER_SIZE. This parameter is not currently used.

[PROP]

The parameters in the PROP section control how the prop is triggered and what else may be triggered or turned on at the same time.

PROP_TRIGGER takes one of three values, PIR, TIMER, or START. PIR will have ChatterPi start the audio and controlling the servo whenever the PIR_PIN is sent a 3V signal from another device. Often, this may be a Passive Infrared (PIR) motion detector sensor, but it can be anything you’d like. When set to TIMER, ChatterPi starts the audio and controls the servo every time the timer expires. The delay time, in seconds, between the end of one activation and the start of the next is set by the DELAY parameter. START is only a valid setting when the SOURCE is MICROPHONE. It turns on the audio input and servo control immediately upon start up and leaves it open and running until the program is ended.

EYES may be set to ON or OFF, to control whether or not the EYES_PIN is set high when the prop is triggered and stay high until the audio input ends.

TRIGGER_OUT may be set to ON or OFF to control whether or not the TRIGGER_OUT_PIN is set to high for 0.5 seconds when the prop is triggered. This can be used to trigger another device or prop.

The effects of the SOURCE and PROP_TRIGGER selections are summarized in Table 2.

Table 2. Behavior is Dependent Upon SOURCE and PROP_TRIGGER Settings

  PIR TIMER START
FILES Audio files and servo control starts when trigger goes to high, stays active until the audio file ends.

Eyes, if ON, light up while active. TRIGGER_OUT, if ON, goes high for 0.5 seconds at start of each activation.

Audio external input and servo control starts time expires, stays active until the audio file ends. Then the timer is restarted.

Eyes, if ON, light up while active. TRIGGER_OUT, if ON, goes high for 0.5 seconds at start of each activation.

N/A. Will cause an error
MICROPHONE Audio external input and servo control starts when trigger goes to high, stays on for length of time set by the MIC_TIME parameter (in seconds)

Eyes, if ON, light up while active. TRIGGER_OUT, if ON, goes high for 0.5 seconds at start of each activation.

Audio external input and servo control starts when trigger goes to high, stays on for length of time set by the MIC_TIME parameter (in seconds).  Then the timer is restarted.

Eyes, if ON, light up while active. TRIGGER_OUT, if ON, goes high for 0.5 seconds at start of each activation.

Audio external input and servo control starts as soon as ChatterPi starts running and stays active until the program is terminated.

Eyes, if ON, light up the entire time. TRIGGER_OUT, if ON, goes high for 0.5 seconds at start of the program.

 

 

Unless you are using the pins for some other purpose, it is fine to leave EYES and TRIGGER_OUT set to ON, even if you are not using them.

[PINS]

This section specifies which GPIO pins are used for the servo control channel (JAW_PIN), the input trigger (PIR_PIN), the LED “eyes” output (EYES_PIN), and the trigger output (TRIGGER_OUT_PIN).

Running the Program: To run the program, you can start main.py from the command line by going to the chatterpi directory and typing:

$ python3 main.py

There will be a large number of ALSA warnings and errors displayed on the screen. Believe it or not, this is normal, and ChatterPi will run properly.  Control-C will stop the program.

Run Automatically On Boot

Your ChatterPi is hopefully now up and running and you have adjusted the parameters. If you want to start ChatterPi from the Pi’s command line or GUI, you’re good to go. But for use in a prop, you may want to configure your Pi to automatically begin running ChatterPi on boot, with no need for user interaction. To do this, move the chatterpi.service file in the chatterpi directory to /lib/system/system. Then type:

$ sudo systemctl enable chatterpi.service

This tells the operating system to run chatterpi at the end of the boot sequence. See https://www.raspberrypi.org/documentation/linux/usage/systemd.md for more information and commands you can use form the command line to restart or stop ChatterPi when it is set to run automatically on boot.

Project Roadmap

ChatterPi is already fully functional, with a wide array of options However there several capabilities and convenience features that I plan to add. The next release will add:

  • A stand-alone utility program with a graphical user interface for editing the configuration file.

Beyond the next release, there are several items on the added capabilities list that may be added in one or more releases (one or two may even sneak into the next release):

  • Ambient tracks: Have ambient soundtracks that play between activations. This idea is borrowed from the Wee Little Talker.
  • A stand-alone utility program to allow the user to automatically maximize the volume levels on each .wav file to just below distortion and save the new files. This minimizes the need to re-tune the sensitivity settings and ensures that each .wav file has the same volume level, if desired.
  • Add a “TUNING” mode so the user can adjust type of triggering, triggering levels, and the servo max and min positions on the fly, while the program is running. This will make it easier to tune these parameters, since currently one must stop the program, change the settings, and then rerun the program to see the effects of any changes.
  • Add the ability to use .mp3 files. Simply playing MP3 files on a Raspberry Pi is easy, but they must be processed in real-time as a stream to drive the servo controller.
  • Use the timer function in PIR mode as a delay before re-triggering is allowed.

The code is open source and published on GitHub (https://github.com/ViennaMike/ChatterPi), and I would welcome anyone who wanted to work on adding any of these advanced features.

A Sliding Puzzle for the PyBadge and PyBadge LC

15 tile sliding puzzle

A plastic  sliding 15 puzzle using numbers

The sliding puzzle has a long history. The first one, a 4×4, 15 tile puzzle, was made in 1890.  A plastic version is shown to the right. The puzzles may have letters forming words or have pictures instead of, or in addition to, numbers. Basically, the tiles are randomized, and then must be returned, by sliding adjacent tiles into the open space one at a time, until the puzzle is returned to the original arrangement.

Many such puzzles have been made, in various sizes. In addition to physical puzzles, video versions can be found on computers and the web.

This is a sliding puzzle game written in #CircuitPython for the @Adafruit #PyBadge and PyBadge LC. It uses pictures for the puzzle, with numbers superimposed to make the puzzles easier to solve. It is configurable so that different images can be used, and it supports both 3×3 (8 tile) and 4×4 (15 tile) puzzles. The tiled graphics approach used in the Adafruit displayio library is ideally suited for a sliding puzzle game.

How it Works

PyBadge showing Santa image

Complete image for Santa puzzle

PyBadge showing scrambled puzzle

PyBadge showing scrambled puzzle

PyBadge showing puzzle solution

PyBadge showing solved puzzle

After initial setup and bookkeeping, the program has an infinite while loop. It roughly follows a state machine pattern with the the states being “intro”, “setup”, “play”, and “solved”.

“intro” displays the puzzle image and then asks the player to select either a 3×3 or 4×4 puzzle. Once a selection is made, the state transitions to “setup.” In this state, the puzzle is scrambled and the scrambled puzzle is displayed. Then the state transitions to “play.” In “play”, the program monitors the up, down, left, and right buttons and moves the tiles accordingly. After each move, the puzzle is checked to see if it is in the solved position. If so, the state transitions to “solved”. Once in the “solved” state, the program displays a “You Won” message, then the full image. It then waits indefinitely until the player either presses START to go back to the “intro” state and play again or turns the badge off. The software is open source and published on GitHub.

A Note on Solvability

If one scrambles the puzzle by starting with the solution and then making random allowed moves the puzzle can always be solved. However, with this approach, it takes many, many such movements to really randomize the puzzle. If, instead, one simply places each tile at random, it turns out that only half of the possible arrangements can be slid back to the solution. Borrowing from what others have done, my code chooses a totally random arrangement, then checks if it is solvable (see the “solvable” function in the code). If not, it randomizes the puzzle again, and repeats this process until a solvable arrangement is found and used. The rules for solvability are:

  1. If the grid width is odd (e.g., 3×3), then the number of inversions in a solvable situation is even.
  2. If the grid width is even (e.g. 4×4), and the blank is on an even row counting from the bottom (second-last, fourth-last etc), then the number of inversions in a solvable situation is odd.
  3. If the grid width is even, and the blank is on an odd row counting from the bottom (last, third-last, fifth-last etc) then the number of inversions in a solvable situation is even.

The plastic slider puzzle shown at the top of this post is not solvable. The blank is on an odd row from the bottom (the first), and there is just one inversion, an odd number.

How to Play the Game

To play, simply load the software onto the PyBadge and turn it on. The display will first show the complete puzzle image, and then will ask you to press the “A” button for a 3×3 (8 tile) puzzle or the “B” button for a 4×4 (15 tile) puzzle. One slot is always empty so that tiles can be moved. Once you have made your selection, you’ll see the puzzle image with the pieces, in the solved state, and then the puzzle will be scrambled for play. The 4×4 puzzle is significantly harder than the 3×3 puzzle and will take more moves to solve, but both are fairly easy with practice.

Use the 4 directional buttons to slide a tile at a time. The goal is to get the tiles arranged in numerical order, left to right and top to bottom, with the empty spot on the bottom right. Once you have done this, you win! After the complete image is displayed upon winning, you can press the START button to play again. Sometimes you need to press the button a couple of times.

Stuck? There are a number of heuristics for humans to use to solve these puzzles (as well as heuristic algorithms for computers). The approach that works for me is documented here.

Changing the Puzzle Images and Creating Your Own

The parameters.py file stores several parameters, including the name of the folder where the puzzle images are stored. To change, for example, from the Santa to the witch puzzle, simply edit the line: puzzle_graphics_folder = “santa” to puzzle_graphics_folder = “witch”. I have provided three sets of images for the puzzle: Santa, a witch, and a Valentine’s Day floral image:

Flowers and "Happy Valentine's Day" text.

 

Santa, sleigh, and reindeer

 

Flying witch in front of moon, with bat.

To make your own puzzles, you need to create 3 bmp images:

  • The full image, to be saved as “full.bmp” in a new folder
  • A tile image for the 3×3 puzzle, to be saves as “tiles3.bmp” in the same folder
  • A tile image for the 4×4 puzzle, to be saved as “tiles4.bmp” in the same folder. .

These images must be exactly the right size in order for the program to work. The full image and 4×4 tile image must be 160 pixels wide by 128 pixels high. The 3×3 itile image must be 159 pixels wide by 126 pixels high.

Start from the full image. To make the 4×4 tile image, black out the pixels on the lower right of the image (x coordinates 121 – 160, y coordinates 96 – 128). You can also put numbers on what will be each tile to make solving the puzzle easier To do this, I use an image editing program to add a layer with a set of grid lines creating a 4×4 grid. Then I blacken out the lower right tile and put numbers in the upper right of each tile. Then I delete the grid layer and save the image as a bmp file. Follow the same process for the 3×3 tile image, but first re-scale the total image to be 159 x 126 and use a 3×3 rather than 4×4 grid. Once you have saved the three files in a new folder, change the puzzle_graphics_folder line in the parameters.py program to point to the new folder name.

 

Multi-Function PyPortal Information Display

Pyportal front viewAdafruit’s Pyportal is a pretty powerful little device. One of it’s main functions are to grab and display data from internet APIs, but it can also easily log data to the cloud using Adafruit IO, has a couple built in sensors, and can be used to play games and perform other functions. Per Adafruit’s overview page, “the PyPortal uses an ATMEL (Microchip) ATSAMD51J20, and an Espressif ESP32 Wi-Fi coprocessor with TLS/SSL (secure web) support built-in. PyPortal has a 3.2″ 320 x 240 color TFT with resistive touch screen. PyPortal includes: speaker, light sensor, temperature sensor, NeoPixel, microSD card slot, 8MB flash, plug-in ports for I2C, and 2 analog/digital pins.”

It’s called the  Pyportal because it provides a portal (window) to information on the Internet and it’s programmed in CircuitPython, which is Adafruit’s fork of MicroPython and is an implementation of the Python programming language for microcontrollers.

There are a lot of examples of single-function applications available, such as weather, air quality, event count-down clock, and even how many astronauts are currently in space, along with their names. What I wanted to do was to have the device cycle through multiple applications. I ended up coding it to provide

  • The day, date, and time (inspired by how cruise ship’s change out a piece of carpet every day in the elevators to tell you what day of the week it is),
  • The current local weather,
  • The latest “shower thought” post from the shower thoughts subreddit, and
  • The current value and change from the previoius day for the S&P 500.

The project ended up to not be quite as easy as I thought it would be, but still pretty easy. I ended up restructuring a lot of the examples code to work better with scrolling through multiple functions, but the final result still borrows heavily from the examples previously posted online.

In this post, I won’t repeat the basics for getting up and running, which can be found online, but I will do a brief walkthrough of my code, which is posted on gitHub. To use it yourself, you’ll need to edit the “secrets.py” file to provide your WIFI login information as well as sign up with the API providers and put your personal keys or ID in that file. You will also want to provide the location id for your location for the local weather, rather than mine, in the main program.

Overview (code module)

The program consists of a main module that first takes care of initial setup, making the wifi connection, and initializing variables, for example, the location ID for the local weather and the URLs for the various data sources. Then it launches an infinite loop to go through and access and display the various types of data. In each case, there is a timer so that the URLs are not constantly being visited each loop.

For the weather and S&P 500 applications and initial background screen is loaded and briefly displayed. This changes to a different background depending on the current data. For Day / Date/ Time and for Shower Thoughts the appropriate background screen is initially loaded and remains the same while that app’s data is displayed.

For each application, the program checks to see if it’s time to get fresh data from the internet (which occurs the first time the app runs and on a fixed schedule after that. If it does, it updates the old raw data from the internet. If not, the old raw data is kept. Then the program calls an application-specific python module t that extracts the relevant data from the JSON, formats it, and returns a text_group object for display. text_group is part of the displayio library for CircuitPython. I encourage interested readers to read the linked guide to the displayio library, as it took me some time to figure out how all the pieces fit together.

The display is then updated by appending the text group to splash attribute of the previously created Pyportal instance. splash is a displayio Group that is automatically displayed on the screen. After a delay to allow the update to remain on the screen for the specified number of seconds the Pyportal splash.pop() method is called to remove the text_group that had been added. If this were not done, Pyportal.splash would just keep getting bigger and bigger as each app runs over and over through the endless main loop.

I could probably refactor the main loop to make some things more generic and then move them into a common function call, rather than having so much separate for each application.

Day / Date / Time (day_graphics module)

Day, date, and time display on the PyPortal

The PyPortal does not have an RTC, so while it has an internal clock function, it tends

to drift. However Adafruit provides their own time service to call and update the local time. This function is called once per hour to update the local time.

The day_graphics module parses out the date and time information from time.localtime() to extract the day of the week, day of the month, the month, and the year. It puts this in the desired string formatting and specifies the font and location on the screen for each displayed element.

Weather (openweather_graphics module)

Current Weather Displayed on the PyPortal

For the weather, I chose to use openweathermap.org as the source for weather data and base my code on Adafruit’s PyPortal Weather Station example. That site has the local weather for my location and also has nice background icons for each weather type and whether it is daytime or nightime. There are several ways to specify the location you want the weather for, and various types of information available. This is all explained on their website. The example cited above used cityname and country, but that is ambiguous for my location, so I used their ID code for my location, which is unique.

The processing is very similar to the day/date/time app, although there is some additional work to convert temperature, which the API returns in degrees Kelvin, to Fahrenheit.

The icons are not downloaded using the API, but rather are stored on the PyPortal. The API returns the id number of the icon to be displayed. For this application, two items are returned from the function call: the text_group and the file name for the background image.

Shower Thoughts (st_graphics module)

A “shower thought” displayed on the PyPortal

Shower Thoughts is a subreddit for “sharing those miniature epiphanies you have that highlight the oddities within the familiar.” Two recent examples are Taking notes in history class is rewriting history and You’d think the generation that grew up playing Oregon Trail would have more respect for some of these diseases. There is a JSON API to get these posts and I decided that it would be fun to have these cycle through.

The concept for this app is similar to the others, and in fact, only one field (the title) needs to be extracted from the JSON field (one of the rules of that subreddit is that the entire thought must fit in the title). However, the thoughts can still be rather long. If I used the small sized font, they generally appear rather small, but if I use the medium ones I have to truncate the thought n the middle. So the program checks the length, and uses the medium sized font if the message is short enough and otherwise uses the small sized font.

S&P 500 (market_graphics module)

The S&P 500 value displayed on the PyPortal

The final application that this cycles through is getting the current value of the S&P 500 and comparing it to the previous day’s close. The data is obtained using a free API provided by Alpha Vantage.

Again, the processing is very similar. The customized features for this app is that the color of the amount and percent change is set as either green or red, depending on whether or not the index is up or down, and the background image is similarly selected to have either an upwards green arrow or a downwards red arrow.

Other Notes

The Pyportal is designed so that when something is written to it from a PC it does a soft reboot and starts running the main program. This is to make it really easy to do updates. The problem some folks have had, including myself, on a Windows 10 machine, is that it constantly does this soft reboot when plugged into either of my PCs. This doesn’t happen when it’s just plugged into a power source, then it runs fine. Apparently something keeps trying to write to it. As a work around, one can go into the REPL and type:

import supervisor
supervisor.disable_autoreload()

This turns off the auto-reload and soft reboot feature. Once that is done, one needs to stop a running program (type CTRL-C in the serial window) and then reload and restart by typing CTRL-D in the serial window. If anyone has a permanent fix for this problem, I’d love to hear it.

Sometimes the ESP32 board seems to have glitches, and an error condition is created and valid data is not returned. The try / except code in the main program just skips out of the application where that occurs and tries the next one. So far, this has caught and provided a work-around for all but one error, which popped up once after the application had been running for several days.

Summary

The PyPortal is a great little device that is capable of doing a lot (this example didn’t make use of the touchscreen features, built-in sensors, or i/o lines). Hopefully this example will be useful to others, especially if you want to have the PyPortal perform multiple functions, rather than only serving a single purpose at a time. Again, the code is available on gitHub.