The guidelines for this module were pretty simple. We were tasked with creating some kind of kinetic sculpture using servos and steppers. In planning out my project I was heavily inspired by one of the example devices we were shown in class, the Brachiograph, an easy to build drawing robot. While having a machine perform such an inherently human activity as drawing is cool in and of itself, I thought that it could be taken a step further. Instead of the robot pulling drawings from a collection of presets, I figured it could benefit from an extra element of creativity, producing entirely new drawings on the fly with each interaction. And so, Salvador was born, a robot capable of interpreting voice commands and creating original drawings with a physical pen. Basically, you speak to Salvador and ask him to draw you something, and then he goes ahead and does it. His work may be abstract, but he’s trying. Here’s a hypothetical scenario illustrating a typical interaction with Salvador:
User: Hey Salvador! Draw me a detailed pencil sketch of a bunny rabbit sitting in the grass.
Salvador: thinks about it… and draws relatively meaningless squiggles that could probably be interpreted as a verdant meadow if properly squinted at
It’s not all bad, though. Salvador practically excels at capturing certain subjects, like trees:
And he does a lovely interpretation of the Eiffel Tower:
Really though, for a proof of concept, this setup works remarkably well. The images generated by Dalle are often impressively accurate, and contain a level of detail that the wobbly pen plotter is unfortunately unable to capture. Paired with a more robust physical design, it could really make some interesting pieces. However, it’s still pretty neat as is, and you can usually tell where it’s going with things.
The Software
This project involved a lot of moving parts, both metaphorically and physically. On the software side, there are a few different APIs and libraries being called to support all the different features. There are two voice recognition libraries, one being the Picovoice wake word detection library Porcupine, and the other being the Google Speech Recognition service. Porcupine is able to constantly listen for the wake word “Hey Salvador!” on device, and then once it detects it, the audio that follows gets sent to Google to be processed with a high degree of accuracy. Once the user’s speech has been converted to text it gets further processed before being sent to Dalle. Using an NLP library called SpaCy, the user’s sentence is parsed and separated into different grammatical categories, so that the direct object can be extracted. This allows you to speak more naturally to Salvador, and say things like “Hey Salvador, draw me a cat” instead of the clunky “Hey Salvador, a cat”. Once the object of the verb has been determined, it’s sent along to the Dalle API (just released last week), which generates a 1024x1024 pixel image from the prompt. The prompt you give it can really be as complicated as you want, as long as you can fit it into a single sentence. For the best results, it’s often worthwhile to describe the style you’d like Salvador to draw in, such as “detailed pencil sketch” (useful because it creates simple forms on a mostly clear background). Once the image has been generated, it has to be converted into a line drawing through a library supplied by the people behind the Brachiograph, which can then then be turned into motor movements on the plotter using more provided code.
The Hardware
The core of the hardware design is just the Brachiograph plotter described at: https://www.brachiograph.art/tutorial/index.html. Following this tutorial, using a couple of popsicle sticks, three servo motors, and a clothespin, you can make a simple pen plotter. On the same site, they provide all the code necessary to do the math to turn an image into movements for the motors. There were a few things I added to the design to support the extra features of Salvador, the main one being an off-the-shelf USB microphone. The box I made for it also has a front panel with an indicator light, several buttons for controlling the device, and a stepper motor acting a a sort of loading spinner so that the user has some feedback while Dalle is doing its generation.
The Power Supply
I’m using the same buck converter that was in the last project, which continues to make setup pretty straightforward. I had originally planned to power everything through it, using the 12v power supply that we were given, but I ran into issues with the PI. While I was careful to power the pi through its USB port so that it would have at least a little power protection, it seemed to brown out about 30 seconds after startup. This is presumably about the time that the servos became active, and the load of everything at once was just too much for the converter to handle. I hadn’t realized before then that the tiny micro-servos were capable of pulling a whole half an amp under load, which meant I had to change my design. The result isn’t that elegant, but it was the best I could come up with. The motors are powered by the buck converter, and the pi is just powered by the USB-C supply that it came with. At the end of the day, I just got worried about messing up the pi, and wanted to be sure its power was as stable as possible.
Relay Tangent
A small digression: While I was still planning on powering everything together I came up with a way of safely shutting down the pi while still having power controlled by one switch. I ended up having to scrap it because of the aforementioned problems, but I’ll just describe it quickly because it was pretty cool. For context, if you simply were to put a switch between the pi and its power supply, you could safely turn it on, but turning it off would be another matter. The pi needs to be warned that power is going to be cut so it can do all of its preparatory shutdown things. If you cut the power prematurely, you run a risk of corrupting the SD card. This means you need the pi to be listening on a pin for the signal from the switch, so that it can handle the shutdown itself. However, the pi can’t be listening at startup, because it’s of course already off. So, this leaves us with the question of how to make a switch that controls power at the circuit level at startup, but then only operates at the software level for shutdown. The idea I came up with was to have the switch control a relay that acts as a sort of “latch”. To turn the device on, you flip the switch, which provides power to the relay, which subsequently powers the pi. Once the pi is on, it also provides power to the relay, so even if the switch gets turned off, power won’t be cut. Since its power is assured, its free to listen for the switch being thrown on one of its pins, and then initiate the required shutdown procedure. This ended up working pretty well, but like I said, I was forced to scrap it for other reasons. Instead, I now have a simple dedicated shutdown button.