Friday, April 22, 2016

Using UI automation to export KiCad schematics

This is my third post in a series about the open source split-flap display I’ve been designing in my free time. I’ll hopefully write a bit more about the overall design process in the future, but for now wanted to start with some fairly technical posts about build automation on that project.

Posts in the series:
Scripting KiCad Pcbnew exports
Automated KiCad, OpenSCAD rendering using Travis CI
Using UI automation to export KiCad schematics

Since I’ve been designing the split-flap display as an open source project, I wanted to make sure that all of the different components were easily accessible and visible for someone new or just browsing the project. Today’s post continues the series on automatically rendering images to include in the project’s README, but this time we go beyond simple programmatic bindings to get what we want: the schematic!

"Wow, I bet someone had to manually click through the GUI to
export such a beautiful schematic!" Nope.

Unfortunately, KiCad’s schematic editor, Eeschema, doesn’t have nice Python bindings like its pcb-editing cousin Pcbnew (and probably won’t for quite some time). And there aren’t really any command line arguments to do this either. So we turn to the last resort: UI automation. That is, simulating interaction with the graphical user interface.

There are two main issues with automating the graphical user interface: the build system (Travis CI) is running on a headless machine with no display, and the script needs to somehow know where to click on screen.

As I mentioned in my last post, we can use X Virtual Framebuffer (Xvfb), which acts as a virtual display server, to solve the first problem. As long as Xvfb is running, we can launch Eeschema even when there’s no physical screen. This time, instead of using `xvfb-run` from a Bash script, I decided to use the xvfbwrapper Python library for additional flexibility. xvfbwrapper provides a Python context manager so you can easily run an Xvfb server while some other code executes.

from xvfbwrapper import Xvfb
with Xvfb(width=800, height=600, colordepth=24):
    # Everything within this block now has access
    # to an 800x600 24-bit virtual display
    do_something_that_needs_a_display()


So how do we actually script and automate interactions with the GUI, such as opening menus, typing text, and clicking buttons? I looked into a number of different approaches, such as Sikuli, which allows you to write high level “visual code” using screenshots and image matching, or Java’s Robot class which lets you program the mouse and keyboard using Java, but the easiest option I found by far was the command-line program xdotool.

With xdotool, you can easily probe and interact with the window system from the command line. For instance, you can output a list of all named windows by running:
xdotool search --name '.+' getwindowname %@

(This is an example of a chained command: the first part (search --name '.+') finds all windows whose name matches the regular expression ‘.+’ (any non-empty string) and places those window ids onto a stack. The second part runs the command getwindowname, with the argument %@ meaning “all window ids currently on the stack.”)

Going back to Eeschema, the option we want to automate (exporting the schematic) lives under the File → Plot → Plot menu. The trick to automating this is not to use the mouse to click (since then we’d need to know the coordinates on screen) but instead use keyboard shortcuts. Opening that menu from the keyboard just requires pressing “Alt+F” then “P” then “P”, which we can automate like this:

# First find and then focus the Eeschema window
xdotool search --onlyvisible --class eeschema windowfocus
# Send keystrokes to navigate the menus
xdotool key alt+f p p



We can similarly write commands to fill out the correct information in the “Plot Schematic” dialog once it opens. To change radio button selections, we can “Tab” numerous times to move focus through the various options. This is a bit fragile, since it relies on there being a stable set of options in the same order to work (and might break if KiCad were to add a new Page Size option, for instance), but is about the best we can do without using more complex UI automation tools.


To make it easier to debug what’s happening in the X virtual display, we can use a screen-recording tool like recordmydesktop to save a screencast of the graphical automation. This is particularly helpful when running on Travis where you can’t actually see what’s going on as the script runs.

Since we’re writing in Python, we can use some syntactic sugar with Python context managers to make it really easy to wrap a section of code with Xvfb and video recording. As a first step, we’ll need a context manager for running a subprocess:

class PopenContext(subprocess.Popen):
    def __enter__(self):
        return self
    def __exit__(self, type, value, traceback):
        if self.stdout:
            self.stdout.close()
        if self.stderr:
            self.stderr.close()
        if self.stdin:
            self.stdin.close()
        if type:
            self.terminate()
        self.wait()


and then we can create a macro that combines both an Xvfb context and a recordmydesktop subprocess context into a single context manager to be used together:

@contextmanager
def recorded_xvfb(video_filename, **xvfb_args):
    with Xvfb(**xvfb_args):
        with PopenContext([
                'recordmydesktop',
                '--no-sound',
                '--no-frame',
                '--on-the-fly-encoding',
                '-o', video_filename], close_fds=True) as screencast_proc:
            yield



You can use that macro like so:
with recorded_xvfb('output_video.ogv', width=800, height=600, colordepth=24):
    # This code runs with an Xvfb display available
    # and is recorded to output_video.ogv
    do_something_that_needs_a_display()

# Once the 'with' block exits, the X virtual display is
# no longer available, and the recording has stopped
run_non_recorded_things()



So, putting all of those elements together, we can use Xvfb to host the Eeschema GUI (even on a headless build machine), run recordmydesktop to save a video screencast to help understand and debug the visual interactions, and use xdotool to simulate key presses in order to click through Eeschema’s menus and dialogs. The code looks roughly something like this:

with recorded_xvfb('output.ogv', width=800, height=600, colordepth=24):
    with PopenContext(['eeschema', 'splitflap.sch']) as eeschema_proc:
        wait_for_window('eeschema', ['--onlyvisible', '--class', 'eeschema'])
        # Focus main eeschema window
        xdotool(['search', '--onlyvisible', '--class', 'eeschema', 'windowfocus'])
        # Open File->Plot->Plot')
        xdotool(['key', 'alt+f', 'p', 'p'])
        wait_for_window('plot', ['--name', 'Plot'])
        xdotool(['search', '--name', 'Plot', 'windowfocus'])

        [...]

        eeschema_proc.terminate()



This is what one of those recordings looks like:


You can find the full scripts in the github repo, particularly in these two files:
/electronics/scripts/export_util.py
/electronics/scripts/export_schematic.py

I also used a similar technique to export the component list .xml file (Tools → Generate Bill of Materials) which is then transformed into a .csv bill of materials:
/electronics/scripts/export_bom.py

Hopefully this was a useful overview of how I used UI automation to export schematics from KiCad. If you have questions, leave a comment here or open an issue on on github and I’ll try to respond. In my next post in this series I’ll switch gears a bit and talk about how I programmatically generate the OpenSCAD 3d animation you see at the top of the project’s README!

No comments:

Post a Comment