UncleGoose: Accessible Blocks-Based Programming

Client: Bootstrap
Date: December 2016 through June 2021
Services: Emerging Technologies, Spot Checks, Consulting Hours
The WeScheme Documentation interface for a project named "Emmanuel and Lauren's Game." At the top are toolbar buttons such as Run, Stop, and Recipe. On the left is a searchable list of programming functions, including math, string, and image-related functions like +, sqrt, string-append, circle, rectangle, and overlay. In the main workspace, a program is being edited using visual block-style code. A section titled "3. Get our Player moving!" defines a function named update-player with parameters y and key. The function uses a cond block to compare the pressed key with strings such as "up", "down", "w", and "s", then adjusts the player's y-coordinate by adding or subtracting values like 20 or 40. A tooltip on the right explains the function signature: update-player : Number String -> Number, describing it as taking the player's y-coordinate and a direction to output the next y-coordinate.

PAC collaborated with Bootstrap to improve the accessibility of CodeMirror-Blocks, producing a language-agnostic CodeMirror wrapper that lets blind and low-vision students navigate, edit, and understand code through tree-based structure and natural-language descriptions alongside their peers.

Media

Project Description

The code of a computer program is not structured like an essay. Code is a hierarchical, tree-shaped structure, more like an outline or sentence diagram. A particular part of a program is often best understood by its relationship to other parts, such as an expression inside a function definition, rather than only by its line or column number. Yet most programmers still write code by editing lines and columns of text.

Modern editors help programmers perceive that underlying structure through visual cues such as syntax highlighting, bracket matching, and auto-indenting. These features help reveal the tree behind the text and are now considered essential in modern coding environments. For programmers using screen readers or refreshable braille displays, however, many of those visual cues are unavailable or difficult to access. This can leave users working primarily with raw syntax, which makes it harder to understand program structure, maintain orientation, and move efficiently through code.

PAC wanted to help bridge this gap, and we were delighted to work with Bootstrap on the accessibility of CodeMirror-Blocks (CMB), which addresses this problem in two important ways:

  • CMB allows users to navigate and edit the tree structure of the code itself, rather than relying only on lines and columns of textual syntax. Prior work showed that tree-based navigation can significantly improve comprehension and orientation for screen reader users, and PAC’s user testing of CMB reflected similar results. The ability to edit the tree directly has the potential to extend those benefits even further.
  • CMB allows code to be described in natural language rather than simply read aloud as raw syntax. For example, function f(x) { return x + 2; } could be spoken as “f, a function definition with one input: x.” When the user navigates into the function body, it could be described as “return an addition expression, of x and 2.” The exact wording can be adapted for the audience, including child-friendly terminology or non-English descriptions. The key innovation is that code can be described structurally and semantically, rather than merely pronounced character by character.

CMB is designed to be language-agnostic, allowing others to write parsers for the programming languages they need. It is implemented as a wrapper for the popular CodeMirror JavaScript library, allowing IDEs that use CodeMirror to incorporate the accessibility benefits of CMB with minimal changes to their codebase. CMB is already in use by Bootstrap, one of the largest providers of in-school computer science education. With CMB, schools using Bootstrap can include blind and low-vision students in the classroom, working side by side with peers through a coding environment designed to support multiple ways of understanding and editing code.

This work is a collaboration between Bootstrap and Prime Access Consulting and would not be possible without the support of the National Science Foundation, the Entertainment Software Association, and Vint Cerf. We are especially grateful to AccessComputing for their guidance, support, and community outreach. To learn more, check out our paper on this work.