Pressbooks 5.6.3, Mcluhan 2.6.1, and WordPress 5.0

We just finished our last releases of 2018, Pressbooks 5.6.3 and McLuhan 2.6.1. We’ll be deploying them to all hosted networks at the start of next week. These are minor bug fix releases which improve shortcodes, a focus style regression in McLuhan, and a bug with Hypothesis highlights which was introduced in McLuhan 2.6.0.

A note about WordPress 5.0 "Bebo": we’ve tested Pressbooks with WordPress 5.0 in our development and staging environments, and we are confident that open source users can proceed to update their networks to WordPress 5.0 without any disruption. As we previously wrote, we’ve disabled Gutenberg for the time being, so the editing experience will not be affected. We will be waiting to update our hosted networks until January.

Refactoring Slow Forms Using PHP Generators and Event Streams

The form will still be slow but the user experience will be better. The user will see a progress bar and see status updates in real time. The idea is to refactor something like this:

/**
 * A task that takes way too loooooooooooooooooooooooong...
 */
function task() {
    step1();
    step2();
    step3();
    //  
    step100();
} 

Into this:

/**
 * Yields a key/value pair
 * The key is between 1-100 and represents percentage completed
 * The value is a string of information for the user
 *
 * @return Generator
 */
function taskGenerator() {
    step1();
    yield 1 => 'Completed step 1';
    step2();
    yield 2 => 'Completed step 2';
    step3();
    yield 3 => 'Completed step 3';
    //  
    step100();
    yield 100 => 'Completed step 100';
}
/**
 * A task that takes way too loooooooooooooooooooooooong...
 */
function task() {
    foreach ( taskGenerator() as $percentage => $info ) {
        // Do nothing, this is a compatibility wrapper  
        // that makes our generator work like a regular function  
    }
}

And this:

let evtSource = new EventSource(url);

Before

Currently, cloning a book in Pressbooks looks like this:

Spinning beach ball of death.

The user clicks submit. They wait, and wait, and wait. The task completes and they receive some feedback saying "everything seems fine". It’s not particularly pleasant but it does the job. Typical open source WordPress installs, and by extension Pressbooks, don’t have the resources, infrastructure, or competence to setup job queues and delegate these kind of tasks into the background. Everyone lives with it. The end.

Insert cliché "What if I told you" meme here.

After

It is possible to use PHP Generators and Event Streams to provide real-time feedback to the web browser.

Screen shot of the EventStream console in Google Chrome.

The EventStream console in Chrome Browser.

With a bit of refactoring cloning a book in Pressbooks, instead, looks like this:

A status bar with real-time status updates.

The heavy lifting is done by an EventStreams class (Source code).

On the front end, the main changes were from this:

<input type="submit">  

To:

<p><input type="submit"></p>  
<div id="pb-sse-progressbar"></div>  
<p id="pb-sse-info"></p>  

And some JavaScript:

$( '#pb-cloner-form' ).on( 'submit', function ( e ) {
    e.preventDefault();
    $( '#pb-cloner-button' ).attr( 'disabled', true );
    let form = $( '#pb-cloner-form' );
    let eventSourceUrl = PB_ClonerToken.ajaxUrl + (PB_ClonerToken.ajaxUrl.includes( '?' ) ? '&' : '?') + $.param( form.find( ':input' ) );
    let evtSource = new EventSource( eventSourceUrl );
    evtSource.onopen = function () {
        $( '#pb-cloner-button' ).hide();
    };
    evtSource.onmessage = function ( message ) {
        let bar = $( '#pb-sse-progressbar' );
        let info = $( '#pb-sse-info' );
        let data = JSON.parse( message.data );
        switch ( data.action ) {
            case 'updateStatusBar':
                bar.progressbar( { value: parseInt( data.percentage, 10 ) } );
                info.html( data.info );
                break;
            case 'complete':
                evtSource.close();
                if ( data.error ) {
                    bar.progressbar( { value: false } );
                    info.html( data.error );
                } else {
                    window.location = PB_ClonerToken.redirectUrl;
                }
                break;
            default:
                break;
        }
    };
    evtSource.onerror = function () {
        evtSource.close();
        $( '#pb-sse-progressbar' ).progressbar( { value: false } );
        $( '#pb-sse-info' ).html( 'EventStream Connection Error' );
    };
} ); 

(Source code)

The JavaScript (and jQuery) snippet:

  • Targets a form with id pb-cloner-form
  • Stops the form from submitting and instead
  • Appends all the form data as $_GET parameters to an ajax URL then
  • Passes that ajax URL to a new EventSource
  • Updates pb-sse-progressbar and pb-sse-info when it receives an event stream message
  • Redirects the user back to where they started when complete

On the back end, the time consuming function was refactored into a generator that yields a key/value pair. The key is between 1-100 and represents percentage completed. The value is a string of information meant for the user. Once you have a generator that follows this convention, pass it to the EventEmitter. The browser will start receiving an event stream.

/**
 * @return Generator
 */
function loooooooooooooooooooooooongGenerator() {
    // Pseudo code
    yield 1 => 'Looking up the source book';
    sleep( 2 );
    yield 10 => 'Creating the target book';
    sleep( 2 );
    // ..
    sleep( 2 );
    yield 100 => 'Done';
}
$emitter = new \Pressbooks\EventStreams();
$emitter->emit( loooooooooooooooooooooooongGenerator() );

(Source code).

Key ideas:

  • It’s not necessary to wait until the request finishes, PHP can emit event-stream responses (SSE) back to the web browser while it is working on something.
  • PHP Generators are a relatively simple refactoring hack to get those responses back to the browser.
  • sleep() is only meant as an example of a function call that takes a long time to finish, don’t put sleep in your production code, you already knew this, I hope?

Originally posted on my personal blog in a more generic format, with references to the movie Office Space, but it turns out it was me, doing research for Pressbooks, the whole time.

WIP: https://github.com/pressbooks/pressbooks/issues/1282

Pressbooks and Gutenberg

We’ve been closely following the development of Gutenberg, WordPress’ new block-based editor, since its earliest technical prototypes. With the impending release of Gutenberg in WordPress 5.0 (currently scheduled for November 19), we want to provide an update to the Pressbooks community on our plans for Gutenberg integration.

In brief: we have deactivated Gutenberg for Pressbooks users, so upgrading to WordPress 5.0 will not change your editing experience. We’ve done this in Pressbooks itself, so additional plugins like Classic Editor are not required. We’ve made this decision for several reasons.

1. Gutenberg has continued to change significantly up to the projected release date, without a sufficient period of API and UI stability.

At Pressbooks, our small development team accomplishes a lot with limited resources, but ensuring compatibility with Gutenberg in time for its November 19th launch has proven to be beyond our capacity. The lack of clarity in Gutenberg’s development process has hindered us from integrating Gutenberg into our roadmap. We are now two weeks from its production release, and Gutenberg’s API freeze is not yet complete. We’ve been tracking blocking issues over the last year and a half and have tried to contribute where possible, but ongoing API and user interface changes have made it difficult for us to keep on top of things without neglecting Pressbooks core development, and have made us hesitant to invest our limited resources in building on a codebase that has not yet stabilized. Once WordPress 5.0 is released, we will be able to evaluate a stable version of Gutenberg and map out a plan for integration.

2. Gutenberg lacks functionality that Pressbooks relies upon.

Because Pressbooks is used to produce content for the web and for export, we leverage WordPress custom post statuses to determine whether chapters and other content appear on the web, in export files, or both. (Custom post statuses do not have a full-featured public API in WordPress, but they were the best solution for our use case. This is not the fault of the Gutenberg team—arguably, it’s our fault—but that’s where we are.) We also use a custom post submission UI to more clearly reflect content status for our users. At this point in time, Gutenberg does not support custom post statuses, which is a fundamental issue for us (and others who leverage this functionality). Furthermore, the new post publish panel is not easily modified, and we feel that it’s important to maintain continuity of this interface for our users. Following the release of WordPress 5.0, we will engage on these specific issues with the WordPress and Gutenberg development teams so that we can provide our users with a familiar experience in this area.

3. Gutenberg does not meet Pressbooks’ accessibility standards.

WordPress Accessibility team lead Rian Rietveld’s resignation in early October and the team’s recent report on Gutenberg have brought widespread attention to the accessibility shortcomings of the Gutenberg project. These issues are not new, as some have implied; Rietveld documented many in March of this year, and tests of Gutenberg by Léonie Watson and Sina Bahram highlighted significant problems around the same time. Rietveld’s resignation, the team’s report, and the conversations surrounding them have highlighted flaws in the Gutenberg development process and have underscored how hard it is to retrofit an existing interface if it was not designed with accessibility and inclusivity in mind. We’re currently making accessibility improvement to elements of our core UI that date back to the early days of our product—so we know this from firsthand experience! More importantly, though, we’re continually seeking to expand our own shared understanding of accessibility and inclusive design principles so that new features are inclusive from day one. We’re grateful to Jess Mitchell, Jonathan Hung, and the rest of the team at OCAD’s Inclusive Design Research Centre for their guidance as we work towards these goals.

We acknowledge the assessment of the WordPress Accessibility team, who write that “based on [Gutenberg’s] current status, we cannot recommend that anybody who has a need for assistive technology allow it to be in use on any sites they need to use at this time.” We’re grateful to the many accessibility experts and advocates who have shone a spotlight on accessibility in WordPress, particularly the volunteers who form the WordPress Accessibility team. We’re also grateful to Rachel Cherry and the WPCampus organization, who are funding a much-needed accessibility audit of Gutenberg. As our focus is on continuously improving Pressbooks’ accessibility and all signs suggest that Gutenberg would undermine this goal in its current state, we will be awaiting the results of WPCampus’ audit following the release of WordPress 5.0, and we will evaluate our next steps at that time.

Closing Thoughts

There are parts of the Gutenberg project that we’re pretty excited about. The ability to add and edit complex elements visually could be extremely helpful for book production, allowing educational textboxes and other specialized content to be edited visually while ensuring clean, semantic output on the front end. We’d love to be able to allow shortcodes to be edited visually, which Gutenberg supports. And we’re eager to see what comes of Gutenberg’s Annotation API, which could support expanded integrations with tools like Hypothesis. However, knowing what we know at this point and for the reasons articulated above, we feel that a “wait and see” approach is most prudent for us and our users. We are committed to tracking each of these issues and doing what we can to ensure that authors using our software have the best available tools for the job at hand. We look forward to a future where Gutenberg can be used for making books.