By: Bee
Spec level:
Last updated: 2021-07-09
Gissues: #1449, #1442
In which we describe how we’re integrating dynamic javascript graphs (built with D3.js and currently hosted at graph.beeminder.com) into Beeminder’s web interface.
We have a sweet javascript graphing library which we run serverside as an external service to generate .png and .svg graph images for Beeminder goals.
Which is great.
But you know where javascript likes to be?
In your browser.
We have identified four steps on the way to the glorious dynagraph future that everyone has harbored in their greatest expectations of Beeminder since 2013.
Potential, baby. We are so going to live up to it.
These four stages will be to integrate first into goal setup, then update the graph editor interface to replace the beta interface at
graph.beeminder.com. This second set of changes, to graph editing, will actually be broken into two stages:
first changing over the admin interface, which will serve as an alpha-testing run, then swapping in the javascript graph editor for the user’s static graph editor.
The fourth and final piece will integrate the dynagraphs into the goal page, replacing the static .svg images with interactive graphs.
One place that having a real-time-dynamic graph reflecting your changes would be helpful, is in setting up a new goal. Make things more WYSIWYG and hopefully users will be less surprised with what they get once the goal is set up. Goal setup is also one of the areas of the site that we have fairly extensive automated test coverage on, and it is quite circumscribed from other things that threaten to start getting complicated (unlike goal page interactive graphs which have to work out how to add data, and pass deadlines, and update settings etc etc…), thus making it a good candidate for the starting point.
[TODO: insert a use-case / description of a user actually using goal creation in here]
We are going to integrate the dynagraphs by adding an additional step to goal setup. This is not the ideal integration of Beeminder goal setup and interactive graphs. An ideal dynagraph-in-goal-setup integration would insert the dynagraph into the layout so that it is always visible as you step through each step. On Step one you’d set your units and rate, and see it instantly in the dynagraph. Step two you’d name it and voilá, the graph shows it. Set starting pledge — graph updates, initial safety buffer — graph updates. What’s that? You got the picture back on “insert the dynagraph into the layout”? Oh.
But that kind of sexiness is outside the scope of the integration at this point for two reasons. The main one is a design issue. The way the goal wizard is designed right now it is fairly compact in terms of horizontal width, and in height. This was done in place of true responsive design. Because of this physical layout, it is manageable to create goals on mobile; the layout looks pretty much the same as on a regular computer screen. To find a way to insert a view of the graph into this wizard layout without breaking the mobile-friendliness of it is no small matter, and will require a more extensive css / design jaunt than this document wants to take.
The second reason is more related to the technical implementation. At the moment the goal setup wizard is an elaborate snarl of javascript that is all highly dependent on the ids and classes of divs in the DOM etc, and thus feels like a bit of a fragile glass house for us to start throwing new javascript library stones at. At some point in the possibly near future we want to move in the direction of some of the new reactive design hotness (vue.js or more likely hotwire), at which point we can redesign things to our hearts’ content in order to make a more complete integration with the interactive sandbox, and not worry about totally breaking the javascript because we changed a div.
There are also issues that need to be addressed with the way that autodata integration setup hands off into the rest of the goal wizard before putting the live graph preview in every step of goal setup. As it stands now we would very likely have to write a custom dynagraph integration piece for every autodata integration. Woof.
There’s actually going to be surprisingly little work in the way of integrating code between the graphing libraries and our existing code. We will insert a “Preview” step into goal setup between the “Pledge your Money” and “Payment Method / add card” steps. At this point in the setting-up we’ve gathered all the goal config info from the user, and so we can feed those params into a sandbox object (provided by github.com/beeminder/road libraries) to build the graph and show what the goal will look like. The user will be able to use the back arrows in the wizard to go back and change any pieces they want and then come back to the Preview step to see the changes.
The wizard code will have to instantiate the graph object with appropriate parameters, and update them from local storage each time the preview step is shown.
One drawback of this approach, and honestly probably of any integration into the goal setup wizard, is that initial graphs all look fairly similar.
By which I mean, there is the obvious differences in yaw, for example, between Do More and Do Less goals, but other than that, because the graphing library does sensible things with picking the bounds to plot, a goal that has an initial rate of nine per day looks pretty identical to a goal that has a rate of one per year.
The other drawback is that this doesn’t make much use of the dyna- part of dynagraphs, though seeing the preview step will certainly be useful.
elf-step and id="preview"showPreview which will use localstorage state to generate the graph previewshowPreview will need to be triggered by showStep
TODO: make up some pseudo code about how showPreview will instantiate the bsandbox??
function showPreview() {
}
I’ve already mentioned several things that we’re not going to do re integrating dynagraphs into goal creation. Yes, it would be absolutely rad if we could set initial safety buffer from the graph preview, for example, but no. I do not love you all that much just now. Here’s a recap of those, and some other ideas that are outside the scope of this step.
use of the “step through the future” feature that uluc has in graph.beeminder.com/sandbox which lets you time-travel forward. (Might be sweet to incorporate that at some future point, but is not critical to the integration path / MVP)
Placeholder. Putting the dynamic graph editor in the admin UI could be a good way to guinea-pig the migration by making our admins use it first, and should serve as a good stepping stone to the next step.
Adding the graph editor interface into Beeminder-proper is a separate project from replacing the svg graphs with dynagraphs. This would be the most self-contained insertion in terms of how much its moving parts would need to integrate with existing Beebody code.
There would be a new tab added below the graph on the goal page. For now let’s assume we label it “Graph Editor”. Selecting that tab would actually replace the graph svg image with the dynamic graph-editor version from bbrain js libraries. [Is this a good idea? can’t you do better than that, self? i think it’s kind of questionable.] The “Commitment Dial” and “Ratchet” sections of Commitment tab would be subsumed by the new tab. [But how to give folks any idea that’s the place to look??] From the Stop/Pause tab, we would move “Take a Break”. From Data and Statistics nothing moves. From Settings, we remove the “Graph Editor” section.
This would trigger a shuffling of premium perks. We’ve long meant for the visual editor to be the “proper” (and hopefully more intuitive) way to make changes to the commitment. We’ve only had “graph editor” paywalled under the Bee Plus plan because editing the graph matrix by manually inserting tuples into a list is not normal human friendly.
This is the big-time for dynagraphs. We ultimately want them to replace the svg / static graphs and be a first class citizen of the Beebody interface.
On one hand, if we wanted to rush in to this as quickly as possible, we could potentially treat the blob of js that describes the dynagraph in the same way that we use the SVG. That would look something like we load the js blob exactly the same as we currently do with the svg, and when you submit a new datapoint, we submit that to the Beeminder server, wait for the new blob to be generated, and then reload that into the DOM. If we did this, we’d have a js graph that let you interact with it, e.g., scrolling, zooming, datapoint hover actions, etc, without the reactive / responsive web app interface. That might be a big enough improvement over the svgs to warrant starting with this. I’m not sure? I think we would want to evaluate whether this would be a step in the direction of full integration, or if it would be more like technical debt, or a dead-end as far as the path to full dynagraphs.
Ok, the other end of the spectrum from the rush-job looks like rewriting most of the Beeminder front-end to be reactive, using something like vue.js or React or Hotwire. We don’t want to take that route and do an actual rewrite. We want to find a compromise that allows incremental movement in the right direction.
What are the questions we need to ask re “properly” integrating js graphs into the front-end of Beeminder? What is the direction we are headed with dynamic / responsive user interface? Hotwire is the Rails answer to this stuff, and it looks promising for us.
Even when we’ve inserted the dynagraphs into the Beeminder interface, and have slick essentially real-time updates from datapoint submits etc, we will still need to generate static images somewhere in the background, because we want to still make those available to clients.
For example, maybe the mobile apps never want to use the javascript version.
And we may only want to provide the .png or .svg version to api clients.
[This is the distraction dump]