Beebrain: The Brains of Beeminder

This is an internal API.

Beebrain is stateless. It accepts all the settings for a Beeminder graph as well as the list of datapoints and it computes everything there is to know about the status of the goal as well as the graph image itself. The complement of Beebrain is Beebody, comprising the whole website and database and API. (There’s also the smartphone apps — Beedroid and BeemiOS — plus third-party apps and integrations.)

Beebrain takes three parameters (described in more detail below):

It returns a JSON hash, also described below, with statistics and metrics like amount of current safety buffer, as well as a pointer to an image of the graph itself.

Dates given to and returned from Beebrain are encoded as daystamps, namely YYYYMMDD strings such as “20140831”. The exception is proctm which Beebrain returns as unixtime (in seconds) because that’s giving a specific absolute point in time.

Input

Beebrain takes the following parameters, given as either GET or POST:

quantum  : 1e-5,   // Precision/granularity for conservarounding baremin etc
timey    : false,  // Whether numbers should be shown in HH:MM format
ppr      : true,   // Whether PPRs are turned on (ignored if not WEEN/RASH)
deadline : 0,      // Time of deadline given as seconds before or after midnight
asof     : null,   // Compute everything as if it were this date; future ghosty
tini     : null,   // (tini,vini) specifies the start of the BRL, typically but
vini     : null,   //   not necessarily the same as the initial datapoint
road     : [],     // List of (time,value,rate) triples defining the BRL
tfin     : null,   // Goal date (unixtime); end of the Bright Red Line (BRL)
vfin     : null,   // The actual value being targeted; any real value
rfin     : null,   // Final rate (slope) of the BRL before it hits the goal
runits   : 'w',    // Rate units for road and rfin; one of "y","m","w","d","h"
gunits   : 'units',// Goal units like "kg" or "hours"
yaw      : 0,      // Which side of the BRL you want to be on, +1 or -1
dir      : 0,      // Which direction you'll go (usually same as yaw)
pinkzone : [],     // Region to shade pink, specified like the graph matrix
tmin     : null,   // Earliest date to plot on the x-axis (unixtime):
tmax     : null,   //   ((tmin,tmax), (vmin,vmax)) give the plot range, ie, they
vmin     : null,   //   control zooming/panning; they default to the entire
vmax     : null,   //   plot -- initial datapoint to past the akrasia horizon
kyoom    : false,  // Cumulative; plot values as the sum of those entered so far
odom     : false,  // Treat zeros as accidental odom resets
maxflux  : 0,      // User-specified max daily fluctuation                      
monotone : false,  // Whether the data is necessarily monotone (used in limsum)
aggday   : null,   // How to aggregate points on the same day, max/sum/last/etc
plotall  : true,   // Plot all the points instead of just the aggregated point
steppy   : false,  // Join dots with purple steppy-style line
rosy     : false,  // Show the rose-colored dots and connecting line
movingav : false,  // Show moving average line superimposed on the data
aura     : false,  // Show blue-green/turquoise (now purple I guess) aura/swath
hashtags : true,   // Show annotations on graph for hashtags in datapt comments 
yaxis    : '',     // Label for the y-axis, eg, "kilograms"
waterbuf : null,   // Watermark on the good side of the BRL; safebuf if null
waterbux : '',     // Watermark on the bad side, ie, pledge amount
hidey    : false,  // Whether to hide the y-axis numbers
stathead : true,   // Whether to add a label w/ stats at top of graph (DEV ONLY)
yoog     : 'U/G',  // Username/graphname, eg, "alice/weight"                

Output

Beebrain returns a JSON hash of the following output fields (plus various deprecated fields):

sadbrink : false,   // Whether we were red yesterday & so will instaderail today
safebump : null,    // Value needed to get one additional safe day
dueby    : [],      // Table of daystamps, deltas, and abs amts needed by day
fullroad : [],      // Road matrix w/ nulls filled in, [tfin,vfin,rfin] appended
pinkzone : [],      // Subset of the road matrix defining the verboten zone
tluz     : null,    // Timestamp of derailment ("lose") if no more data is added
tcur     : null,    // (tcur,vcur) gives the most recent datapoint, including
vcur     : null,    //   flatlining; see asof 
vprev    : null,    // Agged value yesterday 
rcur     : null,    // Rate at time tcur; if kink, take the limit from the left
ravg     : null,    // Overall red line rate from (tini,vini) to (tfin,vfin)
tdat     : null,    // Timestamp of last actually entered datapoint pre-flatline
stdflux  : 0,       // Recommended maxflux, .9 quantile of rate-adjusted deltas
delta    : 0,       // How far from the red line: vcur - rdf(tcur)
lane     : 666,     // Lane number for backward compatibility
cntdn    : 0,       // Countdown: # of days from tcur till we reach the goal
numpts   : 0,       // Number of real datapoints entered, before munging
mean     : 0,       // Mean of datapoints
meandelt : 0,       // Mean of the deltas of the datapoints
proctm   : 0,       // Unixtime when Beebrain was called (specifically genStats)
statsum  : '',      // Human-readable graph stats summary (not used by Beebody)
ratesum  : '',      // Text saying what the rate of the red line is
deltasum : '',      // Text saying where you are wrt the red line
graphsum : '',      // Text at the top of the graph image; see stathead
progsum  : '',      // Text summarizing percent progress, timewise and valuewise
safesum  : '',      // Text summarizing how safe you are (NEW!)
rah      : 0,       // Y-value of the bright red line at the akrasia horizon
safebuf  : null,    // Number of days of safety buffer
error    : '',      // Empty string if no errors generating the graph
graphurl : null,    // Nonce URL for the graph image, based on the provided slug
thumburl : null,    // Nonce URL for the graph image thumbnail

Details and Caveats

The nonce URLs (graphurl and thumburl) will not be ready for a couple seconds. Until then they will return 404 Not Found. We recommend polling them once per second until they stop 404ing.

The bright red line is undefined before (tini, vini). The graph matrix, road, defines every subsequent segment, except the final segment which is defined by (tfin, vfin, rfin). So if the graph matrix contains rows (t1,v1,r1) through (tn,vn,rn) then the red line starts at (tini, vini), continues at rate r1 till (t1,v1), continues at rate r2 till (t2,v2), etc, till (tn,vn) and then has one final segment at rate rfin till the end of the line at (tfin, vfin).

Monotonicity (monotone) means the datapoints are monotone increasing or decreasing, depending on dir. It’s unclear if that matters at all anymore. In the old days, if monotone was true then vmin or vmax could be set equal to vini. Otherwise the width of the old yellow brick road made it extend slightly beyond vini, which was a bit ugly for example for a Do More graph to include negative values on the y-axis.

Example

Actually this never returns for some reason; I guess don’t click this:

https://www.beeminder.com/api/v1/beebrain?slug=tmp&params={“dir”:1\,”waterbux”:”beebrain-test”}&data=[[“20160525”\,1\,”first+point”]]

PS: Now it 404s; I’m not sure how Beebody actually calls Beebrain these days!


Appendix 1: How to fill in the graph matrix

Here’s Python code to fill in a graph matrix. It takes the coordinates of the start of the bright red line (tini, vini) and the road matrix, including the final (tfin, vfin, rfin) row, where each row has exactly two out of three of the columns (t, v, r) specified.

# Util function "foldlist" that's like Ruby's inject but keeps the intermediate results:
# foldlist(f,x, [e1, e2, ...]) -> [x, f(x,e1), f(f(x,e1), e2), ...]

DIY    = 365.25      # this is what physicists use, eg, to define a light year
SID    = 86400       # seconds in a day
BDAWN  = 1202749200  # 2008-02-11, dawn of Kibotzer/Beeminder
BDUSK  = 2147317201  # ~2038, specifically rails's ENDOFDAYS+1 (was 2^31-2weeks)

SECS = { # Number of seconds in a year, month, week, day, and hour
'y' : DIY*SID,
'm' : DIY/12*SID,
'w' : 7*SID,
'd' : SID,
'h' : 3600,
}

siru = SECS[runits] # seconds in rate units

# Given the endpoint of the last redline segment (tprev,vprev) and 2 out of 3 of
#   t = goal date for a redline segment (unixtime)
#   v = goal value 
#   r = rate in hertz (s^-1), ie, redline rate per second
# return the third, namely, whichever one is passed in as null.
def tvr(tprev, vprev, t, v, r):
  if exprd and v != None:  # no such thing as exprd's now so ignore this
    if v     == 0: v     = 1e-6 # zero values and exprds don't mix!
    if vprev == 0: vprev = 1e-6 # just make them near zero I guess?

  if t == None:
    if r == 0: return BDUSK
    else:  return min(BDUSK, tprev + (log(v/vprev)/r if exprd else (v-vprev)/r))
  if v == None: 
    if exprd and r*(t-tprev) > 35: return vprev*1e15 # bugfix: math overflow
    return vprev*exp(r*(t-tprev)) if exprd else vprev+r*(t-tprev)
  if r == None:
    if t == tprev: return 0 # special case: zero-length line segment
    return log(v/vprev)/(t-tprev) if exprd else (v-vprev)/(t-tprev)

# Helper for fillroad for propagating forward filling in all the nulls
def nextrow((tprev, vprev, rprev), (t, v, r)):
  x = tvr(tprev, vprev, t,v,r) # the missing t, v, or r
  if t==None: return (x, v, r)
  if v==None: return (t, x, r)
  if r==None: return (t, v, x)

# Takes graph matrix (with last row appended) and fills it in
def fillroad(road):
  road = [(dayfloor(t), v, r if r==None else r/siru) for (t,v,r) in road]
  road = foldlist(nextrow, (tini, vini, 0), road)[1:]
  return [(t, v, r*siru) for (t,v,r) in road]

Appendix 2: Red line (road) function

Given a filled-in graph matrix, the starting coordinates of the bright red line, and a unixtime t, we can compute the value of the bright red line at time t.

# Helper for roadfunc. Return the value of the segment of the YBR at time x, 
# given the start of the previous segment (tprev,vprev) and the rate r. 
# (Equivalently we could've used the start and end points of the segment, 
# (tprev,vprev) and (t,v), instead of the rate.)
def rseg(tprev, vprev, r, x): 
  if exprd and r*(x-tprev) > 230: return 1e100 # bugfix: math overflow
  return vprev*exp(r*(x-tprev)) if exprd else vprev+r*(x-tprev)

# Take an initial point and a filled-in graph matrix (including the final row) 
# and a time t and return the value of the centerline at time x.
def roadfunc(tini, vini, road, x):
  road = [(tini,vini,None)] + road
  if   x<road[0][0]: return road[0][1] # road value is vini before tini
  for i in range(1, len(road)):
    if x<road[i][0]: return rseg(road[i-1][0], road[i-1][1], road[i][2]/siru, x)
  return road[-1][1]