Seeking refuge from unsafe JavaScript

I joined Plaid in December 2013. At that time, the vast majority of Plaid’s code was JavaScript. Though the proportion has decreased since then, JavaScript still accounts for more than half our code.

Early Plaid code was fairly typical imperative JavaScript. For example:

module.exports = function(raw)
{
  var o = {};

  for (var i = 0; i < raw.values.length; i++) {
    if (raw.values[i].id === 'date')
      o.date = Date.create(raw.values[i].value);
    else if (raw.values[i].id === 'number')
      o.number = parseFloat(raw.values[i].value)*-1
  };

  return o;
}

Soon after I joined we began using Underscore in our various projects. We learned to always provide {} as the first argument to _.extend to avoid unintentional mutation. We accepted the fact that _.chain worked with Underscore functions but not functions defined elsewhere. We tolerated Underscore’s inconvenient argument order, and even added placeholder support to _.partial to make point-free programming with Underscore possible (though not natural). Using Underscore was clearly better than using no library at all. We were contented.

Underscore saved us from writing for loops, but did nothing to fix our type errors.

And we had many type errors.

We were able to avoid some type errors by using a linter and incorporating linting into our pull request workflow, but a significant proportion of type errors cannot be detected statically in a language as dynamic as JavaScript. Improving our test coverage further mitigated regressions.

Even after mitigating type errors caused by human error, many type errors still resulted from inconsistent data. At Plaid we process data from many disparate sources, but even for a single source the data’s shape may vary. We were bitten by this several times. Take the following expression:

data.foo.bar.baz

This looked innocuous, and the test suite passed. We deployed, and then noticed this in the production error logs:

TypeError: Cannot read property 'baz' of undefined

Ah. So data.foo.bar could be undefined in some cases, apparently. So:

data.foo.bar && data.foo.bar.baz

Fixed? We then learned that data.foo was undefined in some cases. We resorted to using guards:

data && data.foo && data.foo.bar && data.foo.bar.baz

As Underscore became an ever more important ingredient in writing JavaScript programs at Plaid, we grew tired of function expressions cluttering our code (arrow functions were not available to us at the time). To define a function that sums a list of numbers, one might write the following in Haskell:

foldl (+) 0

With Underscore, one might have written:

_.partial(_.reduce, _, function(a, b) { return a + b; }, 0)

As a side project I created Nucleotides, a tiny library which makes every JavaScript operator available as a function. We could then write:

_.partial(_.reduce, _, nucleotides.operator.binary['+'], 0)

This was still much less clear than the Haskell equivalent.

One day Graeme Yeates mentioned me on a Ramda issue. It was my first exposure to Ramda, and I was impressed by its terseness:

R.reduce(R.add, 0)

Elegant! Like the Haskell definition, it takes advantage of currying.

Several of us at Plaid caught the Ramda bug, and Plaid became one of the first companies to use Ramda in production. No longer did we need to worry about functions mutating their arguments, as Ramda functions never did. No longer was _.partial necessary to define specialized functions in terms of more general ones, as every Ramda function has support for partial application baked in.

Once again, we were contented.

I then began to worry about Ramda’s unsafe functions. R.head, for example, is of type [a] → a. When applied to [], there is no a, so Ramda returns undefined. This means evaluating an expression such as R.toUpper(R.head(xs)) will result in a run-time exception if xs is [].

This problem can be resolved by changing the type of head to [a] → Maybe a. If head is applied to [] the result is Nothing(), which indicates a failed operation. If head is applied to ['x', 'y', 'z'] the result is Just('x'). The head of the list, 'x', is wrapped in a container which indicates a successful operation.

Nothing() and Just('x') are both members of the Maybe String type. Both values support exactly the same set of operations. To transform the String which may be inside the Maybe, we use map:

                   +------+                 +--------------+
['x', 'y', 'z'] ~~~| head |~~> Just('x') ~~~| map(toUpper) |~~> Just('X')
                   +------+                 +--------------+

                   +------+                 +--------------+
             [] ~~~| head |~~> Nothing() ~~~| map(toUpper) |~~> Nothing()
                   +------+                 +--------------+

To quench my thirst for type safety I defined safe versions of several unsafe Ramda functions (including head). Initially these lived in a file of helper functions in one Plaid project. We realized these would be useful in other projects and to people outside the company, so we released Sanctuary on GitHub and npm. Now, with Ramda and Sanctuary, it’s possible to write terse, declarative programs that work correctly for all inputs.

S.pipe([S.gets(String, ['transaction_info', 'amount']),
        R.chain(N.normalizeAmount),
        R.map(S.negate),
        S.or,
        R.over(L.amount)])

This function, of type Object → Tx → Tx, describes a sequence of transformations to safely extract a particular value from an Object, and possibly update the value of the amount field of a Tx value.

It acknowledges the following possibilities:

  • the transaction_info field may be absent;
  • the value of the transaction_info field may be null or undefined;
  • the amount field may be absent;
  • the value of the amount field may not be of type String (the argument type required by N.normalizeAmount); and
  • the value of the amount field may not actually represent an amount.

It does so with the Maybe data type rather than with incoherent guards and exception handling.

Although Sanctuary was initially developed internally, several people from outside Plaid have become valued collaborators since we released the project under the MIT license. Stefano Vozza was the first external contributor (documenting much of what was a completely undocumented API at the time), and remains one of the most active. Kevin Wallace contributed the wonderful multi-line error messages:

S.fromMaybe(0, S.Just('XXX'));
// ! TypeError: Type-variable constraint violation
//
//   fromMaybe :: a -> Maybe a -> a
//                ^          ^
//                1          2
//
//   1)  0 :: Number, FiniteNumber, Integer, ValidNumber
//
//   2)  "XXX" :: String
//
//   Since there is no type of which all the above values are members, the type-variable constraint has been violated.

In recognition of the fact that the Sanctuary community is now self-sustaining, we’ve transferred the repositories to the sanctuary-js organization on GitHub.

I believe these projects have an important role to play in the future of functional programming in JavaScript, and in exposing JavaScript programmers to ideas from other languages. I’ll continue to work alongside other members of the community to improve our refuge from unsafe JavaScript.

Integrate Today