News: Introducing Promises & Phasing Out Callbacks

Job Writing
1

Hi community!

Not to be too dramatic or anything but we recently a small feature which could change the face of adaptors forever.

That new feature is quite simple really: Operations (top level function calls) now behave like promises.

Operations as Promises

Every operation, like fn() now has a .then() and .catch() function.

So I can do stuff like this:

fn((state) => {
	// do something
	return state;
}).catch((error, state) => {
	// do something with the error
})

This is enabled by magic in the compiler, so it works on all adaptor versions. You can think of this as a runtime improvement ,rather than an adaptor improvement.

The use of .catch is fairly obvious - now you can trap errors and add your own debugging. The catch callback is passed an error and state.

But the really exciting thing here is that .then() allows us to remove callbacks from adaptor APIs.

Best Practice for Callbacks

Many adaptor functions include a callback as the final argument, so you can do stuff like this:

get(www, {}, (state) => {
	state.items = state.data;
	return state;
})

In 99% of cases [citation needed], you can also do this:

get(www)
fn((state) => {
	state.items = state.data;
	return state;
})

I consider this to be Best Practice for job writing. yesterday, today and in the future.

With the new promises support, you can also do this:

get(www).then((state) => {
	state.items = state.data;
	return state;
})

You can even chain promises if you really want to, although I wouldn’t recommend it:

get(www)
	.then((state) => {
		state.items = state.data;
		return state;
	})
	.then((state) => {
		console.log(state)
		return state;
	})

The 1% of cases - which is actually a big 1% - comes with each

each($.data, get($.data.url, {}, (state) => {
   state.responses.push(state.data
  return state
})

In the case of each, the callback is really important, because it allows you to take the “scoped state” from each (ie, data is the item under iteration), and do something with it. You can’t just add an fn() block here because it won’t get scoped state, it won’t be called per iteration.

This is when then() pays for itself: it means every operation gets a “callback” for free, so we can do this:

each($.data, get($.data.url).then((state) => {
   state.responses.push(state.data
  return state
})

Adaptor Changes

Now, I’m quite excited about this, but then I don’t get out much.

The reason I’m excited is that this small change means that we can start phasing out callbacks from our adaptor functions.

Signatures like this:

post(path, body, options, callback)

Will turn into this:

post(path, body, options)

And if you really really need a callback, like in an each block, you can always use .then()

This really helps adaptor API design because we no longer have two optional parameters like options and callback, avoiding ugly signatures like get(www, null, fn())

It’s also one less thing to document. We’ve already got a huge section in the Job Writing guide to explain callbacks - thanks to this promise interface, we can basically remove that entire section and all the complexities in it.

Further Reading

You can read more in the Job Writing Guide on docs.openfn.org

That’s all from me today. I’d love to know what people think about this so please post your comments, thoughts and questions.

Thanks!

1 Like