Hi Community - I need your help!
I am trying to decide whether namespaces in adaptors - like http.get() or util.uuid() - are useful when reading or writing job code.
I’m going to post the question first, and then give a detailed explanation of the problem and solution. So if the question doesn’t make sense, just read on and circle back 
Please leave your thoughts in the thread!
Should we namespace common?
I would love to namespace the common adaptor, so that instead of this:
each(dataSource, operation)
field(key, value)
fn(func)
lastReferenceValue(path)
The API is like this:
each(dataSource, operation)
fn(func)
util.field(key, value)
util.lastReferenceValue(path)
I think this is clearer and more elegant, and helps to tell at a glance how a function can be be used, whether it returns state, etc.
It also separates out Operations - which are big, high level things, the things you first reach for when you write a job - from regular functions, which are usually smaller, more technical.
Operations solve business problems, whereas functions solve JavaScript problems. And this namespacing reflects that.
The reason I ask is that it’s an expensive change with a lot of impact, and will result in major version bumps for most of our adaptors. That’s totally fine if this is the right thing to do - so I’m just gathering data.
What do you think?
The Problem
Because this stuff can be a little complicated I wanted to include more context. Some of you will know exactly what I’m talking about above - but some of you won’t. This bit is for you guys.
When you’re writing job code, you’re calling functions defined in our adaptors. And there are two basic kind of functions:
- Operations
- Plain Javascript Functions (PJFs, anyone? No?)
Operations are super important: they’re the bones of how OpenFn jobs work.
The basic difference is that an Operation can be used at top level of your job code, but not inside callbacks (ie, not inside other functions). And the inverse is true for plain JavaScript functions - they can be used inside callbacks, but not at the top of your job code.
I’ll show this in an example. You can use console.log in your job code, which is a plain Javascript function. But you’ll get an error if you do it outside a callback.
console.log('hello') // error!
So we use our old friend, fn(), which is an Operation which takes a callback.
fn(s => {
console.log('hello') // 👍️
})
We recently released an operation called log(), which is just a wrapper around console.log. So you can do this:
log('hello')
But you can’t use log() or fn() inside an operation callback
fn(s => {
log('hello') // 👎️
})
The actual problem here is that it’s super unclear which are operations and which are functions. The only way you can tell right now is to look at the docs: if it says a function returns an Operation, then it’s an operation (which is itself a bit confusing, I know). Otherwise, it’s a regular Javascript function.
The solution - namespaces?
To help this problem, we’ve started using namespaces across our adaptors.
Namespaces are a convenient way to group closely related functionality.
For example, the common adaptor right now has a bunch of generic HTTP helper operations which live under the http namespace, like this:
- http.get(url, options)
- http.post(url, options)
- http.request(method, url, options)
This groups the functions conveniently in docs, but also when reading job code back it helps set the context.
In Salesforce, we’re experimenting with namespaces for the bulk V1 and V2 APIs - which are similar but also different. So there will be bulk1.query and bulk2.query, so again the namespace sets some key context and makes it super clear what’s happening.
What’s this got to do with operations and stuff?
Enter the util namespace - designed as a catch-all namespace for any plain old JavaScript Functions.
So if you see a function in util. , you can expect that it’s not an operation, and shouldn’t be called top level. You can also assume that anything in the top namespace, like fn() is an operation.