OpenFn Product Update: June 2024

Hello Community :wave:t3:,

It’s been jam-packed, these past few months for all of us at OpenFn. Releasing Lightning was an important milestone, but it marked the beginning of the new user experience that our technology for development (tech4dev) community has demanded. In this product update, I’ll present a few notable features and enhancements that have come out since v2 went live in February.​​

New workflow builder

Workflow design and implementation are at the core of OpenFn and how we deliver on our promise of seamless and scalable automation and interoperability across organizations and teams. In v1, the old workflow builder handled complexity through powerful scripting language and simple job “triggers”. We replaced this in v2 with a simple click-to-add visual canvas that allows users to add steps to workflow triggers and build multi-step workflows with simple logic-based branching. Through our interactions with users, we realized some have challenges building more complex workflows with advanced branching and merging logic. The workaround for this was replicating steps in the workflow or breaking one workflow into two and using the HTTP adaptor to trigger one from the other.

The new workflow builder enables users to build complex multiplex workflows with advanced branch logic for merging two or more steps into a downstream step, eliminating the need to replicate steps across branches or split a task into multiple workflows. This improvement also introduces a new bezier smoother edge type that replaces smoothstep edges to accommodate the scale of complexity that users might want to adopt when building workflows.

Generic OAuth

Many modern platforms, such as Salesforce, MSGraph, and Google, require OAuth2 authentication for external requests to enhance identity and access control. Previously, instance administrators could configure OAuth2 for Google Spreadsheets and Salesforce using instance environment variables, restricting each OpenFn instance to a single OAuth2 client.

We have also received numerous requests for OAuth2 integrations with platforms like DHIS2, MSGraph, ArcGIS Pro, Kobotoolbox, and many others, highlighting the need for more one-by-one integrations. Generic OAuth eliminates the need for these tedious, individual OAuth2 integrations for each application or platform. Now, instance administrators can easily set up OAuth clients with specific scopes and share them seamlessly across projects and users. This bring-your-own-client approach enhances authorization control and collaboration across projects and teams like never before.

GitHub sync

Portability is a frequently used and talked about feature of OpenFn for distributed teams who replicate workflows across various domains and projects. Even single users often need version control capability. The GitHub sync feature allows project admins to leverage GitHub branches and workflows to sync their projects as code and initiate syncs to and from their projects and remote repositories when required.

Configurable retention period

History (including work orders and their related runs, logs, and input/output data) are stored unless otherwise configured through the zero-persistence feature. The configurable retention period enables project administrators to set how long (in days) they want work orders and their related artifacts to be kept on the server. For projects with a retention period set, all work order history and associated artifacts older than the retention period are automatically wiped off the platform and are unrecoverable.

Configurable self-service sign-up

There are two ways to become an OpenFn user - via the admin or through a self-service sign-up on the /users/register page. Now, instance admins can set if they want users to sign up themselves or would rather provision accounts for every new user.

New I/O data with log viewer implemented in Monaco editor

One of the major improvements between v1 and v2 was in OpenFn’s ability to handle large payloads and long-running workflows. As the usage of these new heavy-duty features increased, however, we identified front-end performance issues associated with loading and streaming large data clips and logs to the web viewer interfaces. While overhauling the inspector to handle these big payloads, we’ve also added log line numbering and a clever full-text search that won’t harm browser performance even with up to 15MB of data in a given data clip or log.

We remain ever so thankful for your support and engagement so far. The team continues to work hard to deliver more updates to the platform and we welcome your feedback and ideas.

Happy automating with OpenFn!

Ayodele Adeyemo,
Product Manager,