Discussion regarding Adapter 2.0 Project

Hey folks!
I am Aditya R Rudra, a second year CSE undergraduate at NITK, India.
I read through the ideas list and found the idea related to Adapters 2.0 interesting.
Any help regarding how to get started and some more details about the project, will be greatly appreciated.
Thanks & Regards…

Hey @adityaofficial10 , nice to be connected.

(CC: @Mamadou and @stu on this one. Maybe @chaiwa re: the dhis2 adaptor?)

Very glad to hear your interested. The easiest way to get up and running with OpenFn (and to learn about the adaptors) is to create a free account on the hosted app (openfn.org/signup) and to check out the rest of that docs site (docs.openfn.org).

For this project in particular, the thought is that we could be doing a lot more in terms of introspection and how we help users write jobs more quickly and easily. Currently, adaptors wrap up various critical APIs used by ministries of health, NGOs, humanitarian orgs, etc., and provide helper functions with some basic inline documentation generated by jsdoc.

jsdoc is very cool, but also very old… converting to Typescript would allow us to provide a much cleaner development experience for those writing jobs on the platform or in devtools and enable all sorts of interesting IDE/usability enhancements.

If you know about our work, you know that the ultimate goal is to help scale the worlds most promising health and humanitarian interventions via secure, stable, and scalable integrations that save them time and money while improving data quality and speed of delivery. The process of defining business requirements for a high-sensitivity automation (like transferring case data between a child protection system and a police database) is hard enough—so we want to focus on making the actual scripting of those jobs easier.

Also linking back to the GSOC page in case someone else finds this thread: https://docs.openfn.org/documentation/gsoc/

Thanks a lot @taylordowns2000 and sorry for the late reply.
I have started reading the docs as directed.
Can you please tell me about the repository we’re talking about?
I found one but it seems it is a template for adaptors.
Also, the outcome is to have the typescript counterpart of the current JS code.
Right?
This is just to confirm.

Thanks & Regards.

Hey @adityofficial10! :wave:

Currently there is a very small script/repo called simple-ast (GitHub - OpenFn/simple-ast: A module that generates an OpenFn-readable AST from a language-package) which reads the jsdoc style comments and produces a JSON file that is used by the front end tools found on openfn.org to give the user hints about what is expected from the different functions.

Think VSCode/Intellisense.

This works, but its not particularly clever - nor is it able to validate itself, we have to make sure the comments are correct. And we cant tell before running the users code that the function has been used correctly.

Since typescript has matured so much since this original design (circa 2016) we believe that we could greatly improve the quality of our users experience buy using the typescript compiler/internal tools (the ones I’m assuming vscode etc use) to introspect not only user code but also provide much greater information as they type.

This is a very large task to take all the way to production but it requires some curious research to produce a working prototype of 1. A language pack/adaptor written in typescript with type definitions and 2. An example of how one might validate some provided code against the definitions

@stu
Hey!
Thanks for your input.
So what I understood from this is that we have a script written in JS which parses jsdoc style comments and produces a JSON file.
And the objective of this project is to develop a TypeScript equivalent of that which has a proper validation system which ensures that the comments written are correct.
Am I thinking in the right direction?

You’re on the right track, but rather than reinvent the TS compiler, create a script/module that can produce on of those (or similar) AST files… Using the typescript module in nodejs (essentially find the right functions and build an abstraction on top).

Following that is to validate a user script (except insteadof using the typescript cli, its our cli)

Just gonna add on to this reply, I know this particular use case may seem unusual.

Currently we use a library (that we wrote) called ‘core’, GitHub - OpenFn/core: Core is the central job processing program used in the OpenFn platform.

If you take a look at the ‘compile’ command here: core/compile-command.js at main · OpenFn/core · GitHub

You can trace (via the Compile class) your way to the transforms (core/transforms.js at main · OpenFn/core · GitHub), each one of those transforms are used to check and manipulate the AST (code represented as data).

@stu
Thanks for your time and information.
I’ll have a look at them and follow up with my queries, if required.
Also, I’m working on my draft which I would like you guys, to review.
This will be really helpful to me and would make my proposal better.
Thanks again.