Pulling data from Kobotoolbox form and insert them into DHIS2

mbebana · March 26, 2024, 12:30am

Hi Community,
I created job to extract data from Kobotoolbox form with GetSubmissions function. Data which is extracted from kobotoolbox should be inserted into DHIS2 dataset.
When data gets extracted from Kobotoolbox, i tried to access to specific field (called QteDisp in KB form) but it doesn’t work.
Here is sample of my job to send data to DHIS2 :

create('dataValueSets', state => ({
  dataSet: 'BfMAe6Itzgt', // Child Health
  period: '202402', // Feb 2024
  orgUnit: 'DiszpKrYNg8', // Ngelehun CHC
  dataValues: [
    {
      categoryOptionCombo: 'Prlt0C1RF0s', //Fixed <1yr
      dataElement: 'x3Do5e7g4Qo', // OPV0 doses given
      value: dataValue('data.QteDisp'), //# of OPV0 doses given
       // value: '520',
    },
  ],
}));

if i used constant like ‘520’ it works but trying to pull data from kobotoolbox with datavalue function it doesn’t work.

DHIS2 error message is : “Data value or comment not specified for data element”
Which means missing value.
Appreciated if you can help on this issue.

Thank you

mtuchi · March 26, 2024, 6:05am

Hello @mbebana,
I noticed your use of dataValue() is not accurate.
The dataValue function picks out a single value in state.data. See dataValue() docs to learn more

So let’s say the GetSubmissions step was successfully and the returned data (in state.data) have a key QteDisp. To access the value for QteDisp in state.data. QteDisp you can use dataValue('QteDisp')

Here is how it will look like.

create("dataValueSets", {
  dataSet: "BfMAe6Itzgt", // Child Health
  period: "202402", // Feb 2024
  orgUnit: "DiszpKrYNg8", // Ngelehun CHC
  dataValues: [
    {
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: dataValue('QteDisp'), //# of OPV0 doses given
      // value: '520',
    },
  ],
});

For a more modern approach [Strongly recommended]
You can access the value for QteDisp from state.data.QteDisp without using dataValue() function

See example below

create("dataValueSets", (state) => ({
  dataSet: "BfMAe6Itzgt", // Child Health
  period: "202402", // Feb 2024
  orgUnit: "DiszpKrYNg8", // Ngelehun CHC
  dataValues: [
    {
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: state.data.QteDisp, //# of OPV0 doses given
      // value: '520',
    },
  ],
}));

I hope that is usefully , Let me know if you have more questions

mtuchi · March 26, 2024, 11:04am

Hello @mbebana, I have updated the examples in my early response to remove anti-pattern.

I would still recommend you use this pattern

create("dataValueSets", state => ({
  dataSet: "BfMAe6Itzgt", // Child Health
  period: "202402", // Feb 2024
  orgUnit: "DiszpKrYNg8", // Ngelehun CHC
  dataValues: [
    {
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: state.data.QteDisp, //# of OPV0 doses given
      // value: '520',
    },
  ],
}));

Because it gives you flexibility if you need to transform data

mbebana · March 26, 2024, 6:40pm

Hello @mtuchi, Thank you very much for your appreciated help and your availability.
I tried again the code with your suggestions but still have the issue.
sate.data.QteDisp doesn’t get the information received from kobotoolbox form. DHIS2 still has the same error message : “Data value or comment not specfied for data element”.

You can find below, the entry received from Kobotoolbox :

{
    "data": {
        "count": 4,
        "next": null,
        "previous": null,
        "results": [
            {
                "Date": "2019-05-09",
                "Moughataa": "Barkeol",
                "Position_g_ographique": "6.131302 1.244223 0 36",
                "QteDisp": "220",
                "QteDistr": "100",
                "QteSt": "320",
                "Type_de_d_p_t": "conteneur_frig",
                "version": "vnbZkxVTkMUBUcCDPz84A7",
                "_attachments": null,
                "_bamboo_dataset_id": "",
                "_geolocation": [
                    6.131302,
                    1.244223
                ],
                "_id": 27488009,
                "_notes": null,
                "_status": "submitted_via_web",
                "_submission_time": "2019-05-09T19:42:54",
                "_submitted_by": null,
                "_tags": null,
                "_uuid": "dd590ec9-0bc6-4d13-9d64-460f45fa3df1",
                "_validation_status": {},
                "_xform_id_string": "aaZqVeofmTWgDmj5iTj4nS",
                "end": "2019-05-09T19:38:36.002-00:00",
                "formhub/uuid": "9be1904ef9b640fd841dac7c00bbb7a7",
                "meta/instanceID": "uuid:dd590ec9-0bc6-4d13-9d64-460f45fa3df1",
                "start": "2019-05-09T19:37:55.331-00:00"
            },
            {
                "Date": "2019-05-09",
                "Moughataa": "Guerou",
                "Position_g_ographique": "6.131302 1.244223 0 36",
                "QteDisp": "240",
                "QteDistr": "100",
                "QteSt": "340",
                "Type_de_d_p_t": "conteneur_frig",
                "version": "vnbZkxVTkMUBUcCDPz84A7",
                "_attachments": null,
                "_bamboo_dataset_id": "",
                "_geolocation": [
                    6.131302,
                    1.244223
                ],
                "_id": 27485928,
                "_notes": null,
                "_status": "submitted_via_web",
                "_submission_time": "2019-05-09T19:21:20",
                "_submitted_by": null,
                "_tags": null,
                "_uuid": "d6f3cd00-6e35-4fca-97d0-7afbb9670863",
                "_validation_status": {},
                "_xform_id_string": "aaZqVeofmTWgDmj5iTj4nS",
                "end": "2019-05-09T19:17:01.822-00:00",
                "formhub/uuid": "9be1904ef9b640fd841dac7c00bbb7a7",
                "meta/instanceID": "uuid:d6f3cd00-6e35-4fca-97d0-7afbb9670863",
                "start": "2019-05-09T19:15:59.863-00:00"
            },
            {
                "Date": "2019-05-11",
                "Moughataa": "Barkeol ",
                "QteDisp": "120",
                "QteDistr": "100",
                "QteSt": "220",
                "Type_de_d_p_t": "conteneur_frig",
                "version": "vnbZkxVTkMUBUcCDPz84A7",
                "_attachments": null,
                "_bamboo_dataset_id": "",
                "_geolocation": [
                    null,
                    null
                ],
                "_id": 27610772,
                "_notes": null,
                "_status": "submitted_via_web",
                "_submission_time": "2019-05-11T12:02:12",
                "_submitted_by": "csdpmauritanie",
                "_tags": null,
                "_uuid": "44dbd1e8-d61e-4fab-a3ac-4c02cfe2184d",
                "_validation_status": {},
                "_xform_id_string": "aaZqVeofmTWgDmj5iTj4nS",
                "end": "2019-05-11T11:59:35.158Z",
                "formhub/uuid": "9be1904ef9b640fd841dac7c00bbb7a7",
                "meta/instanceID": "uuid:44dbd1e8-d61e-4fab-a3ac-4c02cfe2184d",
                "start": "2019-05-11T11:56:42.430Z"
            },
            {
                "Date": "2024-03-15",
                "Moughataa": "TVZ",
                "QteDisp": "70",
                "QteDistr": "30",
                "QteSt": "100",
                "Type_de_d_p_t": "plateforme",
                "version": "vnbZkxVTkMUBUcCDPz84A7",
                "_attachments": null,
                "_geolocation": [
                    null,
                    null
                ],
                "_id": 321651495,
                "_notes": null,
                "_status": "submitted_via_web",
                "_submission_time": "2024-03-15T18:12:55",
                "_submitted_by": null,
                "_tags": null,
                "_uuid": "ebf14ed0-9898-45ad-a491-60d36304a0da",
                "_validation_status": {},
                "_xform_id_string": "aaZqVeofmTWgDmj5iTj4nS",
                "end": "2024-03-15T18:12:56.390-00:00",
                "formhub/uuid": "9be1904ef9b640fd841dac7c00bbb7a7",
                "meta/instanceID": "uuid:ebf14ed0-9898-45ad-a491-60d36304a0da",
                "start": "2024-03-15T18:12:33.161-00:00"
            }
        ]
    },
    "references": [
        {}
    ]
}

mtuchi · March 26, 2024, 9:40pm

Hello @mbebana, Based on the state.data you have shared. It looks like the position of QteDisp is inside state.data.results[key]. QteDisp.
So it looks like you need to create an array of dataValues first then create dataValueSet.

This is what i would recommend you do,

First: Map all dataValues from state.data.results

See example code below

fn((state) => {
  const { results } = state.data;

  state.dataValues = results.map((rs) => ({
    categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
    dataElement: "x3Do5e7g4Qo", // OPV0 doses given
    value: rs.QteDisp, //# of OPV0 doses given
    // value: '520',
  }));

  // Remember to remove this console.log, when done debugin
  console.log("Mapped DataValues", state.dataValues);
  return state;
});

Second: Update the create() function to use the mapped dataValues

See example code below

create("dataValueSets", (state) => ({
  dataSet: "BfMAe6Itzgt", // Child Health
  period: "202402", // Feb 2024
  orgUnit: "DiszpKrYNg8", // Ngelehun CHC
  dataValues: state.dataValues,
}));

mbebana · March 26, 2024, 11:15pm

Thank you @mtukoh
it’s done with success but no data is added to DHIS2 (imported =0)

Memory limit: 500mb
Timeout: 60s
Versions:
▸ node.js 18.19.1
▸ worker 1.1.1
▸ @openfn/language-common 1.12.0
▸ @openfn/language-dhis2 4.0.3
▸ @openfn/language-kobotoolbox 2.1.0
Executing 3d7cd967-3a8b-4b30-aee2-1ae69537fb0c
Starting step DHIS2
[linker] loading module @openfn/language-dhis2
[linker] Loading module @openfn/language-dhis2 from /tmp/openfn/worker/repo/node_modules/@openfn/language-dhis2_4.0.3/dist/index.cjs
Resolved adaptor @openfn/language-dhis2 to version 4.0.3
Executing expression (1 operations)
Timeout set to 300000ms
Starting operation 1
Preparing create operation…
Using latest available version of the DHIS2 api on this server.
Sending post request to Login app | DHIS2
Created dataValueSets with response {
“httpStatus”: “OK”,
“httpStatusCode”: 200,
“status”: “OK”,
“message”: “Import was successful.”,
“response”: {
“responseType”: “ImportSummary”,
“status”: “SUCCESS”,
“importOptions”: {
“idSchemes”: {},
“dryRun”: false,
“async”: false,
“importStrategy”: “CREATE_AND_UPDATE”,
“mergeMode”: “REPLACE”,
“reportMode”: “FULL”,
“skipExistingCheck”: false,
“sharing”: false,
“skipNotifications”: false,
“skipAudit”: false,
“datasetAllowsPeriods”: false,
“strictPeriods”: false,
“strictDataElements”: false,
“strictCategoryOptionCombos”: false,
“strictAttributeOptionCombos”: false,
“strictOrganisationUnits”: false,
“strictDataSetApproval”: false,
“strictDataSetLocking”: false,
“strictDataSetInputPeriods”: false,
“requireCategoryOptionCombo”: false,
“requireAttributeOptionCombo”: false,
“skipPatternValidation”: false,
“ignoreEmptyCollection”: false,
“force”: false,
“firstRowIsHeader”: true,
“skipLastUpdated”: false,
“mergeDataValues”: false,
“skipCache”: false
},
“description”: “Import process completed successfully”,
“importCount”: {
“imported”: 0,
“updated”: 0,
“ignored”: 0,
“deleted”: 0
},
“conflicts”: ,
“rejectedIndexes”: ,
“dataSetComplete”: “false”
}
}
Operation 1 complete in 264ms
Expression complete!
Removed configuration from final state
Completed step DHIS2 in 1.147s
Final memory usage: [step 20mb] [system 93mb]
Run complete with status: success

mbebana · March 26, 2024, 11:21pm

I created separated step in the workflow with the code using fn
fn((state) => {
const { results } = state.data;
…

below is the result with fail :
Memory limit: 500mb
Timeout: 600s
Versions:
▸ node.js 18.19.1
▸ worker 1.1.1
▸ @openfn/language-common 1.12.0
▸ @openfn/language-dhis2 4.0.3
▸ @openfn/language-kobotoolbox 2.1.0
Executing f9b7599d-894a-4581-af41-1301c6187171
Starting step Data Trans
[linker] loading module @openfn/language-common
[linker] Loading module @openfn/language-common from /tmp/openfn/worker/repo/node_modules/@openfn/language-common_1.12.0/dist/index.cjs
Resolved adaptor @openfn/language-common to version 1.12.0
Executing expression (1 operations)
Timeout set to 300000ms
Starting operation 1
Removed configuration from final state
Failed step Data Trans after 894ms
TypeError: TypeError: Cannot read properties of undefined (reading ‘map’)
{“message”:“TypeError: Cannot read properties of undefined (reading ‘map’)”,“name”:“RuntimeError”}
Check state.errors.83273986-da56-4263-8c6f-bd2c8fe7a89a for details.
Run complete with status: fail
TypeError: TypeError: Cannot read properties of undefined (reading ‘map’)

mbebana · March 26, 2024, 11:33pm

Dear @mtuchi
just one clarification,
I removed the step with fn and used one step (adaptater = DHIS2) with full code,
It’s working and now record is inserted.
Now, i would like to request your help to add separated step to perform data transformation and want apply filter on
“Moughataa”: “Barkeol”
To send only data where Mouaghtaa = “Barkeol”
I would like also use the date “Date”: “2019-05-09” to define DHIS2 period.

Well appreciated

mbebana · March 27, 2024, 12:55am

More clarifications, to understand the use case :
Workflow start with first step Webhook trigger
2nd step is kobotoolbox adaptor with getsubmissions using kobotoolbox form id
3nd step is DHIS2 adaptor using code which already shared.
Issues :
1- if we add new data from kobotoolbox, data isn’t sent to DHIS2 (no thing from openFn Workflow)
2- manual executing DHIS2 step by creating new order with input = {} we have an error message (map property isn’t recognized)
3- executing new order with input json which is already shared adds new record in DHIS2 successfully

We need to execute the workflow and send data to DHIS2 automatically asap new record is added in the Kobotoolbox form.

mtuchi · March 27, 2024, 7:49am

Hello @mbebana, I am not sure why the mapping did not work. It’s possible the operation failed because there was no state.data.results But if i understand correctly your workflow will look something like this

Let me break it down…

1st: A Webook Trigger
This will be our initial input state. And the webhook request will have the formId data Eg: state.data.formId or somewhere in state.data

2nd: Fetching KoboToolbox Submissions
In this step we use the formId obtained from the webhook step and fetches submissions for that formId. This step will use the latest version kobotoolbox adaptor.
For the expression see sample code below

// An example of formId: aXecHjmbATuF6iGFmvBLBX
getSubmissions(
  (state) => ({ formId: state.data.formId }),
  (state) => {
    // The console.log below is to inspect the results after fetching the form submissions
    // Make sure you remove it after your done testing
    console.log(state.data);
    return state;
  }
);

3rd: Transform Submission Results to DHIS2 Data Model
The initial input for this step is the final output of 2nd Step Fetch Kobotoolbox Submissions. And we can access the result in state.data. For transformation step we recommend you use a common adaptor

See sample code for this step

fn((state) => {
  const { results } = state.data;

  state.dataValues = results
    .filter((rs) => rs.Moughataa === "Barkeol") // Filter data
    .map((rs) => ({
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: rs.QteDisp, //# of OPV0 doses given
      // value: '520',
    }));

  // Remember to remove this console.log, when done debugin
  console.log("Mapped DataValues", state.dataValues);
  return state;
});

Note: I have added a filter logic that make sure we only map the results where Moughataa === "Barkeol"

4th: Create dataValueSets
This step will use the DHIS2 mapping from the previous step [state.dataValues] to create the dataValues

See sample code

create("dataValueSets", (state) => ({
  dataSet: "BfMAe6Itzgt", // Child Health
  period: "202402", // Feb 2024
  orgUnit: "DiszpKrYNg8", // Ngelehun CHC
  dataValues: state.dataValues,
}));

Note: for dataValues we are using the values from state.dataValues which was mapped in a previous step [Transform Submission Results to DHIS2 Data Model]

I hope that shade some light on how you can implement your workflow.

mbebana · March 27, 2024, 9:35am

Hello @mtuchi Thank you again for your availability.
In the first step (webhook), how we can specify the kobotoolbox FormId to request.
kindly keep in mind that we should trigger sending data to DHIS2 every time we have new form submission in Kobotoolbox.

Thank you

mtuchi · March 27, 2024, 10:50am

Hey @mbebana ,
I have created for you two video tutorials

The first option is using a Cron based trigger.
This approach allows you to fetch all submission from kobotoolbox for a specific formId, and sync the data to DHIS2 in bulk

The second approach, Is using the webhook trigger
With this approach you will have to setup a REST Service on Kobotoolbox and add the Webhook URL from OpenFn. Each submission will be sent to OpenFn through the webhook URL and the workflow will be triggered and data will be synced to DHIS2 if there is no error

aleksa-krolls · March 27, 2024, 11:21am

@mbebana if you want to the workflow to run in real-time (i.e., every time a new Kobo is submitted, forward the form data to OpenFn), then I would recommend using the webhook trigger in your OpenFn workflow.

In addition to the info Mtuchi shared, be sure to check out these pages as well:

This docs page on the Kobo adaptor and how to configure the webhook in your Kobo project’s Project Settings
This tutorial on docs.openfn.org for more details.

mbebana · March 27, 2024, 11:49am

Thank you very much Dears @mtuchi and @aleksa-krolls for your helpful posts.
it seems to be progressing well!
Now, i received data from Kobotoolbox for every submission but the code early shared by Emmanuel has some errors :

TypeError: TypeError: Cannot read properties of undefined (reading 'filter')

{"message":"TypeError: Cannot read properties of undefined (reading 'filter')","name":"RuntimeError"}

Check state.errors.fcf845b0-c5ba-45e0-adba-c735e58d8a9a for details.

Edge function (state) { return Boolean(!state?.errors?.["fcf845b0-c5ba-45e0-adba-c735e58d8a9a"] ?? true) } returned false; e76bed8b-06e5-4d63-8107-d30d2860dbc9 will NOT be executed

Run complete with status: fail TypeError: TypeError: Cannot read properties of undefined (reading 'filter')

Below is the code with this issue : (intermediate common adaptor step between webhook and DHIS2)

fn((state) => {
  const { results } = state.data;

  state.dataValues = results
    .filter((rs) => rs.mad1 === "Fievrevalleerift") // Filter data
    .map((rs) => ({
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: rs.NbreMorts, //# of OPV0 doses given
      // value: '520',
    }));

  // Remember to remove this console.log, when done debugin
//  console.log("Mapped DataValues", state.dataValues);
  return state;
});

And for your information, i changed the form. Below is the json input which has been received :

{
  "data": {
    "Grp_troupeau/NbreMalades": "15",
    "Grp_troupeau/Sexe": "m_le",
    "GrpAgent/Adresse_professionnelle": "Test",
    "Numero": "6786",
    "_xform_id_string": "aKeNifn4yEDKri9M6NbBK2",
    "GrpAgent/TitreAgent": "Assistant",
    "GrpLocSuspicion/Nom_aire_dabattage": "Test",
    "_submission_time": "2024-03-27T11:39:42",
    "Grp_date_suspicion/Date_premier_signe": "2024-03-20",
    "GrpAgent/Pr_noms_et_nom": "Test",
    "GrpAgent/StatutAgent": "prive",
    "GrpLocSuspicion/Wilaya": "Assaba",
    "meta/instanceID": "uuid:f8e72b9c-dcbd-42be-8e92-cade68b69b35",
    "_id": 325101636,
    "__version__": "v8QUwvpCZJ33vba7ujfeAz",
    "_uuid": "f8e72b9c-dcbd-42be-8e92-cade68b69b35",
    "GrpMaladieEvol/liste_lesions": "hypertrophiefoieFVRL",
    "_submitted_by": null,
    "_tags": [],
    "ajouter_un_prelevement": "non",
    "GrpLocSuspicion/Commune": "Hseiy Tin",
    "Grp_troupeau/effectif": "34",
    "_notes": [],
    "formhub/uuid": "92241f7470e842308405ee21e2219dc3",
    "GrpMaladieEvol/qMad1": "Fievrevalleerift",
    "_validation_status": {},
    "_geolocation": [
      null,
      null
    ],
    "Voulez_vous_ajouter_des_mesure": "non",
    "Grp_date_suspicion/Date_de_rapportage": "2024-03-20",
    "Grp_troupeau/NbreMorts": "9",
    "_attachments": [],
    "GrpLocSuspicion/lieuSuspicion": "aire_abattage",
    "Grp_date_suspicion/Date_de_declaration": "2024-03-20",
    "GrpMaladieEvol/liste_signes": "FrequenceeleveavortementsFVR",
    "GrpLocSuspicion/Moughataa": "Boumdeid",
    "Grp_troupeau/Age": "adulte",
    "_status": "submitted_via_web",
    "Grp_troupeau/liste_anisensibles": "BovinsFVR"
  },
  "request": {
    "headers": {
      "accept": "*/*",
      "accept-encoding": "gzip, deflate",
      "content-length": "1456",
      "content-type": "application/json",
      "host": "app.openfn.org",
      "sentry-trace": "f2c9ca96d29f4b91bafc6e1c0cc0d862-8d94d9b6177af132-0",
      "user-agent": "KoboToolbox external service #h4vU7brhnnwnH2CmVc3Cq5",
      "via": "1.1 google",
      "x-cloud-trace-context": "178dc072b7819dd462b16fbb392504b1/6785961645902563113",
      "x-forwarded-for": "54.144.106.179,34.36.242.233",
      "x-forwarded-proto": "https"
    }
  }
}

mtuchi · March 27, 2024, 4:44pm

Hello @mbebana , I notice two things

1st: On submission data
The submission data from kobotoolbox does not have the results key, All results are returned in state.data and not state.data.results
This is because we are now getting each submission in real time. So you only get one result. So the mapping for the submitted result will be like this

fn((state) => {
  state.dataValues = {
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: state.data['Grp_troupeau/NbreMorts'], 
      // value: '520',
    }));

  // Remember to remove this console.log, when done debugin
//  console.log("Mapped DataValues", state.dataValues);
  return state;
});

2nd: The Filter
The value you want to check if it matches is available on this path state.data['GrpMaladieEvol/qMad1']. and not rs.mad1

Suppose you don’t want to run the workflow if there is no match. Then on your workflow diagram you can add Match Javascript Expression right after the webhook trigger that will check if state.data['GrpMaladieEvol/qMad1'] === "Fievrevalleerift" is true.

If state.data['GrpMaladieEvol/qMad1'] === "Fievrevalleerift" is true the workflow will proceed with execution when form submission is received.

See example below

mbebana · March 28, 2024, 2:10am

Dear @mtuchi,
I created new workflow according to your previous post with 2 steps :

Step 1 : Adaptor = common

fn((state) => {
  state.dataValues = {
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: state.data['Grp_troupeau/NbreMorts'], 
      // value: '520',
    };

  // Remember to remove this console.log, when done debugin
//  console.log("Mapped DataValues", state.dataValues);
  return state;
});

Step 2 : Adaptor = DHIS2

 create("dataValueSets", (state) => ({
  dataSet: "BfMAe6Itzgt", // Child Health
  period: "202402", // Feb 2024
  orgUnit: "DiszpKrYNg8", // Ngelehun CHC
  dataValues: state.dataValues,
}));

After form submission, first step is Ok but DHIS2 step has following error message :

 Sending post request to https://play.im.dhis2.org/dev/api/dataValueSets
Request failed with status code 409
{
  "httpStatus": "Conflict",
  "httpStatusCode": 409,
  "status": "ERROR",
  "message": "An error occurred, please check import summary.",
  "response": {
    "responseType": "ImportSummary",
    "status": "ERROR",
    "importOptions": {
      "idSchemes": {},
      "dryRun": false,
      "async": false,
      "importStrategy": "CREATE_AND_UPDATE",
      "mergeMode": "REPLACE",
      "reportMode": "FULL",
      "skipExistingCheck": false,
      "sharing": false,
      "skipNotifications": false,
      "skipAudit": false,
      "datasetAllowsPeriods": false,
      "strictPeriods": false,
      "strictDataElements": false,
      "strictCategoryOptionCombos": false,
      "strictAttributeOptionCombos": false,
      "strictOrganisationUnits": false,
      "strictDataSetApproval": false,
      "strictDataSetLocking": false,
      "strictDataSetInputPeriods": false,
      "requireCategoryOptionCombo": false,
      "requireAttributeOptionCombo": false,
      "skipPatternValidation": false,
      "ignoreEmptyCollection": false,
      "force": false,
      "firstRowIsHeader": true,
      "skipLastUpdated": false,
      "mergeDataValues": false,
      "skipCache": false
    },
    "description": "The import process failed: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.util.ArrayList<org.hisp.dhis.dxf2.datavalue.DataValue>` from Object value (token `JsonToken.START_OBJECT`)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 81] (through reference chain: org.hisp.dhis.dxf2.datavalueset.DataValueSet[\"dataValues\"])",
    "importCount": {
      "imported": 0,
      "updated": 0,
      "ignored": 0,
      "deleted": 0
    },
    "conflicts": [],
    "rejectedIndexes": []
  }
}
Removed configuration from final state
Failed step Vers DHIS2 after 1.145s
JobError: Request failed with status code 409
{"message":"Request failed with status code 409","name":"JobError"}

mtuchi · March 28, 2024, 6:29am

Hello @mbebana i think that error suggest the dataValues can not be type Object i think they need to be grouped into an array.

Can you try to wrap dataValues into an array
Eg:

  state.dataValues = [{
      categoryOptionCombo: "Prlt0C1RF0s", //Fixed <1yr
      dataElement: "x3Do5e7g4Qo", // OPV0 doses given
      value: state.data['Grp_troupeau/NbreMorts'], 
      // value: '520',
  }];

mbebana · March 28, 2024, 12:32pm

Dear @mtuchi
Many thanks and appreciations for your very valuable help.
Now it’s working fine!
I already took a look on openFn online documentation which is very interesting but you can advise if there’s is recommended documentation and Tuto for OpenFn/Javascript/Adaptor languages/Json…
Thank you again to you, to @aleksa-krolls and to OpenFn community.

ayodele · March 28, 2024, 3:53pm

Hi @mbebana

It’s Ayodele here (PM at OpenFn).

Glad to see you got the help you needed. Many thanks for the recommendation, we are working on making more tutorials and examples available through the documentation website.

I’d love to learn more about your use cases and if there are any platform specific requests you might have. I am available at ayodele[at]openfn[dot]org.

mtuchi · March 29, 2024, 8:40am

Hello @mbebana , we recently released a new docs section that might help with getting started - see the new Job Writing Guide.