# Fields not updating during insert

**URL:** <https://community.openfn.org/t/fields-not-updating-during-insert/628>\
**Category:** Job Writing\
**Created:** [November 19, 2024, 2:43pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628 "2024-11-19T14:43:06Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [November 19, 2024, 2:43pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/1 "2024-11-19T14:43:06Z")

</div>

Hi all, when trying to insert records using the Salesforce adaptor version 4.6.2, there are some fields that are not updated despite being in the final JSON before the insert. There is also no error message that indicates the field wasn’t updated. In the JSON screenshot, for example, the field `Why_tank_has_no_water__c?` is filled, but it is not reflected in Salesforce (`Why tank has no water ?`). There are other fields where this has happened as well. Do you know what could be the problem? Thank you.

 ![JSON-example](https://community.openfn.org/uploads/default/original/1X/731eef9a10af1c93fcbee5e4c739886af4a02c8f.png)  
 ![NotUpdatedSFRecord](https://community.openfn.org/uploads/default/original/1X/b1e20115816362469b67c5839082576755301acf.png)

---

<div class="post-metadata">

**Author:** ![aleksa-krolls](https://community.openfn.org/user_avatar/community.openfn.org/aleksa-krolls/32/156_2.png) [@aleksa-krolls](https://community.openfn.org/u/aleksa-krolls)\
**Post date:** [November 19, 2024, 4:03pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/2 "2024-11-19T16:03:35Z")

</div>

@djohnson strange that there is no error message! Could you (1) confirm which adaptor function you’re using (e.g., `insert`, `bulk`) and (2) share a snippet of your job code to see if we can spot any issue?

Then, is there any chance there is a permissions issue or Salesforce-side automation triggered after the OpenFn insert operation?

One test to try… via the web browser, if you manually input the same value (e.g., `"No access to water in school"`) into the `Why_tank_has_no_water__c` field and then “save” the record, does the record save? Or does Salesforce provide any error message/feedback?

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [November 20, 2024, 9:18am UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/3 "2024-11-20T09:18:33Z")

</div>

Yeah it’s pretty strange. I’m using `insert`. Here is the snippet. I chunked the records because I was running into timeout issues when working with the entire array at once.

```auto
fn(state => {
    let array = state.filteredNewMETankRecords
    let chunkMETankRecordsArray = []
    const chunkSize = 10;
    for (let i = 0; i < array.length; i += chunkSize) {
        const chunk = array.slice(i, i + chunkSize);
        // do whatever
        chunkMETankRecordsArray.push(chunk)
    }
    if (chunkMETankRecordsArray.length == 0 || chunkMETankRecordsArray == null || chunkMETankRecordsArray == undefined) {
        console.log('No records to add')
    } else {
        console.log(`There are ${chunkMETankRecordsArray[0].length} records to add`)
    }
    return { ...state, chunkMETankRecordsArray }
})

each(
    '$.chunkMETankRecordsArray[*]',
    bulk(
        'ME_tank_information__c',
        'insert',
        { extIdField: state.objIDMapping['ME_tank_information__c'], failOnError: true, allowNoOp: true },
        state => state.data
    )
)

```

I’m able to update the field in the browser though.

 ![updateWhyNoWater](https://community.openfn.org/uploads/default/original/1X/0264e25081ccb7c4b77ee5db16b8d92aef9d0ecf.png)

---

<div class="post-metadata">

**Author:** ![aleksa-krolls](https://community.openfn.org/user_avatar/community.openfn.org/aleksa-krolls/32/156_2.png) [@aleksa-krolls](https://community.openfn.org/u/aleksa-krolls)\
**Post date:** [November 20, 2024, 11:24am UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/4 "2024-11-20T11:24:27Z")

</div>

@djohnson I see that in you’re job you’ve specified `insert` as your operation, but then it looks like you’re passing in an `.extIdField`… do you maybe want to use `upsert` instead?

In Salesforce “insert” operations will always create records and do not support checking for existing records using an externalId. If you want to insert _or_ update, then use the “upsert” operation (where you must specify an externalId).

Check out the [adaptor docs](https://docs.openfn.org/adaptors/packages/salesforce-docs#bulk) for more examples, but I think that issue might be that you’re using `insert` + passing an `externalId`. Can you try either removing the `extIdField` argument or changing the operation to `upsert`?

Or @mtuchi do you spot anything else here that might be off?

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [November 20, 2024, 2:29pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/5 "2024-11-20T14:29:44Z")

</div>

I was using `insert` here because I have to create records. The records are inserted to the `ME_tank_information __c` object which lives under another object. The record for the parent object is created during the same workflow, so I need to fetch the ID of that newly created record and insert the `ME_tank_information__ c` records there. Is this the right way of achieving this?

---

<div class="post-metadata">

**Author:** ![aleksa-krolls](https://community.openfn.org/user_avatar/community.openfn.org/aleksa-krolls/32/156_2.png) [@aleksa-krolls](https://community.openfn.org/u/aleksa-krolls)\
**Post date:** [November 21, 2024, 3:03pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/6 "2024-11-21T15:03:55Z")

</div>

@djohnson Not quite… so `extIdField` is for the target record you’re trying to import. In the below example, `Patient_ID __c` is the external Id for `Patient__ c`:

```auto
bulk(
  "Patient__c",
  "upsert",
  { extIdField: "Patient_ID__c" },
  [
    {
      Patient_ID__c: state.data.patientUid,
      Name: state.data.patientName
    },
  ]
);

```

When you want to link a child record to the parent, you need to map the relationship in the payload body (e.g., `LookupField __r.ParentRecordExtId__ c`), and not specify it is a separate argument in the `bulk()` function. For example:

```auto
bulk(
  "Patient__c",
  "upsert",
  { extIdField: "Patient_ID__c" },
  [
    {
      Patient_ID__c: state.data.patientUid,
      Name: state.data.patientName,
      //link to Patient to parent HH using external id of parent record
      Household __r.HH_ID__ c: state.data.householdId 
    },
  ]
);

```

Alternatively, you can first query Salesforce to get back the record `id` of the parent record (e.g., `householdId`)… and then directly map this when you create the patient - something like this:

```auto
bulk(
  "Patient__c",
  "upsert",
  { extIdField: "Patient_ID__c" },
  [
    {
      Patient_ID__c: state.data.patientUid,
      Name: state.data.patientName,
      Household__c: householdId
    },
  ]
);

```

Check out [this Salesforce post](https://trailhead.salesforce.com/trailblazer-community/feed/0D54S00000A7dS9SAJ) for more background info on Salesforce relationship fields and notation for these. And remember that `bulk()` is using the Salesforce [Bulk API](https://developer.salesforce.com/docs/atlas.en-us.246.0.api_asynch.meta/api_asynch/bulk_api_2_0.htm) in case you want to look at those official docs for more on Salesforce rules.

And again, the [`bulk()` examples](https://docs.openfn.org/adaptors/packages/salesforce-docs#bulk) in the OpenFn Adaptor docs also show this mapping notation in action.

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [November 26, 2024, 8:24am UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/7 "2024-11-26T08:24:36Z")

</div>

Thanks for the examples. I will test these and see if the problem disappears.

---

<div class="post-metadata">

**Author:** ![taylordowns2000](https://community.openfn.org/user_avatar/community.openfn.org/taylordowns2000/32/165_2.png) [@taylordowns2000](https://community.openfn.org/u/taylordowns2000)\
**Post date:** [November 26, 2024, 9:53am UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/8 "2024-11-26T09:53:28Z")

</div>

Hope they helped, @djohnson !

We’ve just added a “mark as solution” button to the forum. If @aleksa-krolls 's examples do the trick would you mind clicking the little grey check box (near the heart) at the bottom of her post so that others can easily find it?

And if not… let us know what errors you’re seeing and we’ll try again 😂

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [November 26, 2024, 2:24pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/9 "2024-11-26T14:24:35Z")

</div>

Thanks Taylor, I’ve marked it as a solution.

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [January 7, 2025, 12:53pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/10 "2025-01-07T12:53:25Z")

</div>

Hi, unfortunately, I am still having problems with inserting records. I am using the following

```auto
fn(state => {
    let array = state.filteredGRRecords
    let chunkGRRecordsArray = []
    const chunkSize = 200;
    for (let i = 0; i < array.length; i += chunkSize) {
        const chunk = array.slice(i, i + chunkSize);
        // do whatever
        chunkGRRecordsArray.push(chunk)
    }
    if (chunkGRRecordsArray.length == 0 || chunkGRRecordsArray == null || chunkGRRecordsArray == undefined) {
        console.log('No GR records to insert')
    } else {
        console.log(`Inserting ${chunkGRRecordsArray[0].length} GR records`)
    }
    return { ...state, chunkGRRecordsArray }
})

each(
    '$.chunkGRRecordsArray[*]',
    bulk(
        'GR_Engagement__c',
        'insert',
        { failOnError: true },
        state => state.data
    )
)

```

And here is part of `state.data`

 ![image](https://community.openfn.org/uploads/default/original/1X/573ab8d5d8ec460cd9fb433304705c749a44d2ef.png)

But for some reason, some of these fields are not updated in Salesforce.

 ![image](https://community.openfn.org/uploads/default/original/1X/00d2b85e8d10f8d168a8db0faf85479669fed674.png)

Changing to `upsert` doesn’t work because there is no External ID field for this object. Is the issue with `state => state.data`?

---

<div class="post-metadata">

**Author:** ![mtuchi](https://community.openfn.org/user_avatar/community.openfn.org/mtuchi/32/81_2.png) [@mtuchi](https://community.openfn.org/u/mtuchi)\
**Post date:** [January 7, 2025, 2:16pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/11 "2025-01-07T14:16:50Z")

</div>

Hi @djohnson ,

From your job code, everything looks fine. However, it might help to get more clarity from the Salesforce response. Does the response contain any errors? You can find the response details in `state.references`.

I’ve cleaned up your job code a bit and switched to using the `insert` function since you’re already chunking records by 200, which is the standard batch size. Note that the Salesforce adaptor already includes a helper function called `chunk()` to handle record chunking.

```javascript
fn((state) => {
  state.chunkGRRecordsArray = chunk(state.filteredGRRecords, 200);
  if (!state.chunkGRRecordsArray || state.chunkGRRecordsArray.length === 0) {
    console.log("No GR records to insert");
    return state;
  }
  console.log(`${state.chunkGRRecordsArray.length} chunks of GR records`);
  return state;
});

each(
  $.chunkGRRecordsArray,
  insert("GR_Engagement__c", (state) => {
    console.log(`Inserting ${state.data.length} GR records`);
    return state.data;
  })
);

```

**A couple of things to note:**

1. The `chunk()` function provided by the adaptor simplifies chunking records into manageable sizes, so you don’t need to implement it from scratch.
2. If you’re dealing with a large number of records, consider using the `bulk` operation. The Bulk API supports a batch size of up to 10,000 records and automatically chunks records if the dataset exceeds this limit.

> bulk(“GR\_Engagement\_\_c”, “insert”, { failOnError: true }, $.filteredGRRecords)

3. For smaller datasets `(<=200)` or real-time operations, your current approach with `insert` works well.

Let me know if you need further help! 😊

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [January 7, 2025, 3:34pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/12 "2025-01-07T15:34:04Z")

</div>

Unfortunately, there aren’t any errors in `state.references`

```auto
{
  "logger": {},
  "references": [
    [
      {
        "id": "a3fRo000000SLfpIAG",
        "success": true,
        "errors": []
      },
      {
        "id": "a3fRo000000SLfqIAG",
        "success": true,
        "errors": []
      }
    ]

```

Thanks for code suggestions. I switched to chunking and `each` after using `bulk` because of timeout issues.

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [January 7, 2025, 3:47pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/13 "2025-01-07T15:47:23Z")

</div>

Thanks, your suggestion worked! The fields appear to be updated in Salesforce.

 ![image](https://community.openfn.org/uploads/default/original/1X/e43b9ebe1f7dbf88187e58b8308c3cc66228d024.png)

Do you know what was wrong the previous code?

---

<div class="post-metadata">

**Author:** ![mtuchi](https://community.openfn.org/user_avatar/community.openfn.org/mtuchi/32/81_2.png) [@mtuchi](https://community.openfn.org/u/mtuchi)\
**Post date:** [January 8, 2025, 9:11pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/14 "2025-01-08T21:11:15Z")

</div>

@djohnson  
In most cases the Bulk operation timeout because of [Salesforce API limits and Governance](https://developer.salesforce.com/docs/atlas.en-us.210.0.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_bulkapi.htm). Bulk operations must adhere to API limits, such as the number of batches or records that can be processed per 24-hour period. High API usage across the org can introduce delays.

But you can increase the `pollTimeout` and `pollInterval` incase of timeout issues. [See `bulk` options docs here](https://docs.openfn.org/adaptors/packages/salesforce-docs#bulk)

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [January 9, 2025, 2:46pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/15 "2025-01-09T14:46:21Z")

</div>

Thanks for the explanation, but I was wondering more about why some fields update and others don’t.

---

<div class="post-metadata">

**Author:** ![aleksa-krolls](https://community.openfn.org/user_avatar/community.openfn.org/aleksa-krolls/32/156_2.png) [@aleksa-krolls](https://community.openfn.org/u/aleksa-krolls)\
**Post date:** [January 13, 2025, 9:29pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/16 "2025-01-13T21:29:20Z")

</div>

Hey @djohnson - it’s hard to say why those fields weren’t updating offhand without a better understanding of your system and closer inspection of the data you attempted sending. Mtuchi’s suggested solution uses the regular REST api to `insert` records, and not the Bulk api (which create async data import jobs in salesforce), so Saleforce does processes these bulk requests a little differently.

Typically, a bulk API job is considered “successful” if the records themselves are imported in Salesforce without critical errors (e.g., validation rule failures or missing required fields). However, if there are issues with specific fields in the data or the target Salesforce configuration (e.g., field-level security, read-only fields, or data type mismatches), those fields may not update while the import job still reports as successful and I’ve noticed these field-level issues don’t always throw an error in the bulk api response.

Some troubleshooting tips I’d explore:

1. Go to `Setup` \> `Bulk Data Load Jobs` to inspect the data sent by OpenFn to your Salesforce org and the response… you can download csv files of the requests and responses sent via the API to verify what you’re seeing in the OpenFn logs. [See Salesforce docs](https://help.salesforce.com/s/articleView?id=sf.monitoring_async_api_jobs.htm&type=5) for more on this monitoring interface.
2. Consider if there is any salesforce-side automation triggered _after insert_. Downstream automation, like Apex Triggers, Flows, or Processes, could override or fail to update specific fields, depending on their logic. You could try setting up [Debug Logs](https://help.salesforce.com/s/articleView?id=platform.code_debug_log.htm&type=5) on the Salesforce-side to track downstream automation action to discover if they might be the culprit.
3. In your original job code, also did you try adding a `console.log()` to troubleshoot what `state.data` looked like before it was sent to the salesforce api? For example:

```auto
each(
    '$.chunkGRRecordsArray[*]',
    bulk(
        'GR_Engagement__c',
        'insert',
        { failOnError: true },
        state => {
            console.log('payload sent:', state.data); // Logs the data to the console
            return state.data;
        }
    )
);

```

1. I _think_ [this](https://trailhead.salesforce.com/trailblazer-community/feed/0D54V00007T4SJDSA3) might also still be an issue, even if the latest bulk api version, in case this happens to be the case with your data.

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [January 22, 2025, 6:16pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/17 "2025-01-22T18:16:20Z")

</div>

Thanks for your suggestions. I will try them out and get back to you. In the meantime, I noticed that the `fetchSubmissions` function in the `surveycto` adaptor doesn’t retrieve all the fields. This may be part of the reason why the fields I expected to update weren’t updated.

Here is a screenshot from downloading the `csv` from the SurveyCTO console.

 ![csvFromSurveyCTO](https://community.openfn.org/uploads/default/original/1X/46a45f9bc98fbc25ad9b0530dc81a72d5fee1c7f.png)

And here is the json response for the same form

```auto
"zone_id": "a2C2M000005G8Sx",
        ...
        "engagement_type": "1",
        "met_officer": "",
        "no_met_officer": "",
        "met_other": "",
        "met_other_who": "",
        "topics": "",
        "topics_1": "",
        "topics_2": "",
        "topics_3": "",
        "topics_4": "",
        "topics_5": "",
        "topics_6": "",
        "topics_997": "",
        "topics_other": ""

```

`topics` is filled in the `csv`, but not in the `json` response. Do you know what could be causing this?

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [January 23, 2025, 3:59pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/18 "2025-01-23T15:59:06Z")

</div>

So from the `result` log from `Setup` \> `Bulk Data Load Jobs`. It looks like everything was created successfully. But when I look at the `request` log, `topics` is still not being mapped  
 ![image](https://community.openfn.org/uploads/default/optimized/1X/3d89414939237c65c6ffd965499ee12d751a4024_2_690x22.png)

I also made sure that each object contains all the keys using

```auto
let fullKeys = Object.keys(anotherObj)
let fullObj = {}
for (const key of fullKeys) {
            fullObj[key] = '';
        }
GRRecords.forEach(e => {
        for (var key in fullObj) {
            e[key] = e[key] || fullObj[key]
        }
    })

```

where `anotherObj` contains all keys.

I also confirmed that there are no triggers or flows touching this object.

The payload from `console.log` seems to be correct, so I think the issue is that values from some fields aren’t being retrieved with `fetchSubmissions`. See here for an example of what is fetched using another API

```auto
  'zone_id': 'a2C2M000005G8Sx',
...
 'engagement_type': '1',
 'topics_1': '1',
 'topics_2': '1',
 'topics_3': '1',
 'topics_4': '1',
 'topics_5': '1',
 'topics_6': '1',

```

What’s strange is that in some cases, it correctly returns the values for `topics`. So I’m not sure where the issue is. I’m happy to jump on a call about this if it’s easier.

---

<div class="post-metadata">

**Author:** ![aleksa-krolls](https://community.openfn.org/user_avatar/community.openfn.org/aleksa-krolls/32/156_2.png) [@aleksa-krolls](https://community.openfn.org/u/aleksa-krolls)\
**Post date:** [January 23, 2025, 10:41pm UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/19 "2025-01-23T22:41:52Z")

</div>

Looking like @djohnson it’s been a long time since anyone has used the `surveycto` adaptor, so we’ll have to check it out - I will flag to our product team.

If you’re in a hurry, in the meantime I might recommend using either (a) the generic [request() function](https://docs.openfn.org/adaptors/packages/surveycto-docs#request), or (b) the [http adaptor](https://docs.openfn.org/adaptors/packages/http-docs#get) until we can sort this out.

FYI [see here](https://github.com/OpenFn/adaptors/blob/main/packages/surveycto/src/Adaptor.js#L92) for our adaptor source code in case you want to check out what http requests it sends behind the scenes when `fetchSubmissions()` is called. Your GET request will probably look something like this:  
GET ‘[https://servername.surveycto.com/api/v2/forms/data/wide/json/formid?date=[${dateToFilterBy}]](https://servername.surveycto.com/api/v2/forms/data/wide/json/formid?date=%5B$%7BdateToFilterBy%7D%5D)’

---

<div class="post-metadata">

**Author:** ![djohnson](https://community.openfn.org/letter_avatar_proxy/v4/letter/d/0ea827/32.png) [@djohnson](https://community.openfn.org/u/djohnson)\
**Post date:** [January 24, 2025, 6:51am UTC](https://community.openfn.org/t/fields-not-updating-during-insert/628/20 "2025-01-24T06:51:39Z")

</div>

Thanks, I will give those a try.