We have added support for using Salesforce.com upsert method. Using this we can make update/add transfers 3-5 times faster and use a lot less API calls.
The use of the upsert API method inside salesforce enables you to send a list of records/objects to salesforce in the same packet and let salesforce find out if a record should update an existing record or add a new record in salesforce.
However, there are a couple of restrictions on the upsert API call from salesforce's side:
- the key or table link used, may only contain one field
- the field used in the link must be either the salesforce id field or a custom field that has been marked as "External ID" when the field was created in salesforce.com. You will probably want to mark the field as Unique as well.
In RapidiOnline this feature is enabled on a Transfer by having both UPDATE and ADD checked and then selecting the "Disable Dest. Lookup" checkbox.
Please note that the statistics on the transfer will show all records read from the source as either added or updated in salesforce.com. Salesforce.com does not give any feedback to us on if a record were actually different than the existing one or not. If the record exists, it will simple show as updated in the statistics.
Huge API call savingsWhen using this new feature, you skip the lookup to salesforce.com for each record in order to find out if its in salesforce or not. This saves a lot of salesforce API calls (one for each record read from the Source) and thereby also a lot of time. In our tests, we have seen a transfer that previously made 8060 API calls and took 17 minutes, go down to just 31 API calls and 3:2o minutes !
Converting existing transfersIt is not all UPDATE+ADD transfers that can be converted to use the new feature, but for example the Item/Product transfer, the Invoice Lines subtransfer and the Ledger Entry transfers can be changed.
There is a little work to do the change on these transfers - basically you need to:
- create a new field on the Object in salesforce.com that will be the key going forward and mark this field as "External ID" and possibly also as Unique.
- add the new field to the existing transfer field list and run the transfer to populate it in salesforce.com (this ensures that all existing records are not added to salesforce again).
- change the transfer to use the new field in the table link
Using "Disable Dest. Lookup" on UPDATE only transfersWe have also enabled another feature in the release. It is now also possible to use the "Disable Dest. Lookup" feature on a UPDATE only transfer. This allows you to just send all records to salesforce.com for update.
This also saves a lot of API calls but please note that the transfer will fail if the record does not exist in salesforce.com
This feature can be used on the Customer/Account UPDATE transfer.
Thats it.- Done!
Please feel free to contact us to get some help with these optimizations.