Using Data Source is a great way to auto-digest data into Creative Force. If you are not yet familiar with Data Source, you can refer to our other articles that guide you through the set up process:

What is a Data Source

How to set up a Data Source

After setting up a Data Source and a syncing schedule, the data will automatically be imported to Creative Force. Along the way, there might be cases where you would need to make some updates or changes to the source file.

This article will explain the logic behind the syncing process and what actions should be taken.

First and foremost, before going into the details, we should clarify the definition of syncing process. In Creative Force, we have three types of syncing processes.

The first sync will happen after you finish creating and setting up the mapping columns for the Data Source. The first sync is only to remember the mapping of the properties. Product data will not be imported. You must select ‘Update from Data Source’ to manually import the data.

If you don’t select ‘Allow delta updates for data source’ in your setup, all the subsequent syncing processes after the first sync will be considered a full sync.

The full sync will scan all available files including the main source file and delta files to look for new values and import those in CF.

The Delta sync happens if you select ‘Allow delta updates for data source’. The delta sync will run after the first sync.

With delta sync, the system will only scan the latest files (delta files) added to the data source and import in the new values or update existing ones

Note:

File size limitation for

Total files for the syncing process should be less than 10.000 files

To check the mapping columns, go to Studio Settings -> Data Source. Hover over the data source you want to check and click Actions -> Mapping Column.

When you have unmapped columns on a running Data Source, you can remap those and click ‘Save’. You will then see further sync options and we will explain the logic in the ‘Sync options’ section of this article.

If your Data Source is using one of the following methods: FTP, Google Cloud Storage, Amazon S3, Brandquad, Akeneo, Airtable and Feed URL, you can fetch the new columns and sync to Creative Force.

Before adding new columns, you must ensure those columns are present in your data source file and the latest delta file.

When you go to the Data Source mapping, you will see the ‘Fetch column’ button on the top right corner.

Clicking ‘Fetch columns’ you will see new columns populating in the mapping setup. You can then adjust the column mappings as needed and click ‘Save’. You will then see further syncying options, and we will explain the logic in this article's ‘Sync options’ section.

When you go to the Data Source mapping, you will see the “Add new’ button.

Enter the column name and select a corresponding property in Creative Force to map then ‘Save’. You will then see further sync options and we will explain the logic in the ‘Sync options’ section of this article.

After saving the updated column mapping using the steps above, you will be prompted to select one of the following options:

Existing product records will not be retroactively updated to the changed mapping.

By selecting this option, the updated column mapping will be automatically applied to all Data Source syncs and updates moving forward from this point. Existing product records from previous syncs will remain unchanged.

This works best if you are planning to capture a new data point moving forward and do not want to affect the existing product data.

Completion time will vary based on size. Once synced, products may be manually updated from Data Source.

By selecting this option, the updated column mapping will be synced to all existing Data Source records, including delta files, if applicable.

Once the sync is completed and the Data Source status returns to “Data Ready,” then products and samples can be manually updated with the new mapping applied by clicking on them and selecting “Update from Data Source”.

Please note that when you have a large data source or a large number of delta files, this sync may take longer to complete.