All Collections
Admin
Studio settings
How to set up a Data Source
How to set up a Data Source
Ha Ngan Nguyen avatar
Written by Ha Ngan Nguyen
Updated over a week ago

Data source is a way of creating and managing jobs. After setting this up, you can create jobs using ‘From Scan’ or ‘Sync from Source’.

To set up a data source, go to your Studio Settings, under Production Settings and select ‘Data sources’. From here, you will see all of your existing Data Sources, their type of connection as well as the number of clients that have the respective Data Source assigned.

Changes to your Data Source will also be managed from here by hovering over the data source and selecting ‘Edit’.

Click ‘Add’ to create a new data source. You will then have the option to add information for the method you will be using. And I will go over them one by one with you.

  • Name: Your data source name

  • Method: We now support 5 ways to upload your data: FTP - Google cloud - Amazon S3 - Feed URL - File upload

  • Protocol: Choose a suitable network protocol from FTP (File transfer protocol) - SFTP (SSH file transfer protocol) - SFTP (SSH file transfer protocol with private key)

  • Port: Controlled port

  • Host: Where do you draw the data

  • Username: Your username in the host

  • Password: Your password for the username

  • File key: its the name of the main file, please note that the encoding supports UTF-8

  • File path: Where is your file located

  • File type: We supported 3 file types CSV - JSON - XML

  • CSV delimiter: You can choose either Comma - Semicolon - TAB character - Other to separate the data.

Note:
​If you are using a JSON file to be your data source, please make sure that the first item and its attributes have a value, it cannot be 'NULL'.

  • Frequency: You can decide how often the data source gets updated automatically. It can be either Hourly - Daily - Weekly - Monthly. If you choose ‘monthly’ you will have to decide which day of the month. If you choose ‘weekly’ you will have to decide which day of the week.

  • Hour in day: The hour in the day that the data get updated.

  • Minute in hour: The minute in the day that the data get updated.

There is also a tick box to allow delta update from the data source. Delta files are files that contain only the changes since the last file upload. The delta files can be used to Add products - Update existing products - Delete products - Deactivate/ Activate products…

Note:

  • The delta file should not be bigger than 50 MB.

  • The delta file must have the same file name as the original file in this format: "original filename" + "_" + a number that keeps growing.

  • The delta file must have the same columns as the original file.

  • To delete or deactivate products via delta files, the original file must have correspondent columns 'isDeleted' and 'isDeactivated'. The value of the column must be '1' for the system to understand which products must be deleted or deactivated.

  • CF only supports encoding UTF-8. If your data is Latin encoding, for example, the data will not be displayed properly in Gamma

Creative Force follows the below priority when ingesting date-type data from the data source files.

  1. "MM'-'dd'-'yyyy",

  2. "MM'/'dd'/'yyyy",

  3. "MM' 'dd' 'yyyy",

  4. "dd'.'MM'.'yyyy",

  5. "MM'.'dd'.'yyyy",

  6. "dd'-'MM'-'yyyy",

  7. "dd'/'MM'/'yyyy",

  8. "dd' 'MM' 'yyyy",

  9. "yyyy'-'MM'-'dd",

  10. "yyyy'/'MM'/'dd",

  11. "yyyy' 'MM' 'dd",

  12. "yyyy'.'MM'.'dd",

  13. "yyyy'-'dd'-'MM",

  14. "yyyy'/'dd'/'MM",

  15. "yyyy' 'dd' 'MM",

  16. "yyyy'.'dd'.'MM"

After finishing setting up the data source, you can assign the data source to a client here.

In the ‘Last success’ column, you will find the following statuses:

  • Data ready: After successfully syncing for the first time, the status will show Data ready

  • Wait for first sync: No sync has been succeeded previously

  • Cannot connect: Creative Force cannot connect to the Data source. Please contact our support team for further details.

  • Syncing: Creative Force is in the process of syncing the data

  • Wait to map: The columns have not been mapped, after these have been mapped, the sync can start

Hover over the Last success column, you can see the following information:

  • Data source status as shown above

  • Lastest time the data source was scanned

  • Lastest delta number that was scanned. For example: If your source file is Data Source EU Studio.csv Then the delta file should contain an increment number in its filename and the the last delta number will be shown here Data Source EU Studio_23082403.csv

Did this answer your question?