Studios considering updates to their internal processes often wonder whether Data Sources are a good fit. In this article, we will cover exactly what a Data Source is, how they work, and what functions make them useful. With this information, you can move forward feeling confident in your next steps.

Within Creative Force, there are a number of different ways that you can bring in product data. You can choose to keep things simple with a file import, or if you have a development team, you can build out an API integration to connect another system, like a PIM, directly to Creative Force.

A third option would be to use what we call a 'Data Source.' With a Data Source, Creative Force can look at a particular location, like an FTP or other storage platform, and wait for new data to be uploaded to that location.

Typically, this is done by another software, like a PIM, that can set up daily, weekly, or monthly exports to a set location. Doing so effectively connects the two systems without the need for a large development project.

If your Product Data System can't export data to an FTP, AWS S3 bucket, or Google Cloud bucket, some customers use scripts to extract data from the first system and upload it to the designated location.

Our Data Source functionality is available to all Enterprise customers, but it may not be the best fit for all customers based on a few different parameters.

A Data Source will work well when the data formatting and needs are the same and consistent. If those needs are changing regularly or prone to having issues (like inconsistent or poor data), then it may be best to stay with a file upload approach, as this is the most forgiving and easiest to correct after the fact. For instance, some commercial studios will shy away from a Data Source as they don't have 100% control over the data they are receiving and its quality.

However, if you are able to ensure consistent data and work with an update frequency of daily or weekly, then a Data Source may be for you. For instance, if you are a Brand or Retailer that is getting your data seasonally, or in some set interval, then a Data Source is a great option to consider.

If you want to have data imported to Creative Force in real-time, then an API integration from your other system would be best, or by file import. Depending on the size of your Data Source, it is recommended to do a daily, weekly, or monthly sync. We will dig into why a little later!

Equally, if you are not getting your data from another system, and either building it manually or getting it from a third-party client, then a Data Source may not be the best solution.

Within Creative Force, you will define the frequency at which Creative Force will scan the Data Source location to see if there are updates. We typically suggest setting the frequency and time of an update to a time when there is little activity within your studio, like the middle of the night or the weekend. That way the update is done when there is little activity on your instance.

Once that is set, Creative Force will scan the data source at the defined interval, looking for new data that can be ingested.

We will not automatically import all the products that are in the Data Source location and put them in your Creative Force instance. Instead, we will place them in our database and store them for you to access at a moment's notice when you try to create a new Job or update Products from the Data Source.
​
Going forward, the area that is between your Creative Force Instance itself and the Data Source location itself will be referred to as the 'Data Source.'

You may be asking why we don't just import all the Product data from the Data Source Location straight into Creative Force. The reason we do not do this is that it can be quite overwhelming to have all of that data in at once in one big bundle in your Instance.

If you can imagine you have hundreds of thousands of Products that are part of your Data Source location that you, as a studio, may want to work on throughout a season, it can be overwhelming to have all of that data in Creative Force as a default.

Best practice is to break up that data into smaller, manageable jobs that are easily identifiable. This is most easily done by setting up specific Sync Rules, which narrow down the scope of imported products by conditional rules and matching value properties in the Data Source.
​
For more information on setting up Sync Rules for your Data Source, see our related article linked below.

To create a job from a Data Source, you would first need to set up the Data Source and assign it to a Workspace. For a dedicated walkthrough on setting up and configuring a new Data Source, please review our related article linked below.

Once you've established the connection and assigned it to a Workspace, you are ready to start the Job import. You can either do this by creating a “Sync Job” that will populate the Products based on the criteria you define, or by creating a Job “From Scan," where CF will then look up the Products on the Data Source file.

Below, we will cover the use case for both options:

Only the initial Data source setup and definition of the criteria are required if you choose to create a job by “Syncing from Data Source”. After that, CF will check for updates in accordance with the schedule you determined in the Data Source Settings.

When your goal is to import products that must meet specific criteria, that is the recommended method. As soon as a new product that meets the set criteria is added to the data source file, it will automatically be added to the job with the next sync.

For instance, you might want to import only products associated with a specific Brand:

For more information on creating a Job from a Data Source Sync, see our related article below.

​

This is useful if your sample team is creating a Job based on a rail, or perhaps a specific shipment of items that are received into the studio. They can scan all the items on the rail/container, and CF will automatically build a Job out of it, pulling data from the Data Source around those specific products.

For more information on creating a Job from Scan, see our related article below.

When working with a Data Source, it is possible to run into some issues. For those, we have a Troubleshooting article which covers the most frequently encountered issues and process questions.
​
For more information on how to troubleshoot your Data Source, see our related article at the link below.
​