Introduction to feeds
This article deals with the overall role of feeds, in a successful integration between your webshop and Hello Retail. For an in-depth guide to the technical requirements and possibilities when creating feeds, we refer to developer.helloretail.com/guides/feeds/
What is a feed?
A feed is a term used to describe a common way of exchanging information over the Internet between a given system and other systems with an interest in this information. In a typical scenario, a webshop will write descriptive information about the individual products of the shop, to a file that is then uploaded to a server, from where it can be read by other people or computers. A well-known example of this would be the Google Shopping Feed, which Hello Retail is also able to utilize.
Formats
Hello Retail can work with feeds in either JSON, .xml or .csv
What should be in the feed?
Broadly speaking, the feed should contain all the information that Hello Retail need to know about, for the scope of the specific setup.
Access
The feeds must be accessible through a url - either by allowing global acces for all, by granting access to specific ips or by enabling Basic Auth access. Get in touch with our support team for the last two options.
How we request a feed
After setting up a feed in our system there are two ways our system can request your feed:
1) You can manually log in and initiate a feed run.
2) You can configure the feed to be run automatically on a schedule.
Updating
Typically Hello Retail will try to check for updates to the product data, with an interval of anything between 1 and 24 hours. It is the responsibility of the webshop, to keep the information updated in the feed, in accordance with how often you would like us to check for updates to the product data.
How we process a feed
Generally, when a feed is running it goes through following stages:
1) First the feed is in a queue waiting to be picked up for run.
2) Next the feed is being downloaded
3) Lastly the feed is being parsed into potential products, in the Hello Retail database
Depending on the size of your feed, the different phases can take longer or shorter periods of time.
Synchronization: create, update, delete
The feed system enables synchronization of data between an external system and ours with high accuracy. There are three types of synchronization that our system is capable of handling:
1) Creation of new data: When new data is received from the feed, our system will, if permitted, create it.
2) Updating existing data: Our system can detect changes in the data received from the feed and update the corresponding parts in our existing data.
3) Deletion of data: If data that was previously received is no longer present in the feed, our system will, if permitted, mark it for deletion.
By integrating these three core functionalities into our feed system, we have developed a robust platform that can effectively integrate your data into our system. Our system provides the flexibility to configure each feed to allow or disallow creation or deletion of data.
Therefore, all feeds can per definition update existing data, while some feeds can create or delete data, or a combination of them.
Naming a feed
Assigning a name to your feeds can make them more easily identifiable than their URLs. This can help you and your colleagues better understand the context of each feed.
To assign it a name, first navigate to the Feeds tool in the menu bar. Here, you will see an overview of your feeds, including the different feed types we support, such as Product feeds, Order feeds, Content feeds, and Permission feeds.
Next, find the table for the desired feed type and click on the name of the feed you want to assign a new name. This will take you to the feed edit view.
[Picture of clicking feed name in table row in view Feeds Overview]
In the top of the view, you will see several tabs. Click on the "Settings" tab to access the settings edit view.
Here, you can enter a name for your feed in the "Name" input field.
[Picture of settings options in view Product Feed Edit]
Once you are satisfied with the name, click the "Save" button at the bottom of the page to apply the changes.
[Picture of Save button in view Product Feed Edit]
Allow a feed only to update data
Per definition all feeds can update data. Then, if configured, some feeds can create data and/or delete data.
To see how a particular feeds permissions are configured, first navigate to the Feeds tool in the menu bar.
Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.
Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.
[Picture of clicking feed name in table row in view Feeds Overview]
In the top of the view, you will see several tabs.
Click on the "Settings" tab to access the settings edit view.
[Picture of tab "Settings" in view Product Feed Edit]
You will see a view of general settings for this particular feed. Among others there are toggles to allow the feed to create and delete.
A feed that only updates data will have create and delete permissions toggled off as such:
[Picture of a Product Feed that only updates data in view Product Feed Edit]
If your feed does not look like above picture, where create and delete permissions are toggled off, then switch them off, and click "Save" button at the bottom of the page to apply the changes.
[Picture of Save button in view Product Feed Edit]
Allowing a feed to create data
To alter the permission of a feed and allow it to create data (be it products, order, content or permissions), first navigate to the Feeds tool in the menu bar.
Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.
Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.
[Picture of clicking feed name in table row in view Feeds Overview]
In the top of the view, you will see several tabs.
Click on the "Settings" tab to access the settings edit view.
[Picture of tab "Settings" in view Product Feed Edit]
You will see a view of general settings for this particular feed. Among others there are toggles to allow the feed to create and delete.
Remember: all feeds have per definition the permission to update existing data, so there will not be a toggle for this kind of permission.
[Picture of tab "Settings" in view Product Feed Edit]
You can toggle the permissions on or off depending on your desired behavior of the feed.
[Picture of create permissions enabled for a Product Feed in view Product Feed Edit]
Once you are satisfied with the permission settings, click the "Save" button at the bottom of the page to apply the changes.
[Picture of Save button in view Product Feed Edit]
Allowing a feed to delete data
Please read section about "Synchronization: create, update, delete" to better understand what behavior feed permissions apply.
To alter the permission of a feed and allow it to delete data (be it products, order, content or permissions), first navigate to the Feeds tool in the menu bar.
Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.
Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.
[Picture of clicking feed name in table row in view Feeds Overview]
In the top of the view, you will see several tabs.
Click on the "Settings" tab to access the settings edit view.
[Picture of tab "Settings" in view Product Feed Edit]
You will see a view of general settings for this particular feed. Among others there are toggles to allow the feed to create and delete.
Remember: all feeds have per definition the permission to update existing data, so there will not be a toggle for this kind of permission.
[Picture of tab "Settings" in view Product Feed Edit]
You can toggle the permissions on or off depending on your desired behavior of the feed.
[Picture of create permissions enabled for a Product Feed in view Product Feed Edit]
Once you are satisfied with the permission settings, click the "Save" button at the bottom of the page to apply the changes.
[Picture of Save button in view Product Feed Edit]
Automatic schedule a feed
To set up automatic scheduling for a feed, first navigate to the Feeds tool in the menu bar. Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.
Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.
[Picture of clicking feed name in table row in view Feeds Overview]
In the top of the view, you will see several tabs.
Click on the "Settings" tab to access the settings edit view.
[Picture of tab "Settings" in view Product Feed Edit]
Here, you can toggle "on" the "Active" switch to turn on scheduling for the feed.
[Picture of settings options in view Product Feed Edit]
You can then select additional scheduling interval options.
[Picture of an active schedule in view Product Feed Edit]
Once you are satisfied with the scheduling options, click the "Save" button at the bottom of the page to apply the changes.
[Picture of Save button in view Product Feed Edit]
Creating a new feed
To create a feed, access the Feeds tool in the menu bar. Once there, you will be presented with an overview of your feeds across the different types we support: Product feeds, Order feeds, Content feeds, and Permission feeds.
To initiate the process of creating a new feed, locate the feed table of your desired type and click the "Add new" button. This will take you to the next step, where you will need to select the appropriate feed connector type.
[Picture of the "Add New" green button on Products feed table in view Feed Overview]
The concrete creation procedure then varies from feed type (Products, Orders, Contents, Permissions) and from feed connector type.
Some of the feed types and connector types will be documented here in combinations, so you can look for one that fits you or get inspiration from one of them in here. You can also go and venture into create a new feed and let the creation process guide you.
Archiving a feed
To archive a feed, go to the Feeds tool in the menu bar where you can see an overview of your feeds categorized by type: Product feeds, Order feeds, Content feeds, Permission feeds.
Locate the table of the feed you want to archive and click on the "..." button on the same row.
[Picture of the "..." feed table row button in view Feed Overview]
This will open a menu where you can select the "Archive" sub-menu button.
[Picture of the sub menu of the "..." feed table row button in view Feed Overview]
When you archive a feed, it will be removed from the feed table and any automatic scheduling of feed runs will stop.
Feed runs that are currently processing will continue until completion and will not be interrupted.
However, any queued feed runs that haven't been picked up for processing will be cancelled once the system detects that the feed has been archived.
Archiving a feed doesn't permanently delete it, so you can always restore it.
To restore an archived feed, please refer to the documentation on "Restoring an archived feed".
Restoring an archived feed
To restore an archived feed, first navigate to the Feeds tool in the menu bar where you can view an overview of the different feed types that are supported: Product feeds, Order feeds, Content feeds, Permission feeds.
Next, click on the "Archive" tab located at the top of the view, which is next to the currently selected "Overview" tab.
This will show you a similar view of tables, but with all archived feeds.
[Picture of the "Overview" and "Archive" tab in view Feeds Overview]
Find the table of your desired feed type and locate the feed you want to restore.
Then, click the right button "..." on the same row and a menu bar will appear.
[Picture of the "..." sub menu in view Archive Overview]
From here, select the "Unarchive" option and the feed will disappear from the current view.
To find the restored feed, return to the top and click on the "Overview" tab.
The default view will show again and you can now locate your restored feed.
Please note: If the restored feed previously had an active schedule, the schedule will now be on pause.
To restart the schedule, toggle the switch located on the left of the feed in the table. (This is a safeguard to prevent the restored feed from automatically restarting its schedule and possibly making modifications to your data in our system).
[Picture of the toggle in table row in view Feeds Overview]
Manually run a feed
To manually run a feed, first navigate to the Feeds tool in the menu bar where you can view an overview of the different feed types that are supported: Product feeds, Order feeds, Content feeds, Permission feeds.
Once you have found the table of your desired feed type:
Click on the green button "Put on queue" on the same row of the feed you want to run.
[Picture of the green "Put on queue" button in table row in view Feeds Overview]
The row will then update, and the button will change its color and text to "Awaiting queue".
[Picture of the grey "Awaiting queue" button in table row in view Feeds Overview]
After a short period of time, the text will change to "Downloading" when the feed queue ticket is picked up for processing.
[Picture of the grey "Downloading" button in table row in view Feeds Overview]
Once the downloading is complete, the text will change to "Parsing".
[Picture of the grey "Parsing" button in table row in view Feeds Overview]
Finally, when parsing is done, the button's color will revert, and the text will display "Put on queue" again.
Tip: If the button is either "Awaiting queue", "Downloading", or "Parsing", you can re-click the button to access the feed run view, where you can track its progress.
Interrupting a running feed
If one of your feeds is running and you need to stop it, you have the ability to interrupt the processing. This can be helpful when you want it to stop quickly, such as when you need to re-queue it due to a configuration change.
When you interrupt a feed, the system will try to stop the processing as quickly as possible. The system has built-in interruption checkpoints that will make the processing stop at controlled places.
Please note: Depending on how far the system has processed the feed, the processing may complete fully, but this is usually not the case.
If you need to interrupt a running feed, start by navigating to the Feeds tool in the menu bar.
From there, you'll see an overview of your feeds, organized by type, including Product feeds, Order feeds, Content feeds, and Permission feeds.
Look for the table of the feed type that you need to interrupt, and locate the feed that is currently running.
You can tell if a feed is running if there is a gray button to the right of the feed name in the table that says "Awaiting queue," "Downloading," or "Parsing".
To interrupt the running feed, click on the gray button with either "Awaiting queue," "Downloading," or "Parsing".
[Picture of the grey "Awaiting queue" button in table row in view Feeds Overview]
[Picture of the grey "Downloading" button in table row in view Feeds Overview]
[Picture of the grey "Parsing" button in table row in view Feeds Overview]
You will be taken to the Feed Run Details view.
[Picture of an active feed run in view Feed Run Details]
From there, you can use the interface to interrupt the feed run.
[Picture of "Interrupt" and "Reparse" button of an active feed run in view Feed Run Details]
Note: If the button is green and says "Put on queue," then the feed is not currently running, and there is no feed run to interrupt.
Feed History
[Picture of view Feed History]
The Feed History (which sometimes can be referred to as the "feed run history" in daily talk) provides valuable insight into the performance of your feed over a period of time.
By reviewing this overview, you can quickly determine if the feed has failed and how many feed items were created, updated, or deleted.
Additionally, the Feed History includes several useful actions that can be performed, including:
- Reparsing a previous feed run: This action allows you to parse a previous feed run again, which can be useful if you need to update information that was previously missed or if there was an issue with the original parse.
- Interrupting a currently running feed run: This action enables you to stop a feed run that is currently in progress. This can be helpful if you need to halt the feed for any reason, such as to make updates or to address any issues that may have arisen during the run.
By clicking on one of the rows in the table in Feed History view you will be taken to the Feed Run Details view of that particular feed run. In that view you can see further details about that run.
You can read more about the Feed Run Details view under section "Feed Run Details".
Understanding the information
The stats of the feed run
While at the Feed History view, can see a table with all the current and previous runs of a particular feed.
Each row in the table resembles a feed run. Each row displays the number of processed items, including the number of created, modified, and deleted items during that particular run.
[Picture of the stats of a particular row in Feed History]
Time: The time the feed queue ticket was created.
Status: The processing status of the feed queue ticket. The value can be either of:
- Interrupted
- Interrupting..
- Cool down
- Awaiting queue
- Downloading
- Parsing
- Failed
- Finished
Total: The total number of items found in this feed run
Added: The number of items that was created in our system from this feed run
Updated: The number of items that was updated in our system from this feed run
Deleted: The number of items that was deleted in our system from this feed run
Reparsing a previous feed from Feed History
In the Feed History, you can view a list of scheduled and previous feed runs.
If a previous feed run successfully downloaded the feed, you can use the "Reparse" button to the right of the table row.
[Picture of "Reparse" button in table row in view Feed History]
Clicking the button initiates a new feed run, but only reparsing the result of that feed with the latest configuration.
This means that the feed will not be downloaded again, but the data from that previous run will be reused.
Reparsing can be a time-saving option when testing or fixing your feed and you don't want to wait for the feed to download again.
Interrupting a feed run from Feed History
In the Feed History, you can view a list of scheduled and previous feed runs.
[Picture of "Interrupt" button in table row in view Feed History]
If a run is scheduled or already being processed you can use the button "Interrupt" to send an interrupt signal to that particular run.
The system will try its best to stop the feed run as fast as possible.
[Picture of "Interrupt" button clicked and table row shows "Interrupting.." in view Feed History]
Feed Run Details
Understanding the information of a feed run
The stats of the feed run
While viewing the Feed Run Details, you can access the same table data as seen in the Feed History, displaying the number of processed items, including the number of created, modified, and deleted items during that particular run.
[Picture of the stats of a particular Feed Run Details]
Time: The time the feed queue ticket was created.
Status: The processing status of the feed queue ticket. The value can be either of:
- Interrupted
- Interrupting..
- Cool down
- Awaiting queue
- Downloading
- Parsing
- Failed
- Finished
Total: The total number of items found in this feed run
Added: The number of items that was created in our system from this feed run
Updated: The number of items that was updated in our system from this feed run
Deleted: The number of items that was deleted in our system from this feed run
The ID of the feed run
Each feed run has an ID, it is sometimes easier to direct support to a particular run by letting them know the ID. You can find the ID in the title next to "Run " text.
You can also copy and send the URL of your browser, and other users (if they have access) or support can get directly to this run.
[Picture of the ID of a particular Feed Run Details]
Unlike the Feed History, you can also view the log containing detailed information about the feed processing.
[Picture of the log of a particular Feed Run Details]
If the feed run is currently queued or being processed by the system, the view will automatically update to reflect the progress.
The log may provide useful information about the current status of the processing or if any issues occurred during the run.