Feeds Guide

What is a feed?

A "feed" is a term used to describe the data that can be taken from an online shop and accessed over the internet.  
This data could be in the form of XML or JSON files that contain information about the shop's products, or an API that can retrieve the product data in either XML or JSON format.  A well-known example of a feed is the Google Shopping Feed.

In our system, a "feed" belongs to a specific website and is configured with connector type of your choice.

A feed in our system is therefore a configuration containing information about scheduling, data permissions, and their relationship to a website. Additionally, the feed is equipped with a specific type of connector, which determines how it requests the feed and how it processes the resulting data, so we can ingest multiple types of feeds. Some feeds are XML data, others have a unique integration to the web shop platform, while others may paginate data in a unique way. We've built a system that can accommodate these differences.

What is a connector?

A feed connector serves as both an indicator and a configuration source for the feed system, instructing it on how to establish a connection with the desired feed. These connectors can either be specific to web shops or more generic capable of just retrieving  a certain type of data. Furthermore, connectors can be customized using HTTP basic authentication, custom HTTP headers, or URL parameters.

As an example, we have already established connectors for both Shopify and Magento 2. We are continuously expanding our library of connectors to accommodate our clients' needs and to include as many setups as possible.

How we request a feed

After setting up a feed in our system there are two ways our system can request your feed:  
1) You can manually log in and initiate a feed run.  
2) You can configure the feed to be run automatically on a schedule.

How we process a feed

Generally, when a feed is running it goes through following stages:  
1) First the feed is in a queue waiting to be picked up for run.  
2) Next the feed is being downloaded  
3) Lastly the feed is being parsed into potential products

Depending on the size of your feed, the different phases can take longer or shorter periods of time due.

On a more practical level the rules of the feed queue also applies to how, when, and for how long the feed process the feed.

For more information, please read "How the feed queue work" and "Requirements for a feed" a few sections below.

Synchronization: create, update, delete

The feed system enables synchronization of data between an external system and ours with high accuracy. There are three types of synchronization that our system is capable of handling:

1) Creation of new data: When new data is received from the feed, our system will, if permitted, create it.

2) Updating existing data: Our system can detect changes in the data received from the feed and update the corresponding parts in our existing data.

3) Deletion of data: If data that was previously received is no longer present in the feed, our system will, if permitted, mark it for deletion.

By integrating these three core functionalities into our feed system, we have developed a robust platform that can effectively integrate your data into our system. Our system provides the flexibility to configure each feed to allow or disallow creation or deletion of data.
Therefore, all feeds can per definition update existing data, while some feeds can create or delete data, or a combination of them.

You can read about how to allow/disallow (in other words alter the a feeds permission to handle data) in the sections "Allow a feed to create" and "Allow a feed to delete".

How the feed queue work

When a "feed" needs to be executed, it is assigned a new ticket on a system that tracks all the active feeds. A considerable number of queue workers are available to pick up the ticket and initiate the processing of the corresponding feed. This approach enables us to regulate the number of feeds being processed at any given time, ensuring that your web shop is not overloaded. The queue ticket also stores important information about the status and other metadata related to monitoring the feed's progress or its final outcome.

There are a few rules that a feed must follow to be queued:

1) Feed queue ticket limit: Each feed is allowed only one active/processing feed queue ticket and one pending ticket at a time. If a feed is automatically scheduled or manually queued and there is already one pending ticket, the system will continue with only one pending ticket.

2) Feed cool down: If a feed is already running or has recently been processed, the system may decide to activate a "cool down" period for the next pending queue ticket.
This effectively parks the ticket for a brief time to prevent potential bottlenecks from high-frequency feeds. This safeguard ensures stability for both your server and our system, promoting reliability across the feeds service for all clients.

Requirements for a feed

In order to establish your feed in our system, it must satisfy a set of basic prerequisites:

1) The feed must be accessible over the internet. Hence, it is essential to have a URL that points to the feed.

2) The feed must return either a JSON or JSONL data. To some degree we support XML, please read section "The XML format".

3) Each item (a product, an order, a content or a permission) in the feed cannot exceed the size of 256kb.

4) As a safe guard the total time it takes for a feed to be processed (downloading + processing time) cannot take longer than 6h. However, usually, feeds never go above this time limit and the part of the process that takes the longest is the downloading of the actual feed.

5) Other requirements may apply to the individual type of feed and/or connector.

The JSONL format

JSONL stands for "JSON Lines" or "JSONLines". It is a format for storing structured data that consists of a sequence of JSON objects, one per line. Each line in the file is a valid JSON object, but there is no wrapping JSON array like in a regular JSON file.

JSONL is often used for storing or transmitting large amounts of data that is too big to fit into a single JSON document. Instead of storing all of the data in a single file, JSONL allows data to be stored as a stream of separate JSON objects, which can be read and processed one at a time.

JSONL files are often used in applications that generate or consume large amounts of data, such as log files, machine learning datasets, or data pipelines. The format is simple and easy to parse, making it a popular choice for data exchange between systems.

This is our now our most preferred format of consuming external data. By leveraging the compact data for high volume data exchanges and HTTP response headers we can support similar features as JSON and XML format in a more performant way.

A partial example of a potential JSONL product feed:

{"title":"Product 1","url":"https://example.com/products/1","price":99.95}
{"title":"Product 2","url":"https://example.com/products/2","price":199}
{"title":"Product 3","url":"https://example.com/products/3","price":399}
{"title":"Product 4","url":"https://example.com/products/4","price":410}

As you can see JSONL is compact exchange format, with one JSON object per line.
You can read more about JSONL on the official website https://jsonlines.org.

The JSON format

JSON is a lightweight and flexible data interchange format that is widely used for transmitting data between applications. It uses a text-based format that is easy for humans to read and write, and for machines to parse and generate. JSON has become a popular alternative to other data interchange formats, such as XML, due to its simplicity and flexibility.

A partial example of a potential JSON product feed:

{
    "products": [
        {"title":"Product 1","url":"https://example.com/products/1","price":99.95}
        {"title":"Product 2","url":"https://example.com/products/2","price":199}
        {"title":"Product 3","url":"https://example.com/products/3","price":399}
        {"title":"Product 4","url":"https://example.com/products/4","price":410}
    ],
    "latePageNumber":47,
    "extensionVersion":"1.1.26",
    "deltaToken":"1682681545"
}

As you can see the JSON example can also be compact but requires more structure than JSONL and therefore the systems exporting and consuming JSON can require a bit more configuration from either parts for a successful data exchange.
You can read more about JSON on the official website https://www.json.org.

The XML format

In the past, our feed system primarily ingested XML feeds. However, the industry has trended towards JSON for quite some time, and this has caused a shift in how APIs and systems consume data.
While we still support XML to a certain extent, all data downloaded from our system is converted to JSON format. This conversion process may result in some data inconsistencies, such as uncertainties in data type or the application of artifacts during the conversion process.

Therefore, we generally consider the XML format for sending data to our system as deprecated. We strongly advise all new feed development for shop platform plugins to utilize JSONL or JSON format for sending data.

A partial example of a potential XML product feed:

<?xml version="1.0"?>
<response>
  <item>
    <products>
      <item>
        <product>
          <title>Product 1</title>
          <url>https://example.com/products/1</url>
          <price>20</price>
        </product>
      </item>
      <item>
        <product>
          <title>Product 2</title>
          <url>https://example.com/products/2</url>
          <price>99</price>
        </product>
      </item>
      <item>
        <product>
          <title>Product 3</title>
          <url>https://example.com/products/3</url>
          <price>125</price>
        </product>
      </item>
      <item>
        <product>
          <title>Product 4</title>
          <url>https://example.com/products/4</url>
          <price>99</price>
        </product>
      </item>
    </products>
    <last_page_number>47</last_page_number>
    <extension_version>1.1.26</extension_version>
    <deltaToken>1682681587</deltaToken>
  </item>
</response>

As you can see the XML example is a bit more verbose and require the same amount of structure as JSON and therefore the systems exporting and consuming JSON can require a bit more configuration for a successful data exchange.

You can read more about XML on the official website https://www.w3.org/XML.

Best practices

JSON, JSONL or XML feeds

Our feed system primarily supports JSON and JSONL formats.

However, we also support XML due to the prevalence of shop platforms and plugins that historically exported in XML. When an XML feed is passed into our system, it is automatically converted to JSON.

It's important to note that the conversion process from XML to JSON may result in unclear data artifacts or incorrect data inference for some field values. For instance, it may not be entirely clear for the converter if a value should be interpreted as text or a number. Therefore, we strongly recommend producing new feeds in JSON or JSONL formats to avoid potential data inconsistencies.

Delta feeds

An ideal approach for data ingestion is to utilize a feed that supports delta data.
Delta data, also referred to as "delta feeds," enables our system to receive the latest product modifications frequently without burdening your web shop server. Our system will keep track of the most recent "delta token" retrieved from the feed and utilize it for the next delta request, thereby, again, obtaining the changes that have occurred since the last delta request.

This process significantly reduces the load on your server, as it no longer needs to send us the entire product catalogue every time. As a result, we can schedule the feed more frequently, ensuring that your data is always up-to-date.

Please note: Once every 24 hours a delta feed will be run fully, obtaining the full catalogue and not only the delta changes. This ensures a complete synchronization between your shop and our system.

Pagination

Feeds that can be paginated provide an additional benefit by considerably reducing the memory allocation required by both our system and yours to process the feed.
The feed requests are divided into smaller chunks of data fetches, allowing for more efficient processing.
Ultimately, our system handles the feed as a single, large feed, but practically split into multiple requests.

Safeguards regarding pagination

TODO:: We have a couple of safeguards regarding

Scheduling

A feed can be initiated manually, or it can be configured with a schedule, allowing the feed system to automatically run the feed at set intervals. The schedule interval denotes how often the feed is run, and subsequently, the freshness of your data. The lower the interval, the more frequently the feed is queued for processing.

The size of the feed can impact your web shop's performance, so some clients prefer running the feed once a day, while others have the resources to run it more frequently.
The optimal frequency depends on the frequency of product modifications and the desired freshness of your data in our system.

Incorporating multiple product feeds can also be useful. You can use a lightweight feed that runs more frequently and a heavier feed that runs once a day or week.

We see, that in most cases our clients can run the feed twice a day, around 12 PM and midnight, without experiencing any issues.

When a feed fails

TODO: single and pagination
Types of fails, system fails, response fails

---

Feeds tool

We have developed a tool within our application that enables you to create, configure, and monitor your feeds. The tool features a sophisticated user interface that assists and guides you through the process of making changes, and provides you with a clear indication of whether or not your data ingestion is successful.

The user interface allows you to manually trigger your feeds, or schedule them and track their progress.

To access the Feeds tool, simply navigate to "Configuration" > "Data Setup" > "Feeds" in the application menu bar.

[Picture of the Feeds tool in the menu bar in my.helloretail.com]

Feeds Overview

[Picture of the the view Feed Overview]

Notice about "Managed feeds":

Please note that some of the feeds that were set up by Hello Retail during your on-boarding prior to the current Feeds tool being published are considered legacy feeds from the previous system. These feeds are managed differently and can only be edited by Hello Retail employees, much like some of you have previously known. However, you can still use functionality such as manually putting them on queue, archiving and restoring the feed, following the process, and inspecting the feed run history.

We are currently exploring ways to convert these legacy feeds to be fully available in the current and new Feeds tool so that you can have all the functionality for them.
[Picture of a managed feed]

Edit: Naming a feed

Assigning a name to your feeds can make them more easily identifiable than their URLs. This can help you and your colleagues better understand the context of each feed.

To assign it a name, first navigate to the Feeds tool in the menu bar. Here, you will see an overview of your feeds, including the different feed types we support, such as Product feeds, Order feeds, Content feeds, and Permission feeds.

Next, find the table for the desired feed type and click on the name of the feed you want to assign a new name. This will take you to the feed edit view.

[Picture of clicking feed name in table row in view Feeds Overview]

In the top of the view, you will see several tabs. Click on the "Settings" tab to access the settings edit view.

Here, you can enter a name for your feed in the "Name" input field.

[Picture of settings options in view Product Feed Edit]

Once you are satisfied with the name, click the "Save" button at the bottom of the page to apply the changes.

[Picture of Save button in view Product Feed Edit]

Edit: Allow a feed only to update data

Please read section about "Synchronization: create, update, delete" to better understand what behavior feed permissions apply.

Per definition all feeds can update data. Then, if configured, some feeds can create data and/or delete data.

To see how a particular feeds permissions are configured, first navigate to the Feeds tool in the menu bar.
Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.

Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.

[Picture of clicking feed name in table row in view Feeds Overview]

In the top of the view, you will see several tabs.

Click on the "Settings" tab to access the settings edit view.

[Picture of tab "Settings" in view Product Feed Edit]

You will see a view of general settings for this particular feed. Among others there are toggles to allow the feed to create and delete.

A feed that only updates data will have create and delete permissions toggled off as such:

  
[Picture of a Product Feed that only updates data in view Product Feed Edit]

If your feed does not look like above picture, where create and delete permissions are toggled off, then switch them off, and click "Save" button at the bottom of the page to apply the changes.

[Picture of Save button in view Product Feed Edit]

Edit: Allowing a feed to create data

Please read section about "Synchronization: create, update, delete" to better understand what behavior feed permissions apply.

To alter the permission of a feed and allow it to create data (be it products, order, content or permissions), first navigate to the Feeds tool in the menu bar.
Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.

Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.

[Picture of clicking feed name in table row in view Feeds Overview]

In the top of the view, you will see several tabs.

Click on the "Settings" tab to access the settings edit view.

[Picture of tab "Settings" in view Product Feed Edit]

You will see a view of general settings for this particular feed. Among others there are toggles to allow the feed to create and delete.
Remember: all feeds have per definition the permission to update existing data, so there will not be a toggle for this kind of permission.

[Picture of tab "Settings" in view Product Feed Edit]

You can toggle the permissions on or off depending on your desired behavior of the feed.

 
[Picture of create permissions enabled for a Product Feed in view Product Feed Edit]

Once you are satisfied with the permission settings, click the "Save" button at the bottom of the page to apply the changes.

[Picture of Save button in view Product Feed Edit]

Edit: Allowing a feed to delete data

Please read section about "Synchronization: create, update, delete" to better understand what behavior feed permissions apply.

To alter the permission of a feed and allow it to delete data (be it products, order, content or permissions), first navigate to the Feeds tool in the menu bar.
Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.

Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.

[Picture of clicking feed name in table row in view Feeds Overview]

In the top of the view, you will see several tabs.

Click on the "Settings" tab to access the settings edit view.

[Picture of tab "Settings" in view Product Feed Edit]

You will see a view of general settings for this particular feed. Among others there are toggles to allow the feed to create and delete.
Remember: all feeds have per definition the permission to update existing data, so there will not be a toggle for this kind of permission.

[Picture of tab "Settings" in view Product Feed Edit]

You can toggle the permissions on or off depending on your desired behavior of the feed.

  
[Picture of create permissions enabled for a Product Feed in view Product Feed Edit]

Once you are satisfied with the permission settings, click the "Save" button at the bottom of the page to apply the changes.

[Picture of Save button in view Product Feed Edit]

Edit: Automatic schedule a feed

To set up automatic scheduling for a feed, first navigate to the Feeds tool in the menu bar. Here, you will see an overview of your feeds, including the different feed types we support such as Product feeds, Order feeds, Content feeds, and Permission feeds.

Next, find the table for the desired feed type and click on the name of the feed you want to schedule. This will take you to the feed edit view.

[Picture of clicking feed name in table row in view Feeds Overview]

In the top of the view, you will see several tabs.

Click on the "Settings" tab to access the settings edit view.

[Picture of tab "Settings" in view Product Feed Edit]

Here, you can toggle "on" the "Active" switch to turn on scheduling for the feed.

[Picture of settings options in view Product Feed Edit]

You can then select additional scheduling interval options.

[Picture of an active schedule in view Product Feed Edit] 

Once you are satisfied with the scheduling options, click the "Save" button at the bottom of the page to apply the changes.

[Picture of Save button in view Product Feed Edit]

Action: Creating a feed

To create a feed, access the Feeds tool in the menu bar. Once there, you will be presented with an overview of your feeds across the different types we support: Product feeds, Order feeds, Content feeds, and Permission feeds.

To initiate the process of creating a new feed, locate the feed table of your desired type and click the "Add new" button. This will take you to the next step, where you will need to select the appropriate feed connector type.

[Picture of the "Add New" green button on Products feed table in view Feed Overview]

The concrete creation procedure then varies from feed type (Products, Orders, Contents, Permissions) and from feed connector type.
Some of the feed types and connector types will be documented here in combinations, so you can look for one that fits you or get inspiration from one of them in here.  You can also go and venture into create a new feed and let the creation process guide you.

Action: Archiving a feed

To archive a feed, go to the Feeds tool in the menu bar where you can see an overview of your feeds categorized by type: Product feeds, Order feeds, Content feeds, Permission feeds.

Locate the table of the feed you want to archive and click on the "..." button on the same row.

[Picture of the "..." feed table row button in view Feed Overview]

 

This will open a menu where you can select the "Archive" sub-menu button.

[Picture of the sub menu of the "..." feed table row button in view Feed Overview]

When you archive a feed, it will be removed from the feed table and any automatic scheduling of feed runs will stop.

Feed runs that are currently processing will continue until completion and will not be interrupted.
However, any queued feed runs that haven't been picked up for processing will be cancelled once the system detects that the feed has been archived.

Archiving a feed doesn't permanently delete it, so you can always restore it.
To restore an archived feed, please refer to the documentation on "Restoring an archived feed".

Action: Restoring an archived feed

To restore an archived feed, first navigate to the Feeds tool in the menu bar where you can view an overview of the different feed types that are supported: Product feeds, Order feeds, Content feeds, Permission feeds.

Next, click on the "Archive" tab located at the top of the view, which is next to the currently selected "Overview" tab.
This will show you a similar view of tables, but with all archived feeds.

[Picture of the "Overview" and "Archive" tab in view Feeds Overview]

Find the table of your desired feed type and locate the feed you want to restore.

Then, click the right button "..." on the same row and a menu bar will appear.
 
[Picture of the "..." sub menu in view Archive Overview]

From here, select the "Unarchive" option and the feed will disappear from the current view.

To find the restored feed, return to the top and click on the "Overview" tab.
The default view will show again and you can now locate your restored feed.

Please note: If the restored feed previously had an active schedule, the schedule will now be on pause.
To restart the schedule, toggle the switch located on the left of the feed in the table. (This is a safeguard to prevent the restored feed from automatically restarting its schedule and possibly making modifications to your data in our system).
 
[Picture of the toggle in table row in view Feeds Overview]

Action: Manually run a feed

To manually run a feed, first navigate to the Feeds tool in the menu bar where you can view an overview of the different feed types that are supported: Product feeds, Order feeds, Content feeds, Permission feeds.

Once you have found the table of your desired feed type:

Click on the green button "Put on queue" on the same row of the feed you want to run.

 
[Picture of the green "Put on queue" button in table row in view Feeds Overview]

The row will then update, and the button will change its color and text to "Awaiting queue".

[Picture of the grey "Awaiting queue" button in table row in view Feeds Overview]

After a short period of time, the text will change to "Downloading" when the feed queue ticket is picked up for processing.

[Picture of the grey "Downloading" button in table row in view Feeds Overview]

Once the downloading is complete, the text will change to "Parsing".

[Picture of the grey "Parsing" button in table row in view Feeds Overview]

Finally, when parsing is done, the button's color will revert, and the text will display "Put on queue" again.

Tip: If the button is either "Awaiting queue", "Downloading", or "Parsing", you can re-click the button to access the feed run view, where you can track its progress.

Action: Interrupting a running feed

If one of your feeds is running and you need to stop it, you have the ability to interrupt the processing. This can be helpful when you want it to stop quickly, such as when you need to re-queue it due to a configuration change.

When you interrupt a feed, the system will try to stop the processing as quickly as possible. The system has built-in interruption checkpoints that will make the processing stop at controlled places.
Please note: Depending on how far the system has processed the feed, the processing may complete fully, but this is usually not the case.

If you need to interrupt a running feed, start by navigating to the Feeds tool in the menu bar.

From there, you'll see an overview of your feeds, organized by type, including Product feeds, Order feeds, Content feeds, and Permission feeds.

Look for the table of the feed type that you need to interrupt, and locate the feed that is currently running.

You can tell if a feed is running if there is a gray button to the right of the feed name in the table that says "Awaiting queue," "Downloading," or "Parsing".

To interrupt the running feed, click on the gray button with either "Awaiting queue," "Downloading," or "Parsing".

[Picture of the grey "Awaiting queue" button in table row in view Feeds Overview]

[Picture of the grey "Downloading" button in table row in view Feeds Overview]

[Picture of the grey "Parsing" button in table row in view Feeds Overview]

You will be taken to the Feed Run Details view.

[Picture of an active feed run in view Feed Run Details]

From there, you can use the interface to interrupt the feed run.

 

[Picture of "Interrupt" and "Reparse" button of an active feed run in view Feed Run Details]

For more information on the Feed Run Details view, please read section "Feed Run Details" section.

Note: If the button is green and says "Put on queue," then the feed is not currently running, and there is no feed run to interrupt.

Feed History

[Picture of view Feed History]

The Feed History (which sometimes can be referred to as the "feed run history" in daily talk) provides valuable insight into the performance of your feed over a period of time.

By reviewing this overview, you can quickly determine if the feed has failed and how many feed items were created, updated, or deleted.

Additionally, the Feed History includes several useful actions that can be performed, including:

• Reparsing a previous feed run: This action allows you to parse a previous feed run again, which can be useful if you need to update information that was previously missed or if there was an issue with the original parse.
• Interrupting a currently running feed run: This action enables you to stop a feed run that is currently in progress. This can be helpful if you need to halt the feed for any reason, such as to make updates or to address any issues that may have arisen during the run.

By clicking on one of the rows in the table in Feed History view you will be taken to the Feed Run Details view of that particular feed run. In that view you can see further details about that run.

You can read more about the Feed Run Details view under section "Feed Run Details".

Understanding the information

The stats of the feed run

While at the Feed History view, can see a table with all the current and previous runs of a particular feed.

Each row in the table resembles a feed run. Each row displays the number of processed items, including the number of created, modified, and deleted items during that particular run.

[Picture of the stats of a particular row in Feed History]

Time: The time the feed queue ticket was created.

Status: The processing status of the feed queue ticket. The value can be either of:

• Interrupted
• Interrupting..
• Cool down
• Awaiting queue
• Downloading
• Parsing
• Failed
• Finished

Total: The total number of items found in this feed run

Added: The number of items that was created in our system from this feed run

Updated: The number of items that was updated in our system from this feed run

Deleted: The number of items that was deleted in our system from this feed run

Action: Reparsing a previous feed from Feed History

In the Feed History, you can view a list of scheduled and previous feed runs.

If a previous feed run successfully downloaded the feed, you can use the "Reparse" button to the right of the table row.

[Picture of "Reparse" button in table row in view Feed History] 

Clicking the button initiates a new feed run, but only reparsing the result of that feed with the latest configuration.

This means that the feed will not be downloaded again, but the data from that previous run will be reused.

Reparsing can be a time-saving option when testing or fixing your feed and you don't want to wait for the feed to download again.

Action: Interrupting a feed run from Feed History

Please read section about the behavior of interrupting a feed named "Interrupting a running feed".

In the Feed History, you can view a list of scheduled and previous feed runs.

[Picture of "Interrupt" button in table row in view Feed History]  

If a run is scheduled or already being processed you can use the button "Interrupt" to send an interrupt signal to that particular run.
The system will try its best to stop the feed run as fast as possible.

[Picture of "Interrupt" button clicked and table row shows "Interrupting.." in view Feed History]    

Feed Run Details

Understanding the information of a feed run

The stats of the feed run

While viewing the Feed Run Details, you can access the same table data as seen in the Feed History, displaying the number of processed items, including the number of created, modified, and deleted items during that particular run.

[Picture of the stats of a particular Feed Run Details]

Time: The time the feed queue ticket was created.

Status: The processing status of the feed queue ticket. The value can be either of:

• Interrupted
• Interrupting..
• Cool down
• Awaiting queue
• Downloading
• Parsing
• Failed
• Finished

Total: The total number of items found in this feed run

Added: The number of items that was created in our system from this feed run

Updated: The number of items that was updated in our system from this feed run

Deleted: The number of items that was deleted in our system from this feed run

The ID of the feed run

Each feed run has an ID, it is sometimes easier to direct support to a particular run by letting them know the ID. You can find the ID in the title next to "Run " text.

You can also copy and send the URL of your browser, and other users (if they have access) or support can get directly to this run.

[Picture of the ID of a particular Feed Run Details]

Unlike the Feed History, you can also view the log containing detailed information about the feed processing.

[Picture of the log of a particular Feed Run Details] 

If the feed run is currently queued or being processed by the system, the view will automatically update to reflect the progress.

The log may provide useful information about the current status of the processing or if any issues occurred during the run.

Action: Reparsing a previous feed from Feed Run Details

Please read section about the behavior of reparsing a feed run named "Reparsing a previous feed from Feed History".

While being at a Feed Run Details view, you can see if this particular feed run is able to be used for a reparse.
If possible, then in the top of the view, the "Reparse" button will be available to initiate a parsing of that feed with the latest configuration.

[Picture of "Reparse" button in view Feed Run Details] 

Action: Interrupting a feed run from Feed Run Details

Please read section about the behavior of interrupting a feed named "Interrupting a running feed".

While being at a Feed Run Details view, you can see if this particular feed run can be interrupted.
If possible, then in the top of the view, the "Interrupt" button will be available which will send an interrupt signal to the feed run.

[Picture of "Interrupt" button in view Feed Run Details]

The feed system will then stop the processing as fast as possible.

[Picture of "Interrupt" button clicked and table row shows "Interrupting.." in view Feed Run Details]

Product Feeds

Hello Retail Product data object

Example JSON of a Hello Retail product data object:

{
    "productNumber": "1",
    "title": "Example title",
    "inStock": true,
    "url": "https://example.com/products/1",
    "imgUrl": "https://cdn.example.com/products/1/thumbnail.png",
    "description": "Description of the product",
    "price": 19.95,
    "ean": "e12345",
    "oldPrice": "22.95",
    "priceExVat": 15.96,
    "oldPriceExVat": 18.36,
    "currency": "DKK",
    "brand": "Examplish",
    "keywords": "example,showcase,feed,data",
    "created": 1682690820,
    "hierarchies": [
        ["fashion", "womens", "shirts"],
        ["fashion", "mens", "shirts"]
    ]
}

Reserved fields and case insensitive variants:

The field names shown in the example above should be treated as "reserved" fields. It is important to note that reserved fields are case-insensitive, meaning that the feed system will recognize variations of reserved field names with different capitalization as the same field. For instance, both "productNumber" and "PRODUCTNUMBER" will be treated as the same field by the feed system during processing.

Fields explained:

Field "productNumber":
A unique identifier value assigned by the web shop platform for the product.

Field "title":
The name or title of the product.

Field "inStock":
A boolean value indicating whether the product is currently in stock or not.

Field "url":
The URL of the product page on the webshop.

Field "imgUrl":
The URL of the product image.

Field "description":
A description of the product.

Field "price":
The price of the product, including any taxes or fees.

Field "ean":
The European Article Number (EAN) of the product.

Field "oldPrice":
The previous price of the product before any discounts or changes.

Field "priceExVat":
The price of the product excluding any taxes or fees.

Field "oldPriceExVat":
The previous price of the product excluding any taxes or fees.

Field "currency":
The currency in which the product price is listed.

Field "brand":
The brand of the product.

Field "keywords":
A list of keywords or tags associated with the product.

Field "created":
The epoch time when the product was created.

"hierarchies":
A list of hierarchies or categories that the product belongs to represented as a nested json array of strings.

Product extraData and extraDataList

 

Requirements for a JSON feed:

Our system will request the JSON feed with HTTP GET request.
The request will have header:

'Accept: application/json'

We expect the response to be a HTTP 200 OK with header indicating that content is JSON in utf-8 charset:

'Content-Type: application/json; charset=utf-8'

The minimum expected response body for a JSON product feed is this, indicating that there are no (or no more) products.

{ "products":[] }

Requirements for a JSON feed with pagination:

Our system will request the JSON feed with HTTP GET request.
The request will have header:

'Accept: application/json'

We expect the response to be a HTTP 200 OK with header indicating that content is JSON in utf-8 charset:

'Content-Type: application/json; charset=utf-8'

If the JSON feed should support pagination, the mininum expected response body is:

{ "products":[] }

Requirements for a JSON feed with delta:

Our system will request the JSON feed with HTTP GET request.
The request will have header:

'Accept: application/json'

We expect the response to be a HTTP 200 OK with headers:

'Content-Type: application/json; charset=utf-8'

If the JSON feed should support being a delta feed, the minimum expected response bodt is:

{ "products":[], "deltaToken":"12345678" }

Requirements for a JSONL feed:

Our system will request the JSONL feed with HTTP GET request.
The request will have header:

'Accept: application/jsonl'

We expect the response to be a HTTP 200 OK with header indicating that content is JSONL in utf-8 charset:

'Content-Type: application/jsonl; charset=utf-8'

{"id": 123, "url": "https://example.com/products/123", "price": 99.95}
{"id": 456, "url": "https://example.com/products/456", "price": 199}

Requirements for a JSONL feed with pagination:

Our system will request the JSONL feed with HTTP GET request.
The request will have header:

'Accept: application/jsonl'

We expect the response to be a HTTP 200 OK with headers:

'Content-Type: application/jsonl; charset=utf-8'
'X-Total: 99'

{"id": 123, "url": "https://example.com/products/123", "price": 99.95}
{"id": 456, "url": "https://example.com/products/456", "price": 199}

Requirements for a JSONL feed with delta:

Our system will request the JSONL feed with HTTP GET request.
The request will have header:

'Accept: application/jsonl'

We expect the response to be a HTTP 200 OK with headers:

'Content-Type: application/jsonl; charset=utf-8'
'X-Delta-Token: 12345678'

{"id": 123, "url": "https://example.com/products/123", "price": 99.95}
{"id": 456, "url": "https://example.com/products/456", "price": 199}

Order Feeds

As the new Feed system is still under development to encompass all the necessary features to enable the setup of a website with products, orders, content pages, and permissions, there are certain functionalities that are not yet fully supported.
Our primary focus is to first fully support Product Feeds, after that Order Feeds, Content Feeds and Permission Feeds.

Supported Order Feed functionality

•  Feeds Overview view:
  • Queuing a feed
  • Interrupting a feed
  • Archiving a feed
  • Restoring an archived feed
• Feed Run view:
  • All functionality supported
• Feed Run Details view:
  • All functionality supported

Above listed functionality is generic to all feed types and is therefore something you can already read about in this guide.

Functionality on road map:

• Creation of Order Feed

So while we continue to work towards completing the Feeds tool, there may be instances where you need to reach out to Hello Retail support for any desired changes.

Content Feeds

As the new Feed system is still under development to encompass all the necessary features to enable the setup of a website with products, orders, content pages, and permissions, there are certain functionalities that are not yet fully supported.
Our primary focus is to first fully support Product Feeds, after that Order Feeds, Content Feeds and Permission Feeds.

Supported Content Feed functionality

•  Feeds Overview view:
  • Queuing a feed
  • Interrupting a feed
  • Archiving a feed
  • Restoring an archived feed
• Feed Run view:
  • All functionality supported
• Feed Run Details view:
  • All functionality supported

Above listed functionality is generic to all feed types and is therefore something you can already read about in this guide.

Functionality on road map:

• Creation of Content Feed

So while we continue to work towards completing the Feeds tool, there may be instances where you need to reach out to Hello Retail support for any desired changes.

Permission Feeds

As the new Feed system is still under development to encompass all the necessary features to enable the setup of a website with products, orders, content pages, and permissions, there are certain functionalities that are not yet fully supported.
Our primary focus is to first fully support Product Feeds, after that Order Feeds, Content Feeds and Permission Feeds.

Supported Permission Feed functionality

•  Feeds Overview view:
  • Queuing a feed
  • Interrupting a feed
  • Archiving a feed
  • Restoring an archived feed
• Feed Run view:
  • All functionality supported
• Feed Run Details view:
  • All functionality supported

Above listed functionality is generic to all feed types and is therefore something you can already read about in this guide.

Functionality on road map:

• Creation of Permission Feed

So while we continue to work towards completing the Feeds tool, there may be instances where you need to reach out to Hello Retail support for any desired changes.