Store and Forward
The store-and-forward system provides a reliable way for Ignition to store data to the database. In Ignition, systems such as Tag Historian and Data Bridge (Transaction Groups) use store-and-forward to ensure that data reaches its destination in the database, and is stored in an efficient manner. The store-and-forward system can be configured in a number of ways, offering both memory buffering for performance and local disk caching for safe storage.
Note: Store-and-forward engines are automatically created for each Database Connection.
Primary Features and Benefits
The store-and-forward system offers a number of benefits over other systems that log directly to the database, such as:
-
Data loss prevention
Data is removed from the system only when the write to the database has executed successfully.
-
Guaranteed ordering
Data is forwarded in the same order that it arrived, even if a database connection is not currently available.
-
Enhanced performance
By first buffering the data in memory, the store-and-forward system can optimize writes, and prevent the originating systems from blocking. This means that the system is less likely to lose data samples in the event of system slow downs.
Store and Forward Data Flow
Although the system offers settings that can affect the pipeline, by default the data flow occurs as follows:
-
Data is generated in some system.
-
Data is placed in a memory buffer.
-
If not removed from memory buffer in some time (the Write Time), or if a certain amount of data accumulates (Write Size), it is placed in the local cache.
-
The data sink, based on a database connection, pulls data in first from the local store, and then the memory buffer, based on the Write Time and Write Size settings under Forward Settings.
-
If the data fails to forward, either due to an error in the connection or in the data itself, it is returned to the buffer or cache.
-
If the data errors out too many times, it becomes quarantined.
-
Quarantined data can be managed through the Gateway, and can be deleted or un-quarantined, once the error is resolved.
Understanding the Forward Triggers
Data is forwarded from one stage to the next based on the Write Time and Write Size triggers. These settings work as an either/or manner, meaning that if either of them is surpassed, the data is forwarded. One important point to note is that the Write Size setting influences the transaction size of similar data to be forwarded, and therefore can have a big impact on performance. As a result, the Write Time should normally be used as the controlling factor, with the Write Size set to something that will provide reasonable transactions, like 100.
Store and Forward for Reliability
The store-and-forward system settings, while seemingly limited, offer a good deal of flexibility in tuning. Different types of situations and goals will likely require different configurations.
When the safety of the data is a concern, the goal is to get the data stored to disk as quickly as possible in order to minimize risk of loss due to a power outage or system failure. The local cache plays a crucial role in this, allowing the system to store data locally for any amount of time until the remote database can accept it. This protects against network failures and database failures, as well.
By setting the write size and write time of both the local cache and forwarder to low values, the data spends less time in the memory buffer. While the memory buffer can be set to 0 to bypass it completely, this is not usually recommended, as the buffer is used to create a loose coupling between the history system and other parts of Ignition that report history. This disconnect improves performance and protects against temporary system slowdowns. In fact, it is recommended that for reliable logging, this value be set to a high value, to allow the maximum possible amount of data to enter the system in the case of a storage slowdown.
Recommended Settings
The following recommended Engine Settings (at Database > Store and Forward from the Configure section of Gateway) are merely a starting point, you can adjust them to fit your goals:
Buffer Settings |
|
|
Memory Buffer Size |
Set to 1000 or higher. While the data won't reside in here for long, a high value allows the data to enter the store-and-forward system, as opposed to being lost if the maximum is hit. |
Store Settings |
|
|
Disk Cache Enabled |
If enabled, records are written to the local disk based cache until they can be forwarded. If not enabled, any records that cannot fit in the memory buffer is lost. |
|
Max Records |
Set to 500,000 or higher. Like the memory store, if the maximum is reached, data is lost, therefore set the value high to protect against long periods of time without database connectivity. |
|
Write Size |
Set to very low, to get data into the cache as quickly as possible. Moving from the memory buffer to the disk store does not use transactions as much as forwarding to the database, so a low value should not impact performance too dramatically. A value of 1 is possible, though that would force all data to go to the cache before going on to the database. A value of 10 is a good starting point. |
|
Write Time |
This should be the controlling factor in trying to get the system to forward as quickly as possible, minimizing the time that data is in the memory buffer. If the write size is 1, this setting is of little consequence, but if the value is greater than one, carefully consider this value. Ultimately, this value should only be as large as what you are willing to lose if there were a power failure. |
Forward Settings |
|
|
Write Size |
Set this value to a decent size to increase transaction throughput. 100 is a good value. |
|
Write Time |
This setting should be less than the Store Write Time to avoid writing to the store when the target database is available. |
Store and Forward for High-speed Buffering
When configuring the store-and-forward system for high-speed buffering, you are expecting the case that data will come in quick bursts. By buffering the data, the system can accommodate more information than would be possible going directly against the database.
The key points in configuring a buffering system is to avoid expensive operations like storing and reading from the local cache, and to set the memory buffer large enough to accommodate the expected burst sizes.
Recommended Settings
The following recommended Engine Settings (at Database > Store and Forward from the Configure section of Gateway) are merely a starting point, you can adjust them to fit your goals:
Buffer Settings |
|
|
Memory Buffer Size |
Set to 500 or higher. It should be high enough to accommodate several bursts of data. For example, if you expect data to be logged at 100 ms burst for 10 seconds at a time, 100 records would be the minimum value. Data is forwarded as it comes in, according to the forward settings, but you should not rely on any particular throughput in order to avoid data loss. |
Store Settings |
|
|
Disk Cache Enabled |
Unselect this check box. Depending on your requirements, disable the disk store, or at least set to a high write size/count settings. Writing and reading from the cache is much slower than memory, so it is desirable to avoid it. Of course, the cache should only be disabled if it is ok to lose some data, should the database connection be down for a period of time. |
Forward Settings |
|
|
Write Size |
This value should be larger than the expected burst size. Burst data is from the same source, and therefore will benefit heavily from the optimizations in the buffer. |
|
Write Time |
This value should be balanced in order to give the buffer time to receive multiple records that can be optimized, as described in Write Size above. However, it should not be so long that too much data becomes scheduled to write, which could cause a system slowdown/back up. |
In This Section ...