The RapidiOnline Mirror Technology is used to detect changes in systems like MS Dynamics AX that do not provide a timestamp or similar feature to identify data changes. The Mirror Technology is fully automatic – once enabled, everything happens in the background. The benefit is that the RapidiConnector will then return only changed records instead of all records. Note that the RapidiOnline Mirror Technology is not standard sql server/database technology.
Errors Caused by Validation Rules in the Destination System When Using the Mirror Technology
When using the Mirror Technolog, it is possible to receive errors when inserting or updating data in the destination system (for example, due to validation rules in Salesforce.com). You could also experience problems correcting these erred data records. This is due to the fact that the mirror will store all of the source system changes that occur in the order that they happen. When reading from the mirror, it is possible to have several changes to the same data record in the changes queue. The mirror will send each of these changes to the destination system in the order they happened (first in, first out). So even if you correct data in the source system, the mirror would still try to deliver the previous changes (before the actual change that corrected the data). As a result, the Transfer in question, will continue to fail with the same error message.
Two Ways to Recover from These Errors
One way to recover from this situation, after correcting the offending data in the source, is to perform a full rebuild of the mirror. A rebuild of the mirror will effectively delete the mirror for this source table and build it again from the source table. Note that other tables in the mirror will not be affected. This will remove the offending data from the mirror, but at the same time it will also send ALL data again to the destination, clearly a disadvantage to this solution.
A better solution is to use our Mirror Cleanup feature which was introduced in version 3.2.86m (a long time ago). It basically runs through the changes that have not yet been successfully delivered to the destination system and tries to detect if several updates were made to the same record. If so, it will update the earlier changes with the data from the newer changes and, thus, fix the problem. It will still send all of the changes (if several changes were made to the same record) to the destination system.
An Updated and Improved Cleanup Feature
In the current release, we have improved the Mirror Cleanup feature. The cleanup will still run through all of the changes that have not been successfully sent to the destination system. Instead of updating the data in the earlier changes, it will now mark these change to be Skipped so they are not sent to the destination system at all. Records are skipped only if it finds a later change to the same record in the mirror. Additionally, the Cleanup feature is more advanced in that not only does it look for subsequent updates to the record, it will also detect deletions and additions. In this way, the cleanup does a much better job now compared to earlier versions.
Updates to the User Interface in MyRapidi
To make it easier to use both the Rebuild and the Cleanup features of the Mirror Technology, we have made a number of small changes to the user interface in MyRapidi.
In order to make it easier to use both the Rebuild and Cleanup features of the Mirror Technology, we have made a number of small changes to the user interface in MyRapidi. We have added two buttons (as seen in the screenshot below) on the RTI page for relevant RTI entries (the ones that relate to Transfers using the Mirror Technology. These two buttons will update the RTI source counter value to trigger either a Rebuild or a Cleanup on the next run of the Transfer.
We have also made it easier to navigate to the correct RTI line from the Log (or Runs) pages. We have added links on relevant Log entries that will bring you directly to the RTI page and showing the relevant RTI entry.
A couple more productivity changes on MyRapidi related to RTI
In addition to the above mentioned changes, we also added a Transfer link (the rightmost icon) to the RTI page as shown above. This allows you to quickly get back to the relevant Transfer from the RTI page.
We also made it possible to completely delete a RTI entry (the trashcan icon). Use this to cleanup the RTI entries in case a Transfer is not using RTI anymore or similar.
On the Transfers page we have changed the RTI link functionality so that the RTI icon is now grey'd out if the Transfer is not using SourceControl or Mirror (in which case RTI is not relevant). It actually gives you a quick way to see which transfers are optimized and using SourceControl or Mirror !
For the transfers that are using SourceControl or Mirror, clicking the icon will check if an RTI entry exists for the Transfer with the specified Source and Destination connections; If the RTI does not exist, then it will be created. The link will of cause also take you to the relevant RTI entry on the RTI page.
Read more on the Mirror Technology on the wiki.
We hope you like these changes !
If you have any questions, please contact us.
Michael
