Introduction
With the advent of time, the degree of application complexity has increased considerably. This dictates that each phase of Development is to be managed with minimal or no margin of error. There are tools originating from multiple vendors that help accomplish this. However, the biggest challenge posed is the accuracy of information flow among the cross-functional teams working on these disparate tools. Though a handful of tool integration solutions are available today, the question lies whether just integration is good enough to bridge the gap. Any integration without having a facility of real-time monitoring of the data progress keeps the stakeholders in the dark in terms of data sanity. This data monitoring is not only pertinent to data integration but also to data migration between these sets of tools. Data monitoring for finding the quantity of data that got missed during data movement and data loss during integration/migration are not the only objectives; some more specificity is required. Details of the data that were to be synced, the reason and details of errors if any that occurred during the synchronization or the time interval it took for the data synchronization to occur are also very important. This is only possible if proper monitoring capabilities of data synchronization are in place in the tool.
Characteristics and Expectations from Data Monitoring
Let us discuss some of the characteristics of monitoring while dealing with an Integrated Toolset and the reasons behind each one of them. On a micro level we may want to know about the specifics of the data synchronization. The monitors should give us information about the failures or delay in synchronization if at all they happen. The attributes leading to the data sync failure must be visible from the monitor itself for taking corrective action when required. It will be beneficial if the datasets that failed to be synced are made visible from the monitor to trace the missed data. On a higher level a graphical data report will be quite helpful as it gives a real time information on the data synchronization. This will be helpful in quick identification of successful or failed data synchronization between multiple tools without having to spend much time delving into detailed data logs as was the case with micro level monitoring. At the macro level of monitoring, the overall statistics of all data synchronization that occurred within a certain time frame will form the basis of decision making.
Achieve all these through Kovair Omnibus
Understanding the needs and ways of monitoring, ESB architected Kovair Omnibus Integration Platform provides an off-the-shelf set of monitoring and diagnostics tools for the integration activities to ensure quality of integration and migration. Kovair Omnibus supports integration between tools through an Event Action based mechanism. That is, every time an event like data insertion or modification or deletion occurs in one tool, a similar action gets triggered in one or more target tools as per their codeless configuration done during set up based on the data synchronization needs. This mechanism is termed as Service Flow in Kovair.
Monitoring Service Flows
Kovair Omnibus having a web based interface for controlling configurations, allows users to monitor the status of each of these service flows whenever necessary.
Data Syncing Integrity with Kovair Omnibus and Reports But what if the user needs comprehensive details about the previous and present performances of data synchronization including the contents of the data that were synchronized. Kovair Omnibus offers a detailed Event and Action Log mechanism for this purpose. The Event Logs provide Event details that took place at any Source Tool including the details of Fields’ values that were to be synced to any target tool. The Action Logs on the other hand provide details of the Action that took place along with the data which were created/modified in the target tool over any particular date range.
Monitoring Data Collection
So far we have seen from the previous scenarios that all the data that are moving in between the different tools can be monitored using the textual interface of the Omnibus Event and Action Logs. However, what if we need to check if the data to be synchronized have been trapped from the source tool. For achieving this we have the Logging mechanisms for each of the tool adapters/connectors which can give us the detailed history of data collection from the source tool based on any event triggers. All this information is readily accessible without leaving the Kovair Application environment.
The Omnibus Monitor
The Omnibus Monitor gives a graphical view for tracking the data synchronization in real time as and when they occur. Any successful integration action is identified by a Green dot , the Failure is represented by a Red Dot and a link symbol is used to represent synchronization of relationship between objects that are occurring. Clicking on any part of the diagram will give the basic information of the time, status and the type of event/action that were performed.
Integration Statistics – Single menu troubleshooting
On an intermediate level between the micro and macro level monitoring, Kovair Omnibus provides the Integration Statistics facility which gives an overview about the number of times the events and actions done performed by the corresponding tool adapter/connector. The total number of successful or failed data collection or data execution into the specified tool can be viewed from this metrics report.
Dashboards for Monitoring
Kovair Omnibus Integrations Platform addresses the need for macro level monitoring of data synchronization with a dashboard for viewing the broad level quantification of all the integration actions performed in any time frame. This dashboard includes several kinds of reports and metrics which provide information about the successful vs failed synchronization attempts. This dashboard also allows users to see the average time taken (in secs) per transaction between a set of tools.
The ESB architected Kovair Omnibus platform as described above, not only allows different tools to communicate with each other but also facilitates different types of real-time monitoring. This provides total confidence to the stakeholders and provides them the details of the correctness of data getting transmitted between different tools.