When SecurityDAM, a DDoS protection service provider, needed to upgrade their NOC operations, they tested out multiple solutions before choosing XiteiT. From increasing efficiency and visibility to improving ticket resolution times and runbook automation, XiteiT provided the NOC manager and team with everything they needed to run operations more smoothly and effectively.
SecurityDAM is a DDoS protection service provider that caters to companies of all sizes, from Fortune 500 companies such as SAP and Samsung, to small businesses around the world. Their NOC is responsible for monitoring 14 data centers spread out around the globe. Due to the vast range of customers SecurityDAM serves, their system presented unique challenges that required XiteiT’s innovative solution.
Managing such a huge network of systems is extremely complex. Many issues can arise when trying to manually manage a system which deals with vast amounts of data from various sources and requires constant monitoring.
Lengthy PoC Process
When SecurityDAM realized their need for a more effective way to manage their NOC operations, they tested out multiple tools and solutions. The company quickly found that XiteiT provided a comprehensive solution to everything they needed, while offering fast setup, a user-friendly interface and full visibility into their NOC operations and runbook management. In additional to XiteiT’s technical advantages, SecurityDAM stated that the high quality service and support they received were an important factor in their final decision.
After three months of working with XiteiT’s solution, significant changes and improvements were seen in SecurityDAM’s visibility, productivity, and efficiency.
With XiteiT, SecurityDAM is able to address several of the challenges their NOC team encounters every day, specifically visibility and notification centralization, pattern identification, and runbook management.
“Since implementing XiteiT, the number of tickets SecurityDAM has issued to clients has doubled,” says Amir Paluch, NOC Manager at Security DAM. “This is a good thing! It means that we’re covering more alerts, that alerts aren’t being overlooked or mismanaged because they can’t be closed until they’re resolved.”
Visibility and Notification Centralization
Prior to the integration of XiteIT into the system, alerts received from the multiple monitoring systems were divided and sent into many channels, and were not concentrated into one place. Some were only presented visually and were not sent anywhere, while others were sent via email and Slack. There was no way for management, clients, or engineers to see the full picture of what was going on in the data centers. The alerts received were sent to many different channels and it was difficult to piece together what was going on across the system when NOC operators only had access to fragments of the whole.
Notifications also presented a challenge. Engineers received notifications of a system error and a separate notification when an error was successfully repaired. This resulted in confusion as to what original alert the “resolved” alert belonged to, as the alerts were not linked. This was particularly challenging when alerts were sent via email. Additionally, if an NOC operator received these messages while not on their shift, they would have to piece together what had happened when they returned, and identify which errors remained unsolved.
XiteiT provided SecurityDAM with a central system that could connect to all the data sources with all the alerts being sent to one centralized location. This allowed NOC operators to see the bigger picture. The new technology connected to a variety of monitoring systems with different abilities and from different domains as well as to the NOC, and all these forms of systems put out notifications which now went to a centralized location. As a result, notifications could be linked and correlated. With XiteiT, engineers no longer had to struggle to identify which messages were linked together.
Identifying patterns is a critical part of NOC management. When a series of notifications or alerts come in, either within a specific time period, about a certain service or feature, or from a particular client, there is likely a bigger issue that needs to be identified and dealt with. NOC operators and engineers were responsible for looking out for patterns in these notifications, which required that they memorize and recognize alerts. This was far from feasible, especially considering that they were sorting through thousands of alerts per day. Additionally, shifts changed, and the person who saw the start of a pattern in the first shift of the day may not be the same person who sees the rest. Different NOC operators managed different errors at the same time, each dealing with the error separately and not necessarily seeing them as parts of the same puzzle or pattern.
With everything centralized in one place, recognizing patterns in notifications became much easier. NOC teams were able to see all of the alerts in one place, even those that weren’t assigned to them in their shift. This provided a bird’s eye view of the notifications and issues, enabling the team and NOC manager to identify patterns more quickly. XiteiT’s ability to automatically recognize alert patterns and notify the engineers allowed for issues to be dealt with in a more timely and efficient manner, and reduced the chances of patterns being missed due to human error.
SecurityDAM managed procedures for resolving errors in a non-centralized manner. Some were shared orally, others were recorded in a knowledge base. When a NOC operator received an alert, they would have to search for the correct procedure. Even though a runbook in the form of a knowledge base was used; it wasn’t always clear how to find the correct procedure for each issue.
One of XiteiT’s really powerful capabilities is runbook management and automation. SecurityDAM managed their entire runbook from XiteiT, which was also where all of the alerts were going. As a result, XiteiT was able to automatically connect the runbook procedures to the relevant alerts as they came in, eliminating the need to go hunting for the correct procedure each time. Runbook automation capabilities were used to recognize alerts that fulfilled certain conditions in order to run a set of procedures. SecurityDAM used this capability to ensure SLAs were fulfilled. If an alert is not resolved within the time period defined in the SLA, the team was notified that the client should be updated. By using XiteiT’s automation features, SecurityDAM was able to “snooze” alerts until the SLA time period was up, after which the alert would have either been resolved or escalated to the NOC operators.
XiteiT allows companies with systems around the world, such as SecurityDAM, to give their clients and NOC managers a complete view of what is going on in their system while improving productivity and efficiency.
“It is difficult to summarize my experience with XiteiT in one sentence,” says Amir. “This system has given me, as a NOC Manager, full transparency, and given us a unified picture of the systems we monitor.”
To learn more about SecurityDam, please click here.