Event Data Warehouse Management
The Challenge
Event data is the fingerprint of internet and corporate system activity and is critical to preventing and minimizing corporate security threats. Recent independent reports peg the cost of stolen corporate data for a mid-sized corporation at $6.6M per incident and rising. Preventing and minimizing these threats requires precise analysis of multiple, complex event data sources in real-time and, especially, over long time frames. For many organizations, Event data is their fastest growing data and, often, their single largest data store. Event data includes:
- Network, security and database logs
- Physical access systems
- Enterprise applications
- Bank transactions
- Telco call records
- Internet traffic detail
- Manufacturing sensor data
Sophisticated long term analysis of event data is the key to addressing emerging security threats, compliance mandates, and a host of risk management initiatives.
Organizations that are not leveraging log and event data to make strategic decisions are putting their firms at risk.
The Solution
The solution’s core product is a purpose-built data management warehouse optimized for event collection, retention and analysis. It uses a patented columnar architecture that organizes data by column and stores it a highly compressed, clustered environment without the use of any indices. Data collection from any log and event source is easy and automated thanks to an embedded data collection layer. In addition, the solution’s unique data abstraction layer, Intellischema™, provides the capability to add new sources or reports on the fly without the need to change existing analytics. Finally, it includes an intuitive management console that is the source of dashboard, alerting, and business intelligence solutions. Data export and workflow is made easy thanks to extensive APIs and customization tools.
The results are dramatic. The solution cost 70% less than "modern" data warehouse or security management solutions. SenSage solutions are currently deployed at small businesses (about 5 terabytes) up to large, global environments (a petabyte or greater).
The solution is designed with the following capabilities:
- Unrivalled and precise data source support
The solution does not use “universal” or generic schemas to support event and log data. The solution is developed IntelliSchema™ that allows all sources, including custom sources, to be parsed into the data warehouse. This allows precise query capabilities. It is capable of loading data without the use of agents and leverages several techniques to provide fast and efficient data loading.
- 100% query accuracy
Compliance requirements and forensic research requires the ability to find every piece of data required. The solution uses SQL to search IntelliSchema data sources and provides correlation to other events. Other log management and SIM products use text type substring searches that do not insure accuracy.
- 360-degree visibility
Correlation and querying of an unlimited amount of log data sources including applications, databases, network infrastructure, identity access management systems and more to provide a complete view of user activity in a single query.
- Online data retention
By leveraging our columnar database, the solution compresses data to the point that years of event data can be kept online and query-able as required.
- Reporting flexibility
Different users have different reporting requirements. The solution was designed for power users as well as analysts and management. Reports are easily created and shared without the requirement for SQL knowledge. Reports can also be organized in folders and in dashboards.
- Enterprise scalability
The solution is deployed on clustered commodity hardware that allows organizations to easily scale-out rather than scale up.
- Low total cost of ownership
Log management and SIM are often viewed as a tax on your business. The solution is designed to not require a third-party relational database and therefore does not require a database administrator (DBA) to manage the system. The database is self tuning and new schemas are automatically added to the system as more data sources come on line. And by using commodity clustered hardware, the expense of adding more capacity is minimal.
Because of the volume and nature of the data being stored, traditional database warehouses quickly become impractical to use for event data for a number of reasons including performance and cost.
The solution is a turnkey solution for collecting, storing and querying event data. Built on a patented columnar database, the entire solution provides all components require for event data warehousing including the ETL and analytics layer. Additionally, it supports open access from a number of methods including SQL, Perl DBI, and JDBC.
The solution provides pre-defined solutions packages for HIPAA, Sarbanes-Oxley (SOX), Payment Card Industry Data Security Standard (PCI DSS), FFIEC, NISPOM, and DCID 6/3, GLBA, EU Data Retention and others. It enables organizations to meet compliance requirements with both pre-defined and customized reports as part of its out-of-the-box solution.
When it comes to regulatory compliance, the solution enables organizations to:
- Ensure that event log collection, correlation and retention are consistent and usable for validating/fortifying controls and investigations
- Leverage pre-defined reports and rules addressing relevant regulation sections, generating dashboards, alerts and trending
- Bridge the gap between real-time compliance monitoring and long-term, broad source investigation and audit
- Empower IT/Security staff to conduct audit and investigation processes with greater efficiency and effectiveness
- Eliminate data management issues that impact compliance
- Reduce cost to capture and store sufficient event log data
- Deliver the performance required to capture and analyze event log data
- Provide the scalability to meet increased volume and evolving requirements