Technology identifies evidence of terrorist activity in 100 billion call data records
EMC Corporation, Intec Telecom Systems and SenSage have pooled resources to provide a comprehensive, cost effective solution to answer the mandates of the EU Data Retention Directive. The combined technologies tackle two issues. First, it provides telecommunication and Internet service providers with a lower cost means to manage communication records that support compliance with the Directive. Second, it enables law enforcement agencies to quickly access historical phone and Internet records to pinpoint and prosecute terrorist activity and other serious crime. The Directive states that all service providers must be compliant by August, 2007.
The EU Data Retention Directive was announced in March 2006. It requires telecommunications operators (Telcos) and Internet Service Providers (ISPs) to securely retain and be able to analyse traffic and location data related to; fixed telephony; mobile telephony; Internet access; Internet e-mail; and Internet telephony for up to two years. The Directive further requires that all service providers must produce answers to law enforcement inquiries without any undue delay. Millions of calls and connections occur each day equating to billions of records and terabytes of transaction data that must be securely stored and rapidly analysed.
EMC, Intec and SenSage have developed a joint EU Data Retention solution that significantly reduces the cost of compliant data management for service providers. The combined technology completely supports the guidelines set out by the Directive and is able to cost-effectively manage and obtain results in minutes from over 100 billion Call Detail Records (CDR).
The joint solution has been fully tested with sample CDR data from a leading telecommunications provider. The sample data, collected and processed through an Intec mediation platform, was cleansed, then subsequently randomised and replicated to generate the necessary volume. SenSage provides the scalable event data management platform that powers the collection, compression, management and high-speed analysis of the CDR data. Retention and online availability of the stored data is fully managed within an EMC Centera storage cluster, which provides increased data integrity, protection, and availability capabilities. Intec provided switch remediation expertise and design guidance. The Proof-of-Concept (POC) contains 100 billion records, representing 13 terabytes of raw compressed data. It is now maintained in an online, referenceable repository where complex analytics can quickly be performed to investigate criminal activity.
The joint PoC simulated retention of two years of CDR data of 10 million Telco subscribers. The system executed un-indexed queries at more than 27.9 million records per second and obtained answers in less than seven minutes within a three month search range. The combined solution achieved these results at a fraction of the overall cost when compared to that of conventional approaches, which rely on relational database management technologies and data archives.
The EU Data Retention technology is comprised of the SenSage scalable event data management platform, pre-defined reports that support directive guidelines and EMC Centera content addressable storage (CAS) to yield a high-performing, extensible and secure information lifecycle technology. SenSage is an EMC Centera Proven solution that transparently integrates with EMC Centera for fast, easy online data access with assured content authenticity and petabyte scalability. This product can be predictably expanded to meet varying Telco telephony, IP and messaging sources, data volumes and implementation requirements at the lowest cost.
The joint SenSage/EMC/Intec solution POC, housed in the EMC labs in Cork, Ireland, has been made possible by the efforts of a number of EMC Technology partners. The SenSage event data management platform leveraged leading edge technologies from Dell with their new range of Dual Core, Dual CPU PowerEdge 2950 servers containing the latest Intel Xeon 5160 “Woodcrest” processors and the latest 64-Bit version of RedHat Enterprise Server 4. When coupled with the EMC Centera Content Addressed Storage via Cisco Catalyst 3750 high performance switch, the overall architecture provides the most price-performing solution stack to the issue of collecting, storing and analysing the vast amount of data required by the EU directive.