Relational databases and SQL continue to be the backbones for data analytics in the enterprise, but present relational databases are inadequate for processing large amounts of high velocity data within real-time limits. While relational database features perform very well with moderate data sets, limits are quickly reached on high velocity data feeds from sensors or customer tracking and tables with many billions of rows.
Swarm64 enables your IT to handle large amounts of high velocity data and eliminates the risks and costs inherent to introducing new IT systems. The Swarm64SDA utilizes the latest generation Intel® Arria® 10 FPGAs to deliver industry leading performance for your use-case:
- Up to 10.6x higher data ingestion speed 1
- Up to 9.2x higher analytics performance 2
- Up to 13.2x database size reduction 3
- Scaling to hundreds of billions of rows
Swarm64SDA fulfills the requirement to scale to many billions of rows and millions of inserts per second for customers in various situations such as the monitoring of 40GbE network connections, gathering the sensor outputs from an array of intelligent devices, ingest behavioral data from customers or telemetry data from datacenters or machine parks. Swarm64’s fast analytics on these datasets enables immediate actions for use cases such as instant threat detection, increased customer conversion or predictive maintenance.
Swarm64SDA is an easy to use, cost-effective solution seamlessly plugging into the most popular databases including PostgreSQL, MariaDB and MySQL.
Swarm64 redesigned the data structures of the database and the data-flow during processing with the objective to touch less data during a query and move the data that needs to be processed faster through the system. The Intel® FPGA serves two purposes: it increases I/O-throughput and it reduces the amount of data and the number of operations that the CPU needs to process. A software layer wraps the FPGA processing and the proprietary data structures into a plug-in that links to supported databases through standard interfaces†.
Furthermore, Swarm64’s cooperation between CPU and FPGA overcomes the latency increase and bandwidth limitations of storage accessed via the network or in a typical cloud infrastructure. Swarm64’s solution thereby decouples storage from compute and enables resource elasticity and an excellent cost/performance ratio.
- Test conducted: Timed ingestion of 2.5 billion rows into the three databases with multiple workers, each row has a total size of 25 bytes and contains 4 fields. Timed table-scan query to gather preliminary information about the ingested data. Timed table-scan queries of 4 different result-set sizes (tiny, small, medium and large). Timed indexed queries of 4 different result-set sizes.
- The machine used for measurements is a Supermicro* SuperServer 2028U-TR4+ with the following configuration: Super X10DRU-i+ Mainboard, 2x Intel® Xeon® E5-2695 v4 CPUs, 8x Samsung* 32GB DDR4-2400 ECC RAM.
- Swarm64DB was compared to PostgreSQL 9.6.3 and MariaDB 10.1.21 by Swarm64 staff members, no external audit. Execution of Swarm64's "Network Security Use Case" where network information data is ingested and queried simultaneously.
†Swarm64DB plugs into PostgreSQL via Foreign Data Wrapper and into MariaDB or MySQL via the Storage Engine interface.