Architecture

• System Architecture

• Technical Architecture

• Components

• Network Access

Note: In version 5.0, a new enterprise connector that uses Kafka has been added to stream changes into the Actian Vector database. This enhancement ensures continued seamless data replication, complementing the existing Qlik connector.

System Architecture

Here is a diagram with the standard System Architecture of a Hubble Enterprise Deployment.

Technical Architecture

Here is a diagram with the standard Technical Architecture of a Hubble Accelerator Deployment.

The Hubble Accelerator techinical architecture integrates various components to ensure efficient data flow from source systems to the target database, addressing challenges encountered during migration. Here are the key components:

• Source Systems: Oracle, SQL Server, and DB2 databases supported by the product.

• Qlik Replicate: Primary replication component that pulls data from source ERP systems and sends it to Kafka.

• Kafka Component:

  • Partitions: Kafka stores data in partitions, managed by brokers.

  • Brokers: Multiple brokers ensure high availability; if one broker fails, another takes over.

  • Topics: Similar to table rows in RDBMS, topics organize data within partitions.

• JDBC Sink Connector: Key engine responsible for reading data from Kafka partitions and pushing it to the target Actian Vector database.

• Schema Registry: Holds schema information for messages, ensuring data consistency between source and target.

• Zookeeper: Control center for Kafka components, managing coordination and high availability.

• Hubble Kafka Sink Configuration Manager API:

  • Retrieves the DDL of the source table and constructs an equivalent DDL on the target database.

  • Runs the DDL on the target database, deleting the table if it already exists.

  • Monitors the current status of Kafka, indicating whether Kafka is online or if a restart is needed.

• Monitoring:

  • Qlik Replicate UI: Provides real-time data replication monitoring.

  • Kafka UI: Included in the software bundle, allowing users to monitor Kafka to an extent.

  • Hubble Accelerator Row Count Comparer Utility: Utility for consultants and support engineers to view row counts on the source and target.

    • Reads Qlik Replicate execution task details and row counts.

    • Runs SELECT COUNT(*) queries on the target database and generates a CSV file with table name, source row count, and target row count.

• Prometheus Data Source and Grafana Dashboards:

  • Optional components for monitoring real-time data replication and identifying performance issues.

  • Not included in the default package to avoid additional resource consumption.

  • Can be installed by consultants if performance issues are observed.

Components

Hubble Accelerator installs:

• Actian Vector: SQL RDBMS designed for high performance in analytical database applications.

• Qlik Replicate (formerly Attunity Replicate): Data replication service.

• Confluent Kafka: Kafka is a distributed streaming system used for real-time data pipelines and data integration at scale.

• Kafka UI: Tool that makes your data flows observable, helps find and troubleshoot issues faster, and delivers optimal performance. The lightweight dashboard makes it easy to track key metrics of your Kafka clusters, including Brokers, Topics, Partitions, Production, and Consumption.

• Hubble Kafka Sink Configuration Manager API

• Hubble Accelerator Row Count Comparer: This utility compares row counts between a source file (.tsv) and a target database.

• Hubble Backup API: Facilitates backups for the Actian Vector data in case of migrations and upgrades.

Network Access

Here are the ports that need to be configured in the firewall to ensure the normal functioning of Hubble Accelerator within a Hubble deployment.