Support Special Characters in Table and Column Names via Global Rules in Qlik Replication

This topic provides a step-by-step guide to support special characters using global rules in the Qlik Replication tool. This approach ensures that the transformation is applied universally across all tasks.

Issue Description

During the replication of tables, if the tables and columns contain special characters, Kafka Connect is unable to serialize the Kafka messages.

Note: Currently, only the special characters # and $ have been identified and handled to support in table/column names. Any other special characters in the table/column names need to be handled separately.

Resolution

Using global transformations, special characters will be replaced with placeholders. These placeholders will then be replaced with the actual special characters in Kafka Connect.

Prerequisites

• Qlik Replication tool installed and configured.

• Access to the Qlik Replication console.

• Necessary permissions to modify global rules.

Steps

1. Access the Qlik Replication Console

   1. Open your web browser and navigate to the Qlik Replication console URL.

   2. Log in with your credentials.

2. Navigate to Global Rules

   1. In the Qlik Replication console, go to the Replication task and designer then click **Global Rules**.

   2. Click **Add Global Rule** to create a new rule.

      

3. Define the Global Rule: Create two rules. One for table names and one for column names.

   • In the **Rule Name** field, enter a descriptive name for the rule (e.g., "Convert Table/Column Names to Support Special Characters").

   • In the **Rule Type** dropdown, select **Transformation**.

     

4. Configure the Transformation:

   1. In the transformation settings, add a new transformation rule.

   2. Set the Transformation Type to Rename table/column.

      

   3. Click Next.

      

   4. In the Rename Table/Columnto field, use a function to convert the table/column name to support special characters. You can use the following expression:

      1. Column Name : replace(replace($AR_M_SOURCE_COLUMN_NAME,"#","HASH"),"$","DOLR")

      2. Table Name : replace(replace($AR_M_SOURCE_TABLE_NAME,"#","HASH"),"$","DOLR")

         

         

5. Apply the Global Rule.

   • Ensure the global rule is enabled.

   • Save the global rule settings.

     

6. Verify the Changes: Run a task to verify that the table/column names are being retained if the table/column has special characters.

   By following these steps, you can successfully retain table/column names using global rules in the Qlik Replication tool.