CREATE TOPIC

Syntax

CREATE TOPIC
    topic_name
    [IN STORE store_name]
[WITH (topic_parameter = value [, ...])];

Description

DeltaStream processes streaming data that is stored in streaming Store such as Apache Kafka and Amazon Kinesis. The Topic represents the data Stream entity that holds the individual data records in these streaming stores. For example, a Topic in a Kafka Store would be the Kafka topic, a Topic in a Kinesis Store would be a Kinesis data stream.

Arguments

topic_name

The name of the new Topic. For case-sensitive names, the name must be wrapped in double quotes, otherwise, the lowercased name will be used.

IN STORE store_name

Optionally, this creates the Topic in the specified Store. For case-sensitive names, the name must be wrapped in double quotes, otherwise, the lowercased name will be used.

WITH (topic_parameter = value [, …​])

This clause specifies #_topic_parameters.

Topic Parameters

Parameter Name	Description
key.descriptor	A qualified https://github.com/deltastreaminc/ds-docs-gitbook/blob/main/reference/sql-syntax/ddl/broken-reference/README.md name used to decode a record's key, if applicable. Clear the Descriptor by specifying NULL. Required: No Default value: None Type: String Valid values: See LIST DESCRIPTORS.
value.descriptor	A qualified https://github.com/deltastreaminc/ds-docs-gitbook/blob/main/reference/sql-syntax/ddl/broken-reference/README.md name used to decode a record's value. Clear the Descriptor by specifying NULL. Required: No Default value: None Type: String Valid values: See LIST DESCRIPTORS.

Kafka Specific Topic Parameters

Parameters to be used if the associated Store is type KAFKA:

Parameter Name	Description
topic.partitions	The number of partitions to use when creating the Topic. Required: No Default value: 1 Type: Integer Valid values: [1,…]
topic.replicas	The number of replicas to use when creating the Topic. Required: No Default value: 1 Type: Integer Valid values: [1,…]

Kinesis Specific Topic Parameters

Parameters to be used if the associated Store is type KINESIS:

Parameter Name	Description
kinesis.shards	The number of shards to use when creating the Topic. Required: No Default value: 1 Type: Integer Valid values: [1,…]

Examples

Create a new Topic with defaults

The following creates a Topic called pageviews using the default parameters in the user's default store:

CREATE TOPIC pageviews;

Create a new Topic with key/value Descriptors

The following creates a Topic called pageviews_pb in the user's default store. It also sets the key and value descriptors that are necessary for serializing its records:

SHOW DESCRIPTORS;
         Name
----------------------
  pb.pageviews_key
  pb.pageviews_value

CREATE TOPIC
    pageviews_pb
WITH (
    'key.descriptor' = 'pb.pageviews_key',
    'value.descriptor' = 'pb.pageviews_value'
);

Create a new Topic in a Kafka Store with Kafka parameters

The following creates a Topic called PageviewsKafka in the store named KafkaStore with 2 partitions and a replication factor of 3:

CREATE TOPIC
    "PageviewsKafka"
    IN STORE "KafkaStore"
WITH ('kafka.partitions' = 2, 'kafka.replicas' = 3);

Create a new Topic in Kinesis Store with Kinesis parameters

The following creates a Topic called pageviews_kinesis in the store named kinesis_store with 3 shards:

CREATE TOPIC
    pageviews_kinesis
    IN STORE kinesis_store
WITH ('kinesis.shards' = 3);