- cuscon@logiticks.com
- +1 206 787 9004 (US)
Microsoft Azure Event Hubs
Event Hubs are Microsoft Azure managed platform for message(event data) streaming. It provides large-scale collection and processing capability which is required for modern applications. It helps to decouple the production and consumption of the events.
Event Hubs are specifically chosen in scenarios which require high throughput and event processing scenarios.It uses AMQP(Advanced Message Queueing Protocol) and HTTP as it’s API interfaces.
Partitioned Consumer Pattern is used in Azure Event Hub. This helps in scaling event processing massively.
A partition is a sequence of ordered events or messages and is used for data organization. Whenever a new event arrives they are added at the end of the sequence. A partition can hold data up to the retention time specified in the event hub level and which is common for all partition in an Event Hub. An Event in a partition expire at a timely basis and it cannot be deleted directly.
An Even Hub may contain multiple partitions and each is independent and contain its own data. Due to this, each partition grows at different rates.The number of partitions in the Azure Event Hub should be chosen appropriately as per requirements and can be between in number 8 and 32. After the creation of Event Hub, the partition cannot be changed. Each partition can be identified individually.
An Event Data will contain the body of the event, user-defined property bag, and metadata such as offset number in partition and number in the partition.
An Event Publisher is an entity who send data to Event Hub. It can publish data using AMQP or HTTPS. Each event publisher uses are Shared Access Signature (SAS ) token to identify them self-selves to event hub. It may be unique or common. Event hubs use this as the authentication mechanism.
Service Bus provides SAS policies at namespace level and Event hub level. A SAS token is generated using SAS key and SHA hash of URL in an encoded format. With the help of policy name and the token the Event Hub authenticate the sender. Normally for publishers, SAS token is created only with Send privilege.
An event can be published using HTTP or AMQP protocol. EventHubClient class in.Net helps to publish events to Event Hub. An event data packet has a max size limit of 256 kb. More than it causes the error.
Publisher policy is used to identify each publisher uniquely when sending data to the event hub.
A partition key in the incoming data packets is used by Event Hub to map incoming data into specific partitions. A unique identifier for each device makes a good partition key. Azure ensures that events with the same partition key are ordered in the sequence and delivered to the same partition.
A consumer is an entity which reads data from the Event Hub. Each partition will only have one consumer at a time and it read in partition through consumer groups. All event consumers user AMQP and does not need to poll every time. An Event is delivered to the consumer when it’s available.
Consumer Group is the view of an Entire hub and each enables to read the stream independently. You can access partition through consumer groups. There will be always one default consumer group and can be increased to 20 per event hub.
Stream Offset is a position of the event in the partition. This enables readers to consume events from the partitions from specific points. Consumers are responsible for storing their own offset values outside of the Event Hubs service. EventProcessorHost class for .NET clients provides a way to manage the position of the socket in the reader.
The capacity of the Azure Event Hub is measured in the Throughput units. Throughput units are pre-purchased units of capacity. A single throughput unit includes the following:
Ingress: Up to 1MB per second or 1000 events per second.
Egress: Up to 2MB per second.
A single partition has a maximum scale of one throughput unit. The number of throughput units should be less than or equal to the number of partitions in an Event Hub
If you are looking for Azure application development services contact Logiticks.
Event Hubs are specifically chosen in scenarios which require high throughput and event processing scenarios.It uses AMQP(Advanced Message Queueing Protocol) and HTTP as it’s API interfaces.
Partitioned Consumer Pattern is used in Azure Event Hub. This helps in scaling event processing massively.
A partition is a sequence of ordered events or messages and is used for data organization. Whenever a new event arrives they are added at the end of the sequence. A partition can hold data up to the retention time specified in the event hub level and which is common for all partition in an Event Hub. An Event in a partition expire at a timely basis and it cannot be deleted directly.
An Even Hub may contain multiple partitions and each is independent and contain its own data. Due to this, each partition grows at different rates.The number of partitions in the Azure Event Hub should be chosen appropriately as per requirements and can be between in number 8 and 32. After the creation of Event Hub, the partition cannot be changed. Each partition can be identified individually.
An Event Data will contain the body of the event, user-defined property bag, and metadata such as offset number in partition and number in the partition.
An Event Publisher is an entity who send data to Event Hub. It can publish data using AMQP or HTTPS. Each event publisher uses are Shared Access Signature (SAS ) token to identify them self-selves to event hub. It may be unique or common. Event hubs use this as the authentication mechanism.
Service Bus provides SAS policies at namespace level and Event hub level. A SAS token is generated using SAS key and SHA hash of URL in an encoded format. With the help of policy name and the token the Event Hub authenticate the sender. Normally for publishers, SAS token is created only with Send privilege.
An event can be published using HTTP or AMQP protocol. EventHubClient class in.Net helps to publish events to Event Hub. An event data packet has a max size limit of 256 kb. More than it causes the error.
Publisher policy is used to identify each publisher uniquely when sending data to the event hub.
A partition key in the incoming data packets is used by Event Hub to map incoming data into specific partitions. A unique identifier for each device makes a good partition key. Azure ensures that events with the same partition key are ordered in the sequence and delivered to the same partition.
A consumer is an entity which reads data from the Event Hub. Each partition will only have one consumer at a time and it read in partition through consumer groups. All event consumers user AMQP and does not need to poll every time. An Event is delivered to the consumer when it’s available.
Consumer Group is the view of an Entire hub and each enables to read the stream independently. You can access partition through consumer groups. There will be always one default consumer group and can be increased to 20 per event hub.
Stream Offset is a position of the event in the partition. This enables readers to consume events from the partitions from specific points. Consumers are responsible for storing their own offset values outside of the Event Hubs service. EventProcessorHost class for .NET clients provides a way to manage the position of the socket in the reader.
The capacity of the Azure Event Hub is measured in the Throughput units. Throughput units are pre-purchased units of capacity. A single throughput unit includes the following:
Ingress: Up to 1MB per second or 1000 events per second.
Egress: Up to 2MB per second.
A single partition has a maximum scale of one throughput unit. The number of throughput units should be less than or equal to the number of partitions in an Event Hub
If you are looking for Azure application development services contact Logiticks.