ModifyShard Count Throttling

ModifyShard Count Throttling


Tolet applications utilize resources up to some level then control themwhen the maximum allowed limit is reached. The system ought toregulate the usage of resources so that when the usage passes themaximum allowed or system-defined threshold, it can monitorentreaties from additional users to ensure that the system continuesto function to attain the desired service level of service levelagreements (SLAs).

Thispattern will be used:

  • To make sure that a system functions as required in meeting the desired service level.

  • To prevent one user from attaining the monopoly of resources provided by the system.

  • To handle increased activity levels.

  • To minimize the costs of running a system by regulating the number of resources needed to keep it running.


Thesystem can utilize some controlling schemes, including:

  • Rejecting additional attempts from a single user who has already attempted accessing the system’s APIs for more than the allowed number of times per second within a certain duration. This necessitates that the application records and stores data on resources usage by the users.

  • Easy and fast to maintain and update. All the operation is linear time, and since we just need to operate a certain queue instead of the clean table every day, it reaches the limit in both efficiency and economy.

Figure1 shows how the requests go through the throttle table and then passinto the Message Log Control Server.

Structureand Algorithm

ModifyShard Count Throttling Table—Hash Table




Operation: ModifyShardCount/Account xxxxxxx/Stream:xxxxxx/TargetShardCount:xxxxxx/ScalingType:UniformScaling

Linux Time

How many times customer called ModifyShardCount API

Key:Operation: ModifyShardCount/Account xx xxx xx/Stream: xx xxxx/TargetShardCount: xxx xxx/ScalingType: Uniform Scaling

Time:Linux Time

Value:How many times customers called ModifyShardCount API


Itis easy to enqueue and dequeue requests. Besides storage of therequests into the Hash Table, it also uses an array list based queueto store the requests. The size of the array is the limitation of acustomer. The reason why use array list is that, compared with thearray, it is easy to increase or decrease the size. In addition,compared with the linked list, it is also easy to get the head andtail.

Forevery request, based on the customer account and stream name, wecreate a queue. Every time, when a new request comes,

  1. ModifyShardCountRequest queue = Get Queue () check whether the queue exists or not if queue==Null means the queue does not exist then create one.

  2. Then, we compare the size of the array list with the Customer limitation.

  3. If Size of Array list &lt Limitation of Customer Enqueuer (request) else we compare the time of head with the new request Head = Get Head () we get the head of the queue.

  4. If the time of head and time of a request, have same date YEAR/MONTH/DAY. Reject the request and through an exception: Customer reaches the limitation of ModifyShardCountAPI. Else Dequeue All () Enqueue (request) // It will update the ModifyShardCountThrottlingTable

Comparisonwith Coral Throttle

SinceModifyShardCount is a lower traffic API, instead of sending anotification to every proxy, creating a small size and maintainabletable in DDB is more economical and efficient.

Issuesand considerations

Thefollowing factors should be put into account when making a decisionon the implementation of the proposed pattern:

  • Throttling a system, and the approach employed, is a structural decision that influences the whole system design. Adjusting should be deemed early in the process of designing the system because it is not easy to modify system parts once it has been executed.

  • Changes should be performed within a short time. The application must be in a position to sense increased activity levels and react in the right manner. It must also be structured in such a way that it will bounce back to its original functionality state when the load has reduced. This needs continuous capture and monitoring of the system’s performance data.

  • If a service denies a user request to access the system, it must be accompanied by a particular error code so that the user comprehends and appreciates the reason behind the unsuccessful request. The error code should direct the user on the amount of time to wait before the system goes back to normal.

  • Adjusting can be used as a short-term measure while a system goes beyond the allowed user limit. In many instances, it would be ideal to control the functionality, instead of scaling when the level of activity increases suddenly. This would be good to avoid high running costs.

  • If there is a rapid growth in resource demand, the functionality of the system may be affected, even when in a throttled mode. If it is not allowed, then the administrators should consider upholding bigger capacity reserves and designing more auto-scaling.


Ifthe DDB is done, just reject the request and return DDB doneexception.

Thereis the need to test the performance under low traffic environment andhigh traffic environment. This can help find the turning point forother APIs that will reveal the most economical and efficientthrottling between coral throttling and throttle table.