Unleashing the Energy of Python Asyncio’s Queue | by Peng Qian | Jun, 2023
[ad_1]
The queue talked about earlier is a First-In-First-Out (FIFO) queue, the place the primary merchandise to enter the queue is the primary to be retrieved. That is appropriate when all duties within the queue have the identical precedence.
Nevertheless, contemplate the next state of affairs:
Suppose there’s a queue with duties ready in line, every requiring an extended processing time.
An error log or VIP person entry is a high-priority job that wants fast consideration. What ought to we do?
That is the place asyncio.PriorityQueue
comes into play.
Briefly describe asyncio.PriorityQueue’s implementation
Not like FIFO queues based mostly on lists, asyncio.PriorityQueue
is predicated on heaps. It’s constructed utilizing a binary tree construction.
You might be accustomed to binary search timber, which be sure that probably the most minor node is all the time the leftmost node.
Nevertheless, the binary tree in asyncio.PriorityQueue
ensures that probably the most minor node is all the time on the high, so the best precedence node is completely eliminated first.
Actual-world instance with asyncio.PriorityQueue
Let’s illustrate the utilization of asyncio.PriorityQueue
with a real-world situation that exists in apply.
Think about we now have an order service API. The API takes time for every order to course of, however we are able to’t maintain customers ready too lengthy.
So when a person locations an order, the API first places the order right into a queue, permitting a background job to course of it asynchronously whereas instantly returning a message to the person.
This API accepts orders from two forms of customers: common customers and VIP customers. It should be sure that VIP person orders are processed with the best precedence.
To maintain the educational curve low for readers, on this instance, we’ll use aiohttp
to implement the server. The precise code is as follows:
First, we outline an enumeration marking the 2 classes: common customers and VIP customers.
Subsequent, we use dataclass
to outline a person’s order, which incorporates the person kind and order processing period. The order period will not be thought of in precedence sorting.
Then we outline the patron technique process_order_worker
, which retrieves orders from the queue and simulates the order processing.
Don’t overlook to make use of queue.task_done()
to inform the queue that we completed processing the order.
Following that, we implement the order API utilizing aiohttp
. This API responds to person requests, generates an order object, and locations it within the asyncio.PriorityQueue
.
It then instantly returns a response to the person, avoiding person wait time.
When this system begins, we use create_order_queue
to initialize the queue and order consumption duties.
When this system ends, we use destroy_order_queue
to make sure that all orders within the queue are processed and the background duties are closed accurately.
queue.be part of()
will watch for all the info within the queue to be processed. asyncio.wait_for
units a timeout of 20 seconds, after which it’s going to now not wait queue.be part of()
to finish.
We are able to take a look at this implementation utilizing PyCharm’s HTTP Request:
As you’ll be able to see, the 2 high-priority duties are processed as anticipated. Excellent!
[ad_2]