Queuing
Queuing is important for various integration scenarios.
If Ziggy receives too many simultaneous requests that take time to process, you will overload the system at some point.
If your Flows are accessing rate-limited APIs then you will likely hit API limits if you try to run them simultaneously or in rapid succession.
Queuing is the solution to these problems.
Enabling Queuing
In order to keep the default installation as light as possible, queuing is not enabled by default. If we are controlling your Ziggy instance for you, then please request that Queuing be enabled.
Define Queues
You can define your Queues in Global Settings.
Generally, you might use the following strategy if you were integrating Hubspot with Microsoft Dynamics.
Define a General queue for miscellaneous requests.
Define a Hubspot queue for all requests that access the Hubspot Api, (which has rate limiting, typically 10 requests per second).
Define a Dynamics queue for requests to Microsoft Dynamics.
Rate limits
You can assign a rate limit to a Queue by clicking on the Queue in Global Settings.
Assigning a Flow to a Queue
You can assign a request to a Queue within the Receiver Block. If no Queue is specified, the Flow will execute immediately. Otherwise, the Flow will be added to the chosen Queue and then processed by the Queue Worker.
The queue worker processes any jobs found in the queue on an FIFO basis.
Monitoring and pausing Queues
You can access Queue stats from Queue menu bar. You can also pause and restart queues.
Security
Users with the most stringent security requirements may be concerned about any form of data persistence.
One thing to consider is queue persistence. If you want your queues to be available after a system restart then you will need to enable persistence in Redis. This means data will be persisted to disk until the queue is processed ane emptied.
Redis persistence is controlled in the redis.conf file.