Queue
For requests that take longer than several seconds, as it is usually the case with AI applications, we have built a queue system.
Utilizing our queue system offers you a more granulated control to handle unexpected surges in traffic. It further provides you with the capability to cancel requests if needed and grants you the observability to monitor your current position within the queue. Besides that using the queue system spares you from the headache of keeping around long running https requests.
Queue endpoints
You can interact with all queue features through a set of endpoints added to you function URL. The endpoints are as follows:
Endpoint | Description |
---|---|
/fal/queue/submit | Adds a request to the queue |
/fal/queue/{request_id}/status | Gets the status of a request |
/fal/queue/{request_id}/response | Gets the response of a request |
/fal/queue/{request_id}/cancel | Cancels a request |
To add a request to the queue, simply incorporate the /fal/queue/submit
path to your function URL.
For instance, should you want to use the curl command to submit a request to the aforementioned endpoint and add it to the queue, your command would appear as follows:
curl -X POST https://1714827-joke.gateway.alpha.fal.ai/fal/queue/submit
Here's an example of a response with the request_id
:
{ "request_id": "30261086-6d27-489c-ae42-332e58a6a0f7" }
Request status
Once you have the request id you may use this request id to get the status of the request. This endpoint will give you information about your request's status, it's position in the queue or the response itself if the response is ready.
curl -X GET https://1714827-joke.gateway.alpha.fal.ai/fal/queue/{request_id}/status
Here's an example of a response with the IN_QUEUE
status:
{
"status": "IN_QUEUE",
"queue_position": 0,
"response_url": "/fal/queue/requests/6bf19c38-eda7-4c55-91cc-c8dc5d600f1d/response"
}
Status types
Queue status
can have one of the following types and their respective properties:
-
IN_QUEUE
:queue_position
: The current position of the task in the queue.response_url
: The URL where the response will be available once the task is processed.
-
IN_PROGRESS
:logs
: An array of logs related to the request.response_url
: The URL where the response will be available.
-
COMPLETED
:logs
: An array of logs related to the request.response_url
: The URL where the response is available.
Logs
The logs
attribute in the queue status contains an array of log entries, each represented by the RequestLog
type. A RequestLog
object has the following attributes:
message
: a string containing the log message.level
: the severity of the log, it can be one of the following:STDERR
|STDOUT
|ERROR
|INFO
|WARN
|DEBUG
source
: indicates the source of the log.timestamp
: a string representing the time when the log was generated.
These logs offer valuable insights into the status and progress of your queued tasks, facilitating effective monitoring and debugging.
Cancelling a request
If your request is still in the queue and not already being processed you may still cancel it.
curl -X PUT https://1714827-joke.gateway.alpha.fal.ai/fal/queue/{request_id}/cancel
Getting the response
Once you get the COMPLETED
status, the response
will be available along with its logs
.
curl -X GET https://1714827-joke.gateway.alpha.fal.ai/fal/queue/{request_id}/response
Here's an example of a response with the COMPLETED
status:
{
"status": "COMPLETED",
"logs": [
{
"message": "2020-05-04 14:00:00.000000",
"level": "INFO",
"source": "stdout",
"timestamp": "2020-05-04T14:00:00.000000Z"
}
],
"response": {
"message": "Hello World!"
}
}
A simple queue recipe
Submit your request and let our client handle the status tracking for you. The next section details how the fal client simplifies building apps with fal functions.