Queue

For requests that take longer than several seconds, as it is usually the case with AI applications, we have built a queue system.

Utilizing our queue system offers you a more granulated control to handle unexpected surges in traffic. It further provides you with the capability to cancel requests if needed and grants you the observability to monitor your current position within the queue. Besides that using the queue system spares you from the headache of keeping around long running https requests.

Queue endpoints

You can interact with all queue features through a set of endpoints added to you function URL. The endpoints are as follows:

To add a request to the queue, simply incorporate the /fal/queue/submit path to your function URL.

For instance, should you want to use the curl command to submit a request to the aforementioned endpoint and add it to the queue, your command would appear as follows:

curl -X POST https://1714827-joke.gateway.alpha.fal.ai/fal/queue/submit

Here's an example of a response with the request_id:

{ "request_id": "30261086-6d27-489c-ae42-332e58a6a0f7" }

Request status

Once you have the request id you may use this request id to get the status of the request. This endpoint will give you information about your request's status, it's position in the queue or the response itself if the response is ready.

curl -X GET https://1714827-joke.gateway.alpha.fal.ai/fal/queue/{request_id}/status

Here's an example of a response with the IN_QUEUE status:

{
  "status": "IN_QUEUE",
  "queue_position": 0,
  "response_url": "/fal/queue/requests/6bf19c38-eda7-4c55-91cc-c8dc5d600f1d/response"
}

Status types

Queue status can have one of the following types and their respective properties:

• IN_QUEUE:

  • queue_position: The current position of the task in the queue.
  • response_url: The URL where the response will be available once the task is processed.

• IN_PROGRESS:

  • logs: An array of logs related to the request.
  • response_url: The URL where the response will be available.

• COMPLETED:

  • logs: An array of logs related to the request.
  • response_url: The URL where the response is available.

Logs

The logs attribute in the queue status contains an array of log entries, each represented by the RequestLog type. A RequestLog object has the following attributes:

• message: a string containing the log message.
• level: the severity of the log, it can be one of the following:
  • STDERR | STDOUT | ERROR | INFO | WARN | DEBUG
• source: indicates the source of the log.
• timestamp: a string representing the time when the log was generated.

These logs offer valuable insights into the status and progress of your queued tasks, facilitating effective monitoring and debugging.

Cancelling a request

If your request is still in the queue and not already being processed you may still cancel it.

curl -X PUT https://1714827-joke.gateway.alpha.fal.ai/fal/queue/{request_id}/cancel

Getting the response

Once you get the COMPLETED status, the response will be available along with its logs.

curl -X GET https://1714827-joke.gateway.alpha.fal.ai/fal/queue/{request_id}/response

Here's an example of a response with the COMPLETED status:

{
  "status": "COMPLETED",
  "logs": [
    {
      "message": "2020-05-04 14:00:00.000000",
      "level": "INFO",
      "source": "stdout",
      "timestamp": "2020-05-04T14:00:00.000000Z"
    }
  ],
  "response": {
    "message": "Hello World!"
  }
}