prompttools.requests package#

Submodules#

prompttools.requests.request_queue module#

class prompttools.requests.request_queue.RequestQueue#

Bases: object

A generic queue for processing requests in the prompttools library. It can be used to handle and time requests to any LLM asynchronously.

enqueue(callable, args)#

Adds another request to the queue.

Parameters:
Return type:

None

get_input_args()#

Joins the queue and gets input args that lead to the result.

Return type:

List[Dict[str, object]]

get_latencies()#

Joins the queue and gets latencies.

Return type:

List[float]

get_results()#

Joins the queue and gets results.

Return type:

List[Dict[str, object]]

shutdown()#

Stops the worker thread from executed and joins it.

Return type:

None

prompttools.requests.retries module#

prompttools.requests.retries.generate_retry_decorator(wait_lower_bound=3, wait_upper_bound=12, max_retry_attempts=5)#

Creates a retry decorator that can be used for requests. It looks for specific exceptions and waits for certain about of time before retrying. This improves the reliability of the request queue.

Parameters:

  • wait_lower_bound (int) – lower bound to the wait time before retry, defaults to 3.

  • wait_upper_bound (int) – upper bound to the wait time before retry, defaults to 12.

  • max_retry_attempts (int) – maximum number of retries before stopping, defaults to 5.

Module contents#