Task

What is Task?

An "Task" is a single execution of a Model's invoke function. You create a new Task by executing either run or run_sync on an InferenceJob.

Queuing Mechanism

Each InferenceJob has its own dedicated queue. All tasks enter this queue first and are then claimed and executed in sequence by the Workers assigned to that InferenceJob.

Input and Output

The input for an task is passed as a String to the model_invoke_function. The result stores the output generated by the Worker's execution of the function.

Timing Metrics

• Queue Time: The time difference between created_at and process_start_time represents the time the task spent waiting in the queue. This should be zero if there are enough Workers.

• Processing Time: The time difference between process_start_time and process_finish_time measures the duration taken by the Worker to complete the inference task.

Example task of the stable diffusion model