Tasks

Tags: Basic

A task serves as the fundamental building block and an extension point within Flyte. It exhibits the following characteristics:

1. Versioned (typically aligned with the git sha)

2. Strong interfaces (annotated inputs and outputs)

3. Declarative

4. Independently executable

5. Suitable for unit testing

A Flyte task operates within its own container and runs on a Kubernetes pod. It can be classified into two types:

1. A task associated with a Python function. Executing the task is the same as executing the function.

2. A task without a Python function, such as a SQL query or a portable task like prebuilt algorithms in SageMaker, or a service calling an API.

Flyte offers numerous plugins for tasks, including backend plugins like Athena.

This example demonstrates how to write and execute a Python function task.

Note

To clone and run the example code on this page, see the Flytesnacks repo.

To begin, import task from the flytekit library:

basics/task.py

from flytekit import task

The use of the task() decorator is mandatory for a PythonFunctionTask. A task is essentially a regular Python function, with the exception that all inputs and outputs must be clearly annotated with their types. Learn more about the supported types in the type-system section.

We create a task that computes the slope of a regression line:

basics/task.py

@task
def slope(x: list[int], y: list[int]) -> float:
    sum_xy = sum([x[i] * y[i] for i in range(len(x))])
    sum_x_squared = sum([x[i] ** 2 for i in range(len(x))])
    n = len(x)
    return (n * sum_xy - sum(x) * sum(y)) / (n * sum_x_squared - sum(x) ** 2)

Note

Flytekit will assign a default name to the output variable like o0. In case of multiple outputs, each output will be numbered in the order starting with 0, e.g., o0, o1, o2, etc.

You can execute a Flyte task just like any regular Python function:

basics/task.py

if __name__ == "__main__":
    print(slope(x=[-3, 0, 3], y=[7, 4, -2]))

Note

When invoking a Flyte task, you need to use keyword arguments to specify the values for the corresponding parameters.

To run it locally, you can use the following pyflyte run command:

pyflyte run \
  https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/basics/basics/task.py \
  slope --x '[-3,0,3]' --y '[7,4,-2]'

If you want to run it remotely on the Flyte cluster, simply add the --remote flag to the pyflyte run command:

pyflyte run --remote \
  https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/basics/basics/task.py \
  slope --x '[-3,0,3]' --y '[7,4,-2]'