Skip to main content

Fetching and Executing Remote Tasks

Fetching a Remote Task​

To interact with a task that has already been registered on a Flyte cluster, you need to obtain a reference to it. The flyte-sdk provides the flyte.remote.Task.get() class method for this purpose.

This method returns a LazyEntity object, which is a proxy that defers the actual fetching of task details from the Flyte backend until they are needed. This approach optimizes performance by avoiding unnecessary network calls.

from flyte.remote import Task

# To fetch a remote task, you need its name, project, domain, and either a specific version
# or an auto_version strategy.
# Example: Fetching a task by a specific version
my_remote_task_lazy = Task.get(
    name="my_task_name",
    project="flytesnacks",
    domain="development",
    version="abcdef12345"
)

# Example: Fetching the latest version of a task
latest_remote_task_lazy = Task.get(
    name="my_task_name",
    project="flytesnacks",
    domain="development",
    auto_version="latest"
)

print(f"Lazy reference to my_task_name: {my_remote_task_lazy}")
print(f"Lazy reference to latest my_task_name: {latest_remote_task_lazy}")

The Task.get() method requires either the version or auto_version parameter to be specified. If auto_version is set to "current", the task can only be accessed from within another task's execution context, as its version is derived from the calling task's context.

Understanding LazyEntity​

The flyte.remote._task.LazyEntity class acts as a placeholder for a remote task. It holds the necessary information to retrieve the full flyte.remote._task.TaskDetails object but doesn't perform the network call until an attribute of the task is accessed or the task is called for execution. This is a core concept for efficient interaction with remote Flyte entities.

You can explicitly trigger the fetching of the TaskDetails object using the fetch() method:

from flyte.remote import Task

my_remote_task_lazy = Task.get(
    name="my_task_name",
    project="flytesnacks",
    domain="development",
    version="abcdef12345"
)

# The fetch() method is asynchronous and returns the TaskDetails object
task_details = await my_remote_task_lazy.fetch()

print(f"Fetched TaskDetails object: {task_details.name}")

Inspecting Task Details and Interface​

Once the TaskDetails object is fetched (either explicitly via fetch() or implicitly by accessing its properties), you can inspect various aspects of the remote task, such as its name, version, type, input/output interface, cache policy, and resource requirements.

from flyte.remote import Task

my_remote_task_lazy = Task.get(
    name="my_task_name",
    project="flytesnacks",
    domain="development",
    version="abcdef12345"
)

task_details = await my_remote_task_lazy.fetch()

print(f"Task Name: {task_details.name}")
print(f"Task Version: {task_details.version}")
print(f"Task Type: {task_details.task_type}")
print(f"Required Arguments: {task_details.required_args}")
print(f"Default Input Arguments: {task_details.default_input_args}")
print(f"Task Interface: {task_details.interface}")
print(f"Cache Policy Behavior: {task_details.cache.behavior}")
print(f"Task Secrets: {task_details.secrets}")
print(f"Task Resources: {task_details.resources}")

The interface property, specifically, provides a NativeInterface object that describes the task's inputs and outputs, allowing you to understand what arguments the task expects.

Executing a Remote Task​

You can execute a remote task by calling the LazyEntity object directly, passing the required input arguments as keyword arguments. This call will implicitly fetch the TaskDetails if it hasn't been fetched already.

It is important to note that executing a remote task in this manner typically occurs within the context of another Flyte task. The flyte-sdk handles the submission of the task reference to the Flyte controller for execution.

from flyte.remote import Task

# Assume