Execution vs. Serialization Contexts

A task has two different lifecycle boundaries in flyte-sdk: it is described before execution, and it is run after an action has started. SerializationContext belongs to the first boundary. TaskContext belongs to the second. They both contain fields such as version, paths, and code-bundle information, but they are not interchangeable settings objects.

SerializationContext supplies the information needed to turn a task into a deployable task template and runtime command. TaskContext is the concrete state installed around an executing task so that the task body and runtime services can access the current action, storage locations, checkpoints, report, and execution mode.

The two contexts at a glance

TaskContext is declared in models.py as a frozen, keyword-only dataclass. Its required runtime fields are action, version, raw_data_path, output_path, run_base_dir, and report; it also accepts optional input and runtime-service state:

@dataclass(frozen=True, kw_only=True)
class TaskContext:
    action: ActionID
    version: str
    raw_data_path: RawDataPath
    input_path: str | None = None
    output_path: str
    run_base_dir: str
    report: Report
    group_data: GroupData | None = None
    checkpoints: Checkpoints | None = None
    code_bundle: CodeBundle | None = None
    compiled_image_cache: ImageCache | None = None
    data: Dict[str, Any] = field(default_factory=dict)
    mode: Literal["local", "remote", "hybrid"] = "remote"
    interactive_mode: bool = False

The raw_data_path field is a RawDataPath, whereas input_path, output_path, and run_base_dir are strings. This distinction matters when passing values onward: the fields are not a uniform collection of filesystem-path strings. mode identifies local, remote, or hybrid execution, and interactive_mode records the interactive/debug execution state.

SerializationContext, also in models.py, is a regular mutable dataclass whose fields describe task identity, packaging, and the command that will be serialized:

@dataclass
class SerializationContext:
    version: str
    project: str | None = None
    domain: str | None = None
    org: str | None = None
    code_bundle: Optional[CodeBundle] = None
    input_path: str = "{{.input}}"
    output_path: str = "{{.outputPrefix}}"
    interpreter_path: str = "/opt/venv/bin/python"
    image_cache: ImageCache | None = None
    root_dir: Optional[pathlib.Path] = None

Its defaults are serialization values, not concrete paths for a local task process. In particular, {{.input}} and {{.outputPrefix}} are Flyte template expressions that remain in the serialized container definition until the execution environment supplies their values.

Serialization before an action runs

The run path creates a SerializationContext before translating a task to its wire representation. _run.py computes or obtains a version, rejects preparation without one, and passes the context to translate_task_to_wire:

version = self._version or (
    code_bundle.computed_version if code_bundle and code_bundle.computed_version else None
)
if not version:
    raise ValueError("Version is required when running a task")
s_ctx = SerializationContext(
    code_bundle=code_bundle,
    version=version,
    image_cache=image_cache,
    root_dir=cfg.root_dir,
)
task_spec = translate_task_to_wire(obj, s_ctx)

Deployment follows the same separation. _deploy.py creates one context containing the configured project, domain, organization, code bundle, version, image cache, and repository root, then passes that context into each task deployment, including dry-run translation:

sc = SerializationContext(
    project=cfg.project,
    domain=cfg.domain,
    org=cfg.org,
    code_bundle=code_bundle,
    version=version,
    image_cache=image_cache,
    root_dir=cfg.root_dir,
)

tasks = []

for env_name, env in deployment_plan.envs.items():
    logger.info(f"Deploying environment {env_name}")
    if isinstance(env, TaskEnvironment):
        for task in env.tasks.values():
            tasks.append(_deploy_task(task, dryrun=dryrun, serialization_context=sc))

The serializer consumes the identity fields directly. In _internal/runtime/task_serde.py, get_proto_task uses project, domain, org, and version to construct the protobuf task identifier:

def get_proto_task(task: TaskTemplate, serialize_context: SerializationContext) -> tasks_pb2.TaskTemplate:
    task_id = identifier_pb2.Identifier(
        resource_type=identifier_pb2.ResourceType.TASK,
        project=serialize_context.project,
        domain=serialize_context.domain,
        org=serialize_context.org,
        name=task.name,
        version=serialize_context.version,
    )

The default task implementation also uses the serialization context to assemble the runtime command. TaskTemplate.container_args places the serialized input and output paths and version into the command, while other values remain Flyte expressions:

def container_args(self, serialize_context: SerializationContext) -> List[str]:
    args = [
        "a0",
        "--inputs",
        serialize_context.input_path,
        "--outputs-path",
        serialize_context.output_path,
        "--version",
        serialize_context.version,
        "--raw-data-path",
        "{{.rawOutputDataPrefix}}",
        "--checkpoint-path",
        "{{.checkpointOutputPrefix}}",
        "--prev-checkpoint",
        "{{.prevCheckpointPrefix}}",
        "--run-name",
        "{{.runName}}",
        "--name",
        "{{.actionName}}",
    ]

SerializationContext.get_entrypoint_path() derives the runtime entrypoint beside the selected interpreter rather than returning the interpreter path itself. With the default interpreter, it produces /opt/venv/bin/runtime.py; an explicit interpreter is used when supplied:

def get_entrypoint_path(self, interpreter_path: Optional[str] = None) -> str:
    if interpreter_path is None:
        interpreter_path = self.interpreter_path
    return os.path.join(os.path.dirname(interpreter_path), "runtime.py")

Consequently, SerializationContext is the appropriate object for serializer and task-extension hooks such as container_args, config, custom_config, data_loading_config, and sql. It describes what should be deployed and how the resulting container should start; it does not represent the action that will eventually execute that container.

Runtime installation of TaskContext

The runtime boundary occurs in _internal/runtime/taskrunner.py. convert_and_run receives concrete invocation values, constructs a TaskContext, and installs it with ctx.replace_task_context(tctx) before loading inputs and running the task:

tctx = TaskContext(
    action=action,
    checkpoints=checkpoints,
    code_bundle=code_bundle,
    input_path=input_path,
    output_path=output_path,
    run_base_dir=run_base_dir,
    version=version,
    raw_data_path=raw_data_path,
    compiled_image_cache=image_cache,
    report=flyte.report.Report(name=action.name),
    mode="remote" if not ctx.data.task_context else ctx.data.task_context.mode,
    interactive_mode=interactive_mode,
)
with ctx.replace_task_context(tctx):
    inputs = await load_inputs(input_path) if input_path else inputs
    inputs_kwargs = await convert_inputs_to_native(inputs, task.native_interface)
    out, err = await run_task(tctx=tctx, controller=controller, task=task, inputs=inputs_kwargs)

This code illustrates the different information available at execution time. The action and report are concrete, paths and checkpoints have been selected for this invocation, and the task is executed while the context is installed. The runner then flushes a task report when appropriate and converts native outputs after the task returns.

Remote and hybrid paths provide the execution metadata used to construct this object. The runtime entrypoint in _bin/runtime.py receives command-line values such as the action name, run name, input and output paths, version, raw-data path, and checkpoint paths, builds an ActionID, and passes those values into the runtime loading path. Local execution is visibly different: _run.py constructs a TaskContext with mode="local", code_bundle=None, compiled_image_cache=None, local metadata paths, and the sentinel version "na".

That mode is observable behavior, not merely descriptive metadata. The blocking map implementation checks the current context and runs every task sequentially when there is no context or when the mode is local:

import flyte

tctx = flyte.ctx()
if tctx is None or tctx.mode == "local":
    logger.warning("Running map in local mode, which will run every task sequentially.")
    for v in zip(*args):
        yield func(*v)
    return

TaskContext.is_in_cluster() is narrower than “not local”: it returns True only when mode == "remote". Therefore, hybrid mode returns False from is_in_cluster() even though hybrid execution can use remote storage or controllers. Logging code uses this method to choose its behavior, so consumers should not infer hybrid semantics from the method name.

Accessing and changing runtime state

User-facing task code obtains the installed runtime object through flyte.ctx(). _context.py returns the TaskContext stored in the current context variable and returns None when no task context has been installed:

def ctx() -> Optional[TaskContext]:
    """Retrieve the current task context from the context variable."""
    return internal_ctx().data.task_context

Code that requires execution state must handle that None result. For example, the local controller checks for an initialized task context before generating a sub-action identifier and output path; otherwise it raises NotInTaskContextError.

TaskContext is frozen, so callers do not mutate fields in place. Its replace method returns a new context. Replacing data has special merge behavior: the existing dictionary is copied and updated with the supplied mapping, while replace(data=None) leaves the existing data unchanged through dataclass replacement. The mapping-style accessor is a convenience over that dictionary and returns None for a missing key:

current = flyte.ctx()
if current is not None:
    value = current["some_task_value"]
    next_context = current.replace(data={"another_task_value": value})

The context machinery uses a contextvars.ContextVar, but the source documentation notes that arbitrary use is not coroutine-safe; callers should use contextual_run() when creating a new context tree. The runtime itself installs the task context with synchronous or asynchronous context managers, rather than treating it as process-global mutable state.

Nested tasks: serialization derived from execution state

The distinction also appears when a remote task launches a child task. The remote controller reads the parent TaskContext for values such as the parent version, compiled image cache, action identity, and code-bundle state. It then creates a new SerializationContext to serialize the child. After serialization, runtime conversion utilities use the parent TaskContext to derive the child action ID and output path.

This is why the two contexts can overlap without being the same object. The parent runtime context is the source of concrete execution state; the child serialization context is a new description of how the child should be represented and launched. The child still needs its own serialized task identity and command before its execution can establish its own runtime state.

Operational constraints and tradeoffs

Several details follow directly from this lifecycle split:

• A remote or hybrid run needs a run_base_dir; _run.py validates it and recommends configuring one with flyte.with_runcontext(run_base_dir="s3://bucket/metadata/outputs") when it is absent. That directory contributes to the concrete runtime paths in TaskContext.
• SerializationContext.input_path and output_path default to {{.input}} and {{.outputPrefix}}. Treating them as ordinary local paths before execution would confuse serialized templates with resolved runtime locations.
• Both contexts have a version, but serialization uses its value in the task identifier and container arguments, while runtime uses it as the version of the currently executing task/action. The comment in TaskTemplate.container_args explicitly leaves open whether the serialized argument should use serialize_context.version or code_bundle.version, so callers should not assume those sources are interchangeable.
• Serialization uses optional image and code-bundle state. Missing image-cache information can cause warnings or inconsistent image resolution, and the serializer warns that compiler and task-environment Flyte SDK versions should match.
• Outside a task invocation, flyte.ctx() is None; code that needs an action, paths, or checkpoints cannot assume that a TaskContext exists.

The practical rule is to use SerializationContext while producing the task specification and container command, and to use TaskContext only for state belonging to a concrete invocation. flyte-sdk’s run, deployment, serializer, controller, and runtime-runner paths preserve that boundary even where the fields necessarily overlap.