Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'ComputeRecurrenceSchedule' object has no attribute '_to_rest_compute_pattern_object' #39744

Open
ABrethome opened this issue Feb 14, 2025 · 1 comment
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@ABrethome
Copy link

ABrethome commented Feb 14, 2025

  • Package Name: azure-ai-ml
  • Package Version: 1.25.0
  • Operating System: Linux abretho-sbx-dsteam-c02 5.15.0-1040-azure 47~20.04.1-Ubuntu SMP Fri Jun 2 21:38:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux (generated from uname -a)
  • Python Version: 3.11.11

Describe the bug
First, I create a compute instance on behalf of a user using the MLClient begin_create_or_update() with specific compute instance configuration (script, schedule, etc.).
Second, I retrieve the created compute object from the poller of begin_create_or_update(), then I update the compute with new parameters (in my case, enable SSO after creation, as I can't enable SSO during creation).
When I run the update of the compute instance with a second begin_create_or_update(), I get the following error:

AttributeError: 'ComputeRecurrenceSchedule' object has no attribute '_to_rest_compute_pattern_object'

To Reproduce
Steps to reproduce the behavior:

  1. Create compute instance object:
    # AssignedUserConfiguration is the class we need to pass
    # when creating compute to tell AML who to assign the compute to
    assigned_user_config = AssignedUserConfiguration(
        user_tenant_id=<MY_USER_TENANT_ID>, user_object_id=<MY_USER_OID>
    )

    # Create shutdown schedule
    # This function is not very well documented, especially what the parameter `start_time` does - from
    # limited trial/error so far it seems to be required, but doesn't actually have an effect on the schedule.
    # The RecurrencePattern is what sets the timing of the schedule.
    stop_schedule = ComputeStartStopSchedule(
        trigger=RecurrenceTrigger(
            start_time=datetime.now().isoformat(
                timespec="seconds"
            ),  # Required with timezone. str with ISO 8601 date format. Naive datetime.
            time_zone=<MY_USER_TIMEZONE>,  # Required with schedule. str or TimeZone
            frequency="day",  # str, e.g. "minute", "hour", "day", "week", "month"
            interval=1,
            schedule=RecurrencePattern(
                hours=20, minutes=0
            ),  # Specifies shutdown at 20:00 each day
        ),
        action=ComputePowerAction.STOP,
        state=ScheduleState.ENABLED,
    )
    compute_schedule = ComputeSchedules(compute_start_stop=[stop_schedule])

    # Configure the setup script
    setup_scripts = SetupScripts(
        creation_script=ScriptReference(
            path=<MY_USER_SCRIPT_PATH>,  # path within the workspace where the compute will be created.
            timeout_minutes=25,  # 25min is the maximum. Any higher time limit will throw an error.
        )
    )

    # Configure the compute
    new_compute = ComputeInstance(
        name=computename,
        description=f"Compute instance created on behalf of {user.name}.",
        size=compute_size.name,  # str
        create_on_behalf_of=assigned_user_config,
        # NetworkSettings doc is bad. Look at code directly (CTRL+F NetworkSettings)
        # https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ml/azure-ai-ml/azure/ai/ml/entities/_compute/compute.py
        # provide the full URI for subnet and leave vnet_name as None.
        network_settings=NetworkSettings(
            vnet_name=None, subnet=workspace.get_full_uri_subnet()
        ),
        ssh_public_access_enabled=False,
        schedules=compute_schedule,
        idle_time_before_shutdown_minutes=65,
        enable_node_public_ip=False,
        setup_scripts=setup_scripts,
        enable_sso=False,  # By design, AzureML does not allow to enable SSO while creating a compute on behalf of a user
        enable_root_access=True,
    )

ComputeInstance object is successfully init.

  1. Create the compute with the MLClient:
    # create the compute instance
    poller = ml_client.begin_create_or_update(new_compute)

    # wait for the operation to complete, returns the created ComputeInstance object
    created_compute: ComputeInstance = poller.result()

    # Check the status of the operation
    if created_compute.provisioning_state != "Succeeded":
        message = f"Compute instance creation of '{computename}' failed. See compute instance in AzureML workspace for more details."
        raise ComputeError(message)

Compute is successfully created.

  1. Try enabling SSO (for instance) afterwards:
    # update sso to true (cannot be done at compute creation)
    created_compute.enable_sso = True

    # update the compute instance
    poller = ml_client.begin_create_or_update(created_compute)

    # wait for the operation to complete, returns the created ComputeInstance object
    updated_compute: ComputeInstance = poller.result()

Throws error:
AttributeError: 'ComputeRecurrenceSchedule' object has no attribute '_to_rest_compute_pattern_object'

This confuses me, as the error is related to the schedule, which I didn't touch at that stage.

  1. Note: this doesn't work either:
   created_compute = ml_client.compute.get(name=created_compute.name)

    # update sso to true (cannot be done at compute creation)
    created_compute.enable_sso = True

    # update the compute instance
    poller = ml_client.begin_create_or_update(created_compute)

    # wait for the operation to complete, returns the created ComputeInstance object
    updated_compute: ComputeInstance = poller.result()

Throws error:
AttributeError: 'ComputeRecurrenceSchedule' object has no attribute '_to_rest_compute_pattern_object'

This confuses me, as the error is related to the schedule, which I didn't touch at that stage.

Expected behavior
I expect to be able to get a Compute(Instance) object returned from the poller or the compute operations, modify a parameter, then update compute with begin_create_or_update() without having errors.

Screenshots
Not applicable.

Additional context

  1. Full error:
  File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/abretho-sbx-dsteam-c02/code/Users/RA_ABRETHO/Git/AML_Workspace_Management/src/inf_aml_mgt/computes.py", line 275, in update_compute_instance_sso
    poller = ml_client.begin_create_or_update(created_compute)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/_ml_client.py", line 1292, in begin_create_or_update
    return _begin_create_or_update(entity, self._operation_container.all_operations, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/functools.py", line 909, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/_ml_client.py", line 1386, in _
    return operations[AzureMLResourceType.COMPUTE].begin_create_or_update(entity, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 116, in wrapper_use_tracer
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/_telemetry/activity.py", line 288, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/operations/_compute_operations.py", line 182, in begin_create_or_update
    compute_rest_obj = compute._to_rest_object()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/entities/_compute/compute_instance.py", line 322, in _to_rest_object
    compute_instance_prop.schedules = self.schedules._to_rest_object() if self.schedules else None
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/entities/_compute/_schedule.py", line 138, in _to_rest_object
    rest_schedules.append(schedule._to_rest_object())
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/entities/_compute/_schedule.py", line 82, in _to_rest_object
    rest_object.recurrence = self.trigger._to_rest_compute_recurrence_object()
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anaconda/envs/inf-ai-mgt-dev/lib/python3.11/site-packages/azure/ai/ml/entities/_schedule/trigger.py", line 276, in _to_rest_compute_recurrence_object
    schedule=self.schedule._to_rest_compute_pattern_object(),
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ComputeRecurrenceSchedule' object has no attribute '_to_rest_compute_pattern_object'
  1. Tactical fix by looking at the doc:
    Taken from: https://learn.microsoft.com/en-us/python/api/azure-ai-ml/azure.ai.ml.operations.computeoperations?view=azure-python#azure-ai-ml-operations-computeoperations-enable-sso
    ml_client.compute.enable_sso(
        name=created_compute.name,
        enable_sso=True
    )

This is experimental though, so not really keen on relying on this long term.

@github-actions github-actions bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Feb 14, 2025
@kristapratico kristapratico added bug This issue requires a change to an existing behavior in the product in order to be resolved. Machine Learning Service Attention Workflow: This issue is responsible by Azure service team. Client This issue points to a problem in the data-plane of the library. and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Feb 14, 2025
@github-actions github-actions bot added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Feb 14, 2025
Copy link

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/azure-ml-sdk @azureml-github.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

2 participants