Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error Download via UI for Azure Blob Storage #247

Open
muhammadAgfian96 opened this issue Jul 1, 2024 · 1 comment
Open

Error Download via UI for Azure Blob Storage #247

muhammadAgfian96 opened this issue Jul 1, 2024 · 1 comment

Comments

@muhammadAgfian96
Copy link

muhammadAgfian96 commented Jul 1, 2024

Hi, I tried to migrate our storage from S3 to Azure. First, I tried to upload artifacts with Azure as the storage. The upload was successful, but it failed to download via the UI.

Upload Artifacts

import os
from time import sleep

from clearml import Task
from clearml.backend_config.bucket_config import AzureContainerConfig
from clearml.storage.helper import StorageHelper


def get_storage_uri() -> str:
    # Set up logging to Azure
    bucket_config = AzureContainerConfig(
        account_name=os.environ["AZURE_STORAGE_ACCOUNT"],  # type: ignore
        account_key=os.environ["AZURE_STORAGE_KEY"],  # type: ignore
        container_name="zzzzzzzz",  # type: ignore
    )
    StorageHelper.add_azure_configuration(bucket_config)
    return StorageHelper.get_azure_storage_uri_from_config(bucket_config)


storage_uri = get_storage_uri()
print(storage_uri)
task1: Task = Task.init(
    project_name="playground/boy",
    task_name="Create artifact Example",
    task_type=Task.TaskTypes.testing,
    output_uri=storage_uri + "/testing",

# upload data file to the initialized task, inputting a name and file location
task1.upload_artifact(
    name="data file",
    artifact_object="assets/image.jpg",
    metadata={
        "key": "hoi",
    },
    wait_on_upload=True,
)

# close the task, to be able to initialize a new task
task1.close()

it success upload 🎉
image

Failed to download Via UI

But when trying to download the file via the UI, the blob is not found.
image
i guess its produce not correct link: https://xxxxxxxxx.blob.core.windows.net/zzzzzzzz/testing/playground/boy/Create%20artifact%20Example.fb504e00f3524295ab8342c5ea1adb24/artifacts/data%20file/image.jpgsp=r&st=2024-06-30T17:38:17Z&se=2024-07-01T01:38:17Z&spr=https&sv=2022-11-02&sr=c&sig=XXXXXXXXXXXXX

There should be a '?' between the Azure Blob URL and the SAS token, so: URL_AZURE_BLOB + '?' + SAS_TOKEN, i think?

..../artifacts/data%20file/image.jpgsp=r&st=2024.....
                                   ^^^ here should be there a "?"

# so it will be:
..../artifacts/data%20file/image.jpg?sp=r&st=2024.....
                                    ^

ClearML Version

WebApp: 1.15.1-478 • Server: 1.15.1-478 • API: 2.29
Thanks in Advance!

@ainoam
Copy link
Collaborator

ainoam commented Jul 1, 2024

Thanks for reporting @muhammadAgfian96 - Hope to be able to provide a fix soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants