Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Airlock Manager Not Receiving Requests #4285

Open
alanmegaw77 opened this issue Jan 28, 2025 · 38 comments
Open

Airlock Manager Not Receiving Requests #4285

alanmegaw77 opened this issue Jan 28, 2025 · 38 comments

Comments

@alanmegaw77
Copy link

alanmegaw77 commented Jan 28, 2025

Hi,

I am airlock manager within my workspace, but 'awaiting my review' is empty:

Image

I am assigned 'Airlock manager'

Submitted Requests:

Many thanks

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

Can you click clear filters? Looks like there is a filter on the status column.

If that doesn't work, please check using the workspace API to see if the API returns the correct list of requests.

@alanmegaw77
Copy link
Author

alanmegaw77 commented Jan 28, 2025

Hi Marcus,

Thanks for responding. Clicking on clear filters I can see the request, but cannot action anything:

Could you advise which workspace API to try, I see the following:

Image

Alan

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

Have you signed out and back in after grating yourself the role? The roles exist in the token issued at login.

If scroll down the API list you should see airlock.

Also - if you have an account contact in Microsoft, are you able to get them to drop me a message, keen to ensure we can work out how best to support you to be successful. Thanks.

@alanmegaw77
Copy link
Author

Hi,

Yes, signed out and in several times.

I don't see anything on the API list for 'airlock'

Thanks,
Alan

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

Should be /api/workspaces/{workspace_id}/requests

It looks like the requests are showing submitted, rather than in review. See: https://microsoft.github.io/AzureTRE/v0.19.1/azure-tre-overview/airlock/#ingressegress-mechanism

This leads us to the airlock processor, can you take a look at the function execution logs, here - https://microsoft.github.io/AzureTRE/v0.19.1/troubleshooting-faq/airlock-troubleshooting/#airlock-request-does-not-move-through-the-workflow-as-expected

(note green tick, might still have errors in the logs).

@alanmegaw77
Copy link
Author

thanks Marcus, I will look into this, could i ask you remove the airlock graphic in your response please

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

Ah, apologies, see you have edited the original post. The image will still exist in the comment history. Will see if is a way to remove it.

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

You need to do this:

Image

@alanmegaw77
Copy link
Author

Hi Marcus,

I looked at the function logs:

Image

and went into invocations for each of the logs. For the imports showing as 'submitted' no errors at all

Many thanks,
Alan

@alanmegaw77
Copy link
Author

I also tried another browser / incognito to rule out any caching.

Is there any further configuration required to enable the import to move from submitted > 'in review' ? I noted this in documentation you provided:

The request is ready to be reviewed. This state can be reached directly from Submitted state or after going through a successful security scan (found clean).

Do we need to configure the security scan manually?

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

What does the API return as status for these requests? Submitted or In review?

Can you also check the API logs as per - https://microsoft.github.io/AzureTRE/v0.19.1/troubleshooting-faq/app-insights-logs/

For some reason the status isn't changing.

@LizaShak @tamirkamara any ideas why these would stick in submitted?

@SvenAelterman
Copy link
Collaborator

Do we need to configure the security scan manually?

In your config.yaml for the TRE deployment, what's the value of enable_airlock_malware_scanning?

@alanmegaw77
Copy link
Author

Hi Sven,

Set = true

Kind regards,
Alan

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

Good shout. You should see something like:

Image

In the airlock processor function, ScanResultTrigger invocation logs. Can you check?

If you don't see anything. It might be worth setting to false, run make deploy-core and try again to rule this out.

@alanmegaw77
Copy link
Author

Image

@SvenAelterman
Copy link
Collaborator

Just to be clear, there were no errors during the deployment, right?

Can you check the Defender for Storage status on the storage accounts?

I agree with Marcus' suggestion of turning it off and running the deployment again and see what happens with a new request.

@alanmegaw77
Copy link
Author

Hi Sven,

Defender for storage off for all storage accounts.

Any other logs / configurations to check?

Many thanks,
Alan

@SvenAelterman
Copy link
Collaborator

Likely that's where your problem lies... the malware scanning is based on the Defender for Storage capabilities. The TRE deployment should have enabled it, so if it isn't, then perhaps there's someone or some policy that turned it off?

@marrobi
Copy link
Member

marrobi commented Jan 28, 2025

@alanmegaw77

I agree with Marcus' suggestion of turning it off and running the deployment again and see what happens with a new request.

Can you give this a go? So we are sure?

@marrobi
Copy link
Member

marrobi commented Jan 29, 2025

the other thing to check is the service bus queue: airlock-status-changed (sb-<tre-id>/airlock-status-changed)

Look at the message counts:

Image

@alanmegaw77
Copy link
Author

alanmegaw77 commented Jan 29, 2025

Hi @SvenAelterman @marrobi , appreciate the ongoing help with this issue.

Message Counts:

Image

Just to confirm the suggested actions are:

  1. config.yaml > enable_airlock_malware_scanning = false > make deploy-core
  2. config.yaml > enable_airlock_malware_scanning = true > make deploy-core

(essentially turning malware scanning off and on)

Should we check an upload between 1 & 2 - presumably this would fail though because the imports need to be scanned before they can move from submitted > awaiting review?

Last - is there a log from the initial core deployment that we can check for errors (in case other items were missed) before redeploying the core?

Many thanks,
Alan

@marrobi
Copy link
Member

marrobi commented Jan 29, 2025

Just do 1 for now, and test. If malware scanning is not enabled it should automatically progress to the next stage without a scan.

If you deployed locally there will be some log files in core/terraform

@alanmegaw77
Copy link
Author

Hi @marrobi ,

Disabling malware scanning has enabled us to progress to review - many thanks, however we are trying to enable auto review creation and are struggling finding the correct values for the highlighted:

Image

@marrobi
Copy link
Member

marrobi commented Jan 29, 2025

@alanmegaw77
Copy link
Author

hi Marcus,

we have looked at that link, but are still unsure where to get the x3 ID's.

-for the first (Import Review Workspace ID) - we used the resource ID of our (base) workspace, but I don't think this is correct?

Additionally, I'm not sure where exactly to check for the 'Airlock Import Review workspace' / obtain the GUID. As the other items require the Airlock Import Review workspace I assume this would be the first item to check?

@marrobi
Copy link
Member

marrobi commented Jan 29, 2025

Need to create a review workspace:

Image

Can get the GUID here:

Image

The workspace service ID is the ID of the virtual desktop service in the import review workspace for import, or the project workspace for export.

@alanmegaw77
Copy link
Author

Thanks Marcus,

we registered the airlock import templates via - https://microsoft.github.io/AzureTRE/latest/tre-admins/registering-templates/

however when creating 'Airlock Import Review Workspace' we get the following error:

Image

@SvenAelterman
Copy link
Collaborator

Are you using version 0.19.1 or a more recent commit?

@alanmegaw77
Copy link
Author

Hi @SvenAelterman,

yes 0.19.1

Image

name: tre-workspace-airlock-import-review
version: 0.14.1

@marrobi
Copy link
Member

marrobi commented Jan 30, 2025

That is a really odd error, azure_location is picked up from the API environment variables.

Can you just check api-<tre_id> environment variables and the RESOURCE_LOCATION value:

Image

@alanmegaw77
Copy link
Author

hi @marrobi

Image

@alanmegaw77
Copy link
Author

I just retried creating again, but this time I got a different error:

Image

@marrobi
Copy link
Member

marrobi commented Jan 30, 2025

Hi @SvenAelterman,

yes 0.19.1

Image

name: tre-workspace-airlock-import-review version: 0.14.1

Ok, Looking at the releases, that's a newer version of the airlock review template than was tested with 0.19.1?

Image

Can you confirm that you are on 0.19.1 or deploying from main?

I am going to test from main as I'm on an older version of the airlock import review template.

@marrobi
Copy link
Member

marrobi commented Jan 30, 2025

I've just deployed 0.14.1 using the main branch of the repo and it worked.

@alanmegaw77
Copy link
Author

apologies @marrobi , deploying from main I believe, the files I am working with were downloaded from Github on 23/01/2025

@SvenAelterman
Copy link
Collaborator

Are you deploying with the tooling provided in the AzureTRE-Deployment repo?

@alanmegaw77
Copy link
Author

AzureTRE-Deployment repo

no @SvenAelterman, I downloaded files from - https://github.com/microsoft/AzureTRE

@SvenAelterman
Copy link
Collaborator

That might have put you in some indeterminate state between releases.

I recommend following the steps in the Quick Start guide here: https://microsoft.github.io/AzureTRE/latest/tre-admins/setup-instructions/prerequisites/.

I will let others provide guidance on whether it's best to start over or if the existing resources could be fixed. I would recommend the former, i.e., remove all resources and Entra objects and start over. Change the TRE ID (add a sequence number perhaps) to avoid issues due to recovering Key Vault from purge protection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants