refactor: (GenAI) Reorganized Rag Samples (Group С) #12612

Thoughtseize1 · 2024-09-20T09:00:26Z

Description

Created new /generative_ai/rag folder.
I left the skip mark on tests but I fixed them all.

Region_tag	New file
generativeaionvertexai_rag_retrieval_query	retrieval_query_example.py
generativeaionvertexai_rag_generate_content	generate_content_example.py
generativeaionvertexai_rag_quickstart	quickstart_example.py
generativeaionvertexai_rag_import_files	import_files_example.py
generativeaionvertexai_rag_import_files_async	import_files_async_example.py
generativeaionvertexai_rag_get_file	get_file_example.py
generativeaionvertexai_rag_list_files	list_files_example.py
generativeaionvertexai_rag_delete_file	delete_file_example.py
generativeaionvertexai_rag_delete_corpus	delete_corpus_example.py
generativeaionvertexai_rag_create_corpus	create_corpus_example.py
generativeaionvertexai_rag_get_corpus	get_corpus_example.py
generativeaionvertexai_rag_list_corpora	list_corpora_example.py
generativeaionvertexai_rag_upload_file	upload_file_example.py

Actions after approval:

Delete /generative_ai/rag.py and /generative_ai/rag_test.py files
Rename generative_ai/rag/test_rag.py -> rag_test.py

❗If we want our tests to run successfully, we need to remove the skip marker and ensure that the necessary permissions are granted in the test environment.

Currently, we are catching the following error when tests are running:
google.auth.exceptions.RefreshError: ('invalid_scope: Invalid OAuth scope or ID token audience provided.', {'error': 'invalid_scope', 'error_description': 'Invalid OAuth scope or ID token audience provided.'})

Checklist

I have followed Sample Guidelines from AUTHORING_GUIDE.MD
README is updated to include all relevant information
Tests pass: nox -s py-3.9 (see Test Environment Setup)
Lint pass: nox -s lint (see Test Environment Setup)
These samples need a new API enabled in testing projects to pass (let us know which ones)
These samples need a new/updated env vars in testing projects set to pass (let us know which ones)
This sample adds a new sample directory, and I updated the CODEOWNERS file with the codeowners for this sample
This sample adds a new Product API, and I updated the Blunderbuss issue/PR auto-assigner with the codeowners for this sample
Please merge this PR for me once it is approved

snippet-bot · 2024-09-20T13:50:47Z

Here is the summary of changes.

You are about to add 13 region tags.

generative_ai/rag/create_corpus_example.py:27, tag generativeaionvertexai_rag_create_corpus
generative_ai/rag/delete_corpus_example.py:21, tag generativeaionvertexai_rag_delete_corpus
generative_ai/rag/delete_file_example.py:21, tag generativeaionvertexai_rag_delete_file
generative_ai/rag/generate_content_example.py:25, tag generativeaionvertexai_rag_generate_content
generative_ai/rag/get_corpus_example.py:23, tag generativeaionvertexai_rag_get_corpus
generative_ai/rag/get_file_example.py:23, tag generativeaionvertexai_rag_get_file
generative_ai/rag/import_files_async_example.py:28, tag generativeaionvertexai_rag_import_files_async
generative_ai/rag/import_files_example.py:27, tag generativeaionvertexai_rag_import_files
generative_ai/rag/list_corpora_example.py:23, tag generativeaionvertexai_rag_list_corpora
generative_ai/rag/list_files_example.py:23, tag generativeaionvertexai_rag_list_files
generative_ai/rag/quickstart_example.py:29, tag generativeaionvertexai_rag_quickstart
generative_ai/rag/retrieval_query_example.py:25, tag generativeaionvertexai_rag_retrieval_query
generative_ai/rag/upload_file_example.py:30, tag generativeaionvertexai_rag_upload_file

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

Refresh this comment

generative_ai/rag/create_corpus_example.py

holtskinner · 2024-09-20T15:36:02Z

generative_ai/rag/delete_corpus_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # corpus_name = "projects/{PROJECT_ID}/locations/us-central1/ragCorpora/{rag_corpus_id}"


Note: This can be either format:

projects/{project}/locations/{location}/ragCorpora/{rag_corpus}
or {rag_corpus}

holtskinner · 2024-09-20T15:37:48Z

generative_ai/rag/delete_file_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # file_name = "projects/{PROJECT_ID}/locations/us-central1/ragCorpora/{rag_corpus_id}/ragFiles/{rag_file_id}"


Note: this can be either a full file path or the user can pass a corpus id and the file id to the method.

From the library:

Args

name
Either a full RagFile resource name must be provided, or a RagCorpus name and a RagFile name must be provided. Format:

``projects/{project}/locations/{location}/ragCorpora/{rag_corpus}/ragFiles/{rag_file}`` or ``{rag_file}``.

corpus_name
If name is not a full resource name, an existing RagCorpus name must be provided. Format:

``projects/{project}/locations/{location}/ragCorpora/{rag_corpus}`` or ``{rag_corpus}``.

Create a tracker to re-check this @ #12658

holtskinner · 2024-09-20T15:38:40Z

generative_ai/rag/generate_content_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # corpus_name = "projects/{PROJECT_ID}/locations/us-central1/ragCorpora/{rag_corpus_id}"


Similar as before, this parameter can be the full path or just the corpus id

rag_corpus
A Rag corpus resource name or corpus id. Format:

``projects/{project}/locations/{location}/ragCorpora/{rag_corpus_id}`` or ``{rag_corpus_id}``.

Create a tracker to re-check this @ #12658

holtskinner · 2024-09-20T15:39:27Z

generative_ai/rag/import_files_async_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # corpus_name = "projects/{PROJECT_ID}/locations/us-central1/ragCorpora/{rag_corpus_id}"


Same as before re: full resource names

holtskinner · 2024-09-20T15:39:57Z

generative_ai/rag/import_files_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # corpus_name = "projects/{PROJECT_ID}/locations/us-central1/ragCorpora/{rag_corpus_id}"


Same as before re: full resource paths

holtskinner · 2024-09-20T15:40:11Z

generative_ai/rag/list_files_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # corpus_name = "projects/{PROJECT_ID}/locations/us-central1/ragCorpora/{rag_corpus_id}"


Same as before re: full resource paths

holtskinner · 2024-09-20T15:40:33Z

generative_ai/rag/retrieval_query_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # corpus_name = "projects/[PROJECT_ID]/locations/us-central1/ragCorpora/[rag_corpus_id]"


Same as before re: full resource paths

holtskinner · 2024-09-20T15:40:44Z

generative_ai/rag/upload_file_example.py

+
+    # TODO(developer): Update and un-comment below lines
+    # PROJECT_ID = "your-project-id"
+    # corpus_name = "projects/{PROJECT_ID}/locations/us-central1/ragCorpora/{rag_corpus_id}"


Same as before re: full resource names

Thoughtseize1 · 2024-09-23T15:26:05Z

Separated document with detailed description of issue.

Thoughtseize1 · 2024-09-25T16:39:22Z

@holtskinner PTAL

msampathkumar · 2024-10-01T12:47:46Z

Create a tracker to re-check the mentioned comment @ #12658

For now, proceeding with the migration work.

…orm#12612) * New Samples for Rag folder * Try to run test with created test_file.txt * Returned pytest.skip mark * Added type annotations for Python 3.8

New Samples for Rag folder

adf709d

product-auto-label bot added the samples Issues that are directly related to samples. label Sep 20, 2024

Thoughtseize1 added 3 commits September 20, 2024 12:56

Try to run test with created test_file.txt

3bd4f5a

Returned pytest.skip mark

76d6943

Added type annotations for Python 3.8

48d2536

Thoughtseize1 added the snippet-bot:force-run Force snippet-bot runs its logic label Sep 20, 2024

snippet-bot bot removed the snippet-bot:force-run Force snippet-bot runs its logic label Sep 20, 2024

Thoughtseize1 assigned msampathkumar Sep 20, 2024

Thoughtseize1 requested a review from msampathkumar September 20, 2024 14:02

Thoughtseize1 marked this pull request as ready for review September 20, 2024 14:03

Thoughtseize1 requested review from a team as code owners September 20, 2024 14:03

holtskinner approved these changes Sep 20, 2024

View reviewed changes

Thoughtseize1 requested a review from holtskinner September 23, 2024 15:13

msampathkumar mentioned this pull request Oct 1, 2024

Genearative AI: RAG sample's corpus_name can be either a full file path or the user can pass a corpus id and the file id to the method. #12658

Open

msampathkumar merged commit cbd4f4f into main Oct 1, 2024
14 checks passed

msampathkumar deleted the Reorganize-GenAI-Samples-Rag branch October 1, 2024 12:47

Thoughtseize1 mentioned this pull request Oct 8, 2024

refactor (generative-ai): Cleaning old samples - Group C #12680

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: (GenAI) Reorganized Rag Samples (Group С) #12612

refactor: (GenAI) Reorganized Rag Samples (Group С) #12612

Thoughtseize1 commented Sep 20, 2024 •

edited

Loading

snippet-bot bot commented Sep 20, 2024

holtskinner Sep 20, 2024

holtskinner Sep 20, 2024

msampathkumar Oct 1, 2024

holtskinner Sep 20, 2024

msampathkumar Oct 1, 2024

holtskinner Sep 20, 2024

holtskinner Sep 20, 2024

holtskinner Sep 20, 2024

holtskinner Sep 20, 2024

holtskinner Sep 20, 2024

Thoughtseize1 commented Sep 23, 2024

Thoughtseize1 commented Sep 25, 2024

msampathkumar commented Oct 1, 2024

refactor: (GenAI) Reorganized Rag Samples (Group С) #12612

refactor: (GenAI) Reorganized Rag Samples (Group С) #12612

Conversation

Thoughtseize1 commented Sep 20, 2024 • edited Loading

Description

Checklist

snippet-bot bot commented Sep 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Args

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Thoughtseize1 commented Sep 23, 2024

Thoughtseize1 commented Sep 25, 2024

msampathkumar commented Oct 1, 2024

Thoughtseize1 commented Sep 20, 2024 •

edited

Loading