-
Notifications
You must be signed in to change notification settings - Fork 18
Update README and add Orion Document Understanding notebook #61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Summary of ChangesHello @shahrear33, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request expands the VLM Run Cookbook by introducing a new example notebook focused on Orion's document understanding features, providing users with a practical demonstration of extracting structured data from documents. Concurrently, the main README file has been updated to reflect this new addition and enhance its overall readability through minor formatting adjustments. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request adds a new 'Orion Document Understanding' notebook to the README.md and makes some formatting improvements. My review focuses on improving the structure and correctness of the notebook table. I've suggested reordering the notebooks for better readability and fixing a potential typo in the notebook numbering.
| | [API Quickstart](./notebooks/00_quickstart.ipynb) | | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/00_quickstart.ipynb) | 02-08-2025 | | ||
| | [Schema Showcase](./notebooks/01_schema_showcase.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/01_schema_showcase.ipynb) | 02-08-2025 | | ||
| | [Visual Grounding](./notebooks/04_visual_grounding.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/04_visual_grounding.ipynb) | 02-18-2025 | | ||
| | [Long-form Video Transcription](./notebooks/advanced_video_transcription.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/advanced_video_transcription.ipynb) | 03-13-2025 | | ||
| | [Video Inference (Fine-Tuning)](./notebooks/advanced_finetuning_video_inference.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/advanced_finetuning_video_inference.ipynb) | 02-18-2025 | | ||
| | [US Drivers License](./notebooks/02_case_study_drivers_license.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/02_case_study_drivers_license.ipynb) | 02-08-2025 | | ||
| | [Parsing Financial Presentations](https://colab.research.google.com/drive/15_iRDucKj2I33p3m5X3ULdXby_DHWgjS) | <kbd>application</kbd> | [](https://colab.research.google.com/drive/15_iRDucKj2I33p3m5X3ULdXby_DHWgjS) | 02-04-2025 | | ||
| | [TV News Analysis](./notebooks/03_case_study_tv_news.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/03_case_study_tv_news.ipynb) | 02-15-2025 | | ||
| | [Fashion Product Catalog](./notebooks/05_case_study_image_catalogue.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/05_case_study_image_catalogue.ipynb) | 02-20-2025 | | ||
| | [Fashion Images Hybrid Search](./notebooks/06_fashion_images_hybrid_search.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/06_fashion_images_hybrid_search.ipynb) | 02-21-2025 | | ||
| | [Generate Custom Schema](./notebooks/07_generate_schema.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/07_generate_schema.ipynb) | 03-13-2025 | | ||
| | [Document Markdown Extraction](./notebooks/08_document_markdown.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/08_document_markdown.ipynb) | 06-02-2025 | | ||
| | [Video RAG](./notebooks/11_video_rag.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/11_video_rag.ipynb) | 07-14-2025 | | ||
| | [Orion Image Understanding](./notebooks/12_orion_image_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/12_orion_image_understanding.ipynb) | 12-01-2025 | | ||
| | [Orion Video Understanding](./notebooks/12_orion_video_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/12_orion_video_understanding.ipynb) | 12-02-2025 | | ||
| | [Orion Document Understanding](./notebooks/14_orion_document_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/14_orion_document_understanding.ipynb) | 12-03-2025 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The table of notebooks seems to have grown organically and the order is a bit confusing. Since you're modifying it, this is a good opportunity to reorder the entries for better readability. I'd suggest grouping them by Type (feature, application, showcase) and then sorting them numerically. I've also noticed a duplicate notebook number 12 for 'Orion Image Understanding' and 'Orion Video Understanding'. I've corrected the latter to 13 assuming it's a typo, which makes sense with your addition of notebook 14.
| | [API Quickstart](./notebooks/00_quickstart.ipynb) | | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/00_quickstart.ipynb) | 02-08-2025 | | |
| | [Schema Showcase](./notebooks/01_schema_showcase.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/01_schema_showcase.ipynb) | 02-08-2025 | | |
| | [Visual Grounding](./notebooks/04_visual_grounding.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/04_visual_grounding.ipynb) | 02-18-2025 | | |
| | [Long-form Video Transcription](./notebooks/advanced_video_transcription.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/advanced_video_transcription.ipynb) | 03-13-2025 | | |
| | [Video Inference (Fine-Tuning)](./notebooks/advanced_finetuning_video_inference.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/advanced_finetuning_video_inference.ipynb) | 02-18-2025 | | |
| | [US Drivers License](./notebooks/02_case_study_drivers_license.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/02_case_study_drivers_license.ipynb) | 02-08-2025 | | |
| | [Parsing Financial Presentations](https://colab.research.google.com/drive/15_iRDucKj2I33p3m5X3ULdXby_DHWgjS) | <kbd>application</kbd> | [](https://colab.research.google.com/drive/15_iRDucKj2I33p3m5X3ULdXby_DHWgjS) | 02-04-2025 | | |
| | [TV News Analysis](./notebooks/03_case_study_tv_news.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/03_case_study_tv_news.ipynb) | 02-15-2025 | | |
| | [Fashion Product Catalog](./notebooks/05_case_study_image_catalogue.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/05_case_study_image_catalogue.ipynb) | 02-20-2025 | | |
| | [Fashion Images Hybrid Search](./notebooks/06_fashion_images_hybrid_search.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/06_fashion_images_hybrid_search.ipynb) | 02-21-2025 | | |
| | [Generate Custom Schema](./notebooks/07_generate_schema.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/07_generate_schema.ipynb) | 03-13-2025 | | |
| | [Document Markdown Extraction](./notebooks/08_document_markdown.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/08_document_markdown.ipynb) | 06-02-2025 | | |
| | [Video RAG](./notebooks/11_video_rag.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/11_video_rag.ipynb) | 07-14-2025 | | |
| | [Orion Image Understanding](./notebooks/12_orion_image_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/12_orion_image_understanding.ipynb) | 12-01-2025 | | |
| | [Orion Video Understanding](./notebooks/12_orion_video_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/12_orion_video_understanding.ipynb) | 12-02-2025 | | |
| | [Orion Document Understanding](./notebooks/14_orion_document_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/14_orion_document_understanding.ipynb) | 12-03-2025 | | |
| | [API Quickstart](./notebooks/00_quickstart.ipynb) | | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/00_quickstart.ipynb) | 02-08-2025 | | |
| | [Schema Showcase](./notebooks/01_schema_showcase.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/01_schema_showcase.ipynb) | 02-08-2025 | | |
| | [Visual Grounding](./notebooks/04_visual_grounding.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/04_visual_grounding.ipynb) | 02-18-2025 | | |
| | [Generate Custom Schema](./notebooks/07_generate_schema.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/07_generate_schema.ipynb) | 03-13-2025 | | |
| | [Document Markdown Extraction](./notebooks/08_document_markdown.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/08_document_markdown.ipynb) | 06-02-2025 | | |
| | [Long-form Video Transcription](./notebooks/advanced_video_transcription.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/advanced_video_transcription.ipynb) | 03-13-2025 | | |
| | [Video Inference (Fine-Tuning)](./notebooks/advanced_finetuning_video_inference.ipynb) | <kbd>feature</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/advanced_finetuning_video_inference.ipynb) | 02-18-2025 | | |
| | [US Drivers License](./notebooks/02_case_study_drivers_license.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/02_case_study_drivers_license.ipynb) | 02-08-2025 | | |
| | [TV News Analysis](./notebooks/03_case_study_tv_news.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/03_case_study_tv_news.ipynb) | 02-15-2025 | | |
| | [Fashion Product Catalog](./notebooks/05_case_study_image_catalogue.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/05_case_study_image_catalogue.ipynb) | 02-20-2025 | | |
| | [Fashion Images Hybrid Search](./notebooks/06_fashion_images_hybrid_search.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/06_fashion_images_hybrid_search.ipynb) | 02-21-2025 | | |
| | [Video RAG](./notebooks/11_video_rag.ipynb) | <kbd>application</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/11_video_rag.ipynb) | 07-14-2025 | | |
| | [Parsing Financial Presentations](https://colab.research.google.com/drive/15_iRDucKj2I33p3m5X3ULdXby_DHWgjS) | <kbd>application</kbd> | [](https://colab.research.google.com/drive/15_iRDucKj2I33p3m5X3ULdXby_DHWgjS) | 02-04-2025 | | |
| | [Orion Image Understanding](./notebooks/12_orion_image_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/12_orion_image_understanding.ipynb) | 12-01-2025 | | |
| | [Orion Video Understanding](./notebooks/13_orion_video_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/13_orion_video_understanding.ipynb) | 12-02-2025 | | |
| | [Orion Document Understanding](./notebooks/14_orion_document_understanding.ipynb) | <kbd>showcase</kbd> | [](https://colab.research.google.com/github/vlm-run/vlmrun-cookbook/blob/main/notebooks/14_orion_document_understanding.ipynb) | 12-03-2025 | |
No description provided.