Skip to content

Conversation

@ianpittwood
Copy link
Contributor

@ianpittwood ianpittwood commented Aug 18, 2025

  • Dependency updates
    • Upgrade typer from 0.12.5 to 0.16.1 for compatibility with latest click distribution.
    • Upgrade rich from 13.8.1 to 14.1.0.
    • Patches for jinja2 and gitpython.
    • Remove packaging as a strict requirement. It seemed unnecessary and inconvenient to explicitly pin.
    • Upgrade and loose pin all development packages. Also upgrade the ruff and shellcheck versions used by pre-commit.
  • CLI changes
    • Switch all logic from old models to new single YAML config models.
    • Remove run snyk command.
    • build
      • Remove --builder in favor of letting the user change the builder themselves ahead of time, at least for now. This was more because it was confusing to implement in Python on Whales, particularly with the addition of sequential builds.
      • Add --fail-fast option for stopping on first failure for sequential builds.
      • The default for --load has been changed to True. I find it's exceedingly rare that I would not want to load images after building.
    • create
      • Add --subpath to create commands to specify a subpath different from a version or image name.
      • Add --display-name, --description, and --documentation-url to create image to specify those labeling values ahead of time.
  • Model changes
    • General
      • Extract common regex patterns to constants.
    • BakeryConfigDocument
      • Add support for supplying display_name, description, and documentation_url for create_image_model method.
      • Fix Containerfile naming patterns for creating default image templates. It now correctly condenses the base image tag (e.g. docker.io/library/ubuntu:22.04 creates Containerfile.ubuntu2204.jinja2).
    • BakeryConfig
      • Fix output indentation and quote preservation with YAML.
      • Log an error if the config fails to load.
      • Add from_context method to discover a bakery.yaml or bakery.yml file in the given path to load a project. This is a shortcut for the CLI.
      • Add helper functions to retrieve indices for existing images and versions in the YAML file.
      • Add support for supplying display_name, description, and documentation_url for create_image method.
      • Fix create_version logic.
        • Correctly update latest flag on other versions if given version is marked latest.
        • Fix handling of updating versus adding new versions in YAML document.
        • Sort version output in descending order.
      • Add bake_plan_targets function to return plan JSON without building.
      • Implement "fail fast" logic for sequential builds.
    • Image
      • Fix serialization of documentationUrl to return a string instead of an object representation.
      • Fix VersionPath in templating values to always return a relative path regardless of which value is used.
      • Fix missing latest logic in create_version_model.
    • GossOptions
      • Simplify update logic.
      • Add runtimeOptions field for supporting images that require special run options, such as Connect with --privileged.
    • ImageTarget
      • Add remove method to delete image builds. This is not yet used or exposed.
      • Update build method.
        • Use a context manager for changing directory for builds. This ensures the directory is changed back on errors.
        • Improve error handling.
  • Other changes
    • Default logging traceback frames to 0 and to 20 with show_locals enabled when verbose.
    • Removal of lots of old and/or unused code.
    • Updates for tests.

@ianpittwood ianpittwood changed the base branch from main to single-yaml-config-functional August 18, 2025 18:58
@github-actions
Copy link

github-actions bot commented Aug 18, 2025

Test Results

196 tests   196 ✅  1m 15s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 9747868.

♻️ This comment has been updated with latest results.

@ianpittwood ianpittwood force-pushed the single-yaml-config-cli branch from 344f2b3 to a38302e Compare August 20, 2025 19:33
@ianpittwood ianpittwood marked this pull request as ready for review August 20, 2025 19:34
Base automatically changed from single-yaml-config-functional to main August 21, 2025 13:04
@posit-dev posit-dev deleted a comment from ianpittwood Aug 21, 2025
Copy link
Contributor

@bschwedler bschwedler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for doing this! I really like these changes.

I want to play with the CLI a bit before approving the PR.

Comment on lines +258 to +259
self.yaml.preserve_quotes = True
self.yaml.indent(mapping=2, sequence=4, offset=2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💜

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just so you know, this is causing some weird behavior. Like the quoting has been inconsistent for me when dumping models. For instance it quotes subpath, but not name for images and versions like this:

- name: 1.0.0
  subpath: '1.0'

I'm not sure why that is though. They're both strings and I haven't spotted anything that would force quoting. 🤷‍♂️

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also force writing of the YAML file to always include the trailing newline at the end of the file?

Copy link
Contributor Author

@ianpittwood ianpittwood Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ruamel.yaml cannot, but I could do it manually other ways.

I was wrong, it does so automatically.

Copy link
Contributor

@bschwedler bschwedler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some basic comments on the CLI functionality. I didn't do an in-depth test, just the basics

  • Should we return with a non-zero exit code calling bakery create project if one already exists?
  • --subpath flag
    • Only appears to support creating directories one leve deep.
    • Does not seeom to be respected when calling bakery create version
  • Should bakery help do the same thing as bakery --help?
  • bakery create version appears to try to validate whether there are images even when creating the first version

I'll come back and address adding versions when the second failure below is resolved.

Failures

± bakery create image --base-image ubuntu:24.04 --subpath fancy/take-3 fancy-3                                     1 ↵ 
[11:20:50] INFO     Loading Bakery config from                                                             config.py:293
                    /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/brjs_test/bakery.yaml                   
           WARNING  No versions found in image 'fancy-1'. At least one version is required for most         image.py:444
                    commands.                                                                                           
           WARNING  No versions found in image 'fancy-2'. At least one version is required for most         image.py:444
                    commands.                                                                                           
           WARNING  No versions found in image 'fancy-3'. At least one version is required for most         image.py:444
                    commands.                                                                                           
           ERROR    Error creating image                                                                    create.py:80
                    ╭───────────────────────── Traceback (most recent call last) ─────────────────────────╮             
                    │ /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/posit_bakery/cli/create │             
                    │ .py:71 in image                                                                     │             
                    │                                                                                     │             
                    │    68 │   """                                                                       │             
                    │    69 │   try:                                                                      │             
                    │    70 │   │   c = BakeryConfig.from_context(context)                                │             
                    │ ❱  71 │   │   c.create_image(                                                       │             
                    │    72 │   │   │   image_name,                                                       │             
                    │    73 │   │   │   base_image=base_image,                                            │             
                    │    74 │   │   │   subpath=subpath,                                                  │             
                    │                                                                                     │             
                    │ /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/posit_bakery/config/con │             
                    │ fig.py:373 in create_image                                                          │             
                    │                                                                                     │             
                    │   370 │   │   │   description=description,                                          │             
                    │   371 │   │   │   documentation_url=documentation_url,                              │             
                    │   372 │   │   )                                                                     │             
                    │ ❱ 373 │   │   self.model.create_image_files_template(new_image.path, new_image.name │             
                    │       base_image or DEFAULT_BASE_IMAGE)                                             │             
                    │   374 │   │   new_image_dict = new_image.model_dump(exclude_defaults=True, exclude_ │             
                    │       exclude_unset=True)                                                           │             
                    │   375 │   │   self._config_yaml.setdefault("images", []).append(new_image_dict)     │             
                    │   376 │   │   self.write()                                                          │             
                    │                                                                                     │             
                    │ /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/posit_bakery/config/con │             
                    │ fig.py:146 in create_image_files_template                                           │             
                    │                                                                                     │             
                    │   143 │   │   exists: bool = image_path.is_dir()                                    │             
                    │   144 │   │   if not exists:                                                        │             
                    │   145 │   │   │   log.debug(f"Creating new image directory [bold]{image_path}")     │             
                    │ ❱ 146 │   │   │   image_path.mkdir()                                                │             
                    │   147 │   │                                                                         │             
                    │   148 │   │   image_template_path = image_path / "template"                         │             
                    │   149 │   │   if not image_template_path.is_dir():                                  │             
                    │                                                                                     │             
                    │ /usr/lib64/python3.13/pathlib/_local.py:722 in mkdir                                │             
                    │                                                                                     │             
                    │   719 │   │   Create a new directory at this given path.                            │             
                    │   720 │   │   """                                                                   │             
                    │   721 │   │   try:                                                                  │             
                    │ ❱ 722 │   │   │   os.mkdir(self, mode)                                              │             
                    │   723 │   │   except FileNotFoundError:                                             │             
                    │   724 │   │   │   if not parents or self.parent == self:                            │             
                    │   725 │   │   │   │   raise                                                         │             
                    ╰─────────────────────────────────────────────────────────────────────────────────────╯             
                    FileNotFoundError: [Errno 2] No such file or directory:                                             
                    '/home/bschwedler/repos/posit-dev/images-shared/posit-bakery/brjs_test/fancy/take-3'                
❌ Failed to create image 'fancy-3'

± bakery create version fancy-2 2025.07.1
[11:18:07] INFO     Loading Bakery config from                                                             config.py:293
                    /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/brjs_test/bakery.yaml                   
           WARNING  No versions found in image 'fancy-1'. At least one version is required for most         image.py:444
                    commands.                                                                                           
           WARNING  No versions found in image 'fancy-2'. At least one version is required for most         image.py:444
                    commands.                                                                                           
           WARNING  No OSes defined for image version '2025.07.1'. At least one OS should be defined for    image.py:139
                    complete tagging and labeling of images.                                                            
           WARNING  No OS marked as primary for image version '2025.07.1'. At least one OS should be marked image.py:202
                    as primary for complete tagging and labeling of images.                                             
           ERROR    Error creating version                                                                 create.py:141
                    ╭──────────────────────── Traceback (most recent call last) ─────────────────────────╮              
                    │ /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/posit_bakery/cli/creat │              
                    │ e.py:132 in version                                                                │              
                    │                                                                                    │              
                    │   129 │                                                                            │              
                    │   130 │   try:                                                                     │              
                    │   131 │   │   c = BakeryConfig.from_context(context)                               │              
                    │ ❱ 132 │   │   c.create_version(                                                    │              
                    │   133 │   │   │   image_name=image_name,                                           │              
                    │   134 │   │   │   subpath=subpath,                                                 │              
                    │   135 │   │   │   version=image_version,                                           │              
                    │                                                                                    │              
                    │ /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/posit_bakery/config/co │              
                    │ nfig.py:428 in create_version                                                      │              
                    │                                                                                    │              
                    │   425 │   │   image_index = self._get_image_index(image_name)                      │              
                    │   426 │   │   if latest:                                                           │              
                    │   427 │   │   │   # If this is the latest version, we need to remove the latest fl │              
                    │       other versions.                                                              │              
                    │ ❱ 428 │   │   │   for v in self._config_yaml["images"][image_index]["versions"]:   │              
                    │   429 │   │   │   │   if v.get("latest", False) and v["name"] != version:          │              
                    │   430 │   │   │   │   │   v.pop("latest", None)                                    │              
                    │   431 │   │   if not existing_version:                                             │              
                    │                                                                                    │              
                    │ /home/bschwedler/repos/posit-dev/images-shared/posit-bakery/.venv/lib/python3.13/s │              
                    │ ite-packages/ruamel/yaml/comments.py:854 in __getitem__                            │              
                    │                                                                                    │              
                    │    851 │                                                                           │              
                    │    852 │   def __getitem__(self, key: Any) -> Any:                                 │              
                    │    853 │   │   try:                                                                │              
                    │ ❱  854 │   │   │   return ordereddict.__getitem__(self, key)                       │              
                    │    855 │   │   except KeyError:                                                    │              
                    │    856 │   │   │   for merged in getattr(self, merge_attrib, []):                  │              
                    │    857 │   │   │   │   # if isinstance(merged, tuple):                             │              
                    ╰────────────────────────────────────────────────────────────────────────────────────╯              
                    KeyError: 'versions'                                                                                
❌ Failed to create version 'fancy-2/2025.07.1'

@bschwedler
Copy link
Contributor

These are all candidates for breaking out into follow- up QoL issues. Just putting them down so I don't forget.

Functionality questions:

Create version

Should we be able to specify the OS and primary OS via CLI flags?

build --plan

After creating an image and a version, I would expect to be able to run bakery build --plan
Note: I did not modify any of the existing templates here so do not know if that comes into play at all.

± bakery build --plan
[09:55:22] INFO     Loading Bakery config from /home/bschwedler/repos/posit-dev/images-shared/brjs/bakery.yaml                                                                                                                        config.py:293
           WARNING  No versions found in image 'test-1'. At least one version is required for most commands.                                                                                                                           image.py:444
           WARNING  No OSes defined for image version '0.9.1'. At least one OS should be defined for complete tagging and labeling of images.                                                                                          image.py:139
           WARNING  No OS marked as primary for image version '0.9.1'. At least one OS should be marked as primary for complete tagging and labeling of images.                                                                        image.py:202
{
  "group": {
    "default": {
      "targets": []
    }
  },
  "target": {}
}

🐧venv:(.venv) aws:(platform-team) k8s:(chronicle) git:(single-yaml-config-cli*)
~/repos/posit-dev/images-shared/brjs
± tree
.
├── bakery.yaml
├── test-1
│   └── template
│       ├── Containerfile.ubuntu2204.jinja2
│       ├── deps
│       │   └── packages.txt.jinja2
│       └── test
│           └── goss.yaml.jinja2
└── test-2
    ├── 0.9.1
    │   ├── Containerfile.ubuntu2204.min
    │   ├── Containerfile.ubuntu2204.std
    │   ├── deps
    │   │   └── packages.txt
    │   └── test
    │       └── goss.yaml
    └── template
        ├── Containerfile.ubuntu2204.jinja2
        ├── deps
        │   └── packages.txt.jinja2
        └── test
            └── goss.yaml.jinja2

Me being picky

Unnecessary warnings on create

When creating a version for test-2, I get warned that there are no versions. This is likely due to the validation that occurs when reading the initial configuration file.

Maybe a solution here is when running create, we output validation messages on initial read only if there is a error. Then we perform validation on the object after we have modified it with whatever is being created?

± bakery create version test-2 0.9.1
[09:48:45] INFO     Loading Bakery config from /home/bschwedler/repos/posit-dev/images-shared/brjs/bakery.yaml                                                                                                                        config.py:293
           WARNING  No versions found in image 'test-1'. At least one version is required for most commands.                                                                                                                           image.py:444
           WARNING  No versions found in image 'test-2'. At least one version is required for most commands.                                                                                                                           image.py:444

documentation-url validation

If I specify a URL that does not include a scheme (http(s)) it fails validation. Do we want to attempt to pre-pend the URL with a scheme and then perform validation?

                    ValidationError: 1 validation error for Image                                                                                                                                                                                  
                    documentationUrl                                                                                                                                                                                                               
                      Input should be a valid URL, relative URL without a base [type=url_parsing, input_value='docs.posit.co', input_type=str]                                                                                                     
                        For further information visit https://errors.pydantic.dev/2.11/v/url_parsing       

Copy link
Contributor

@bschwedler bschwedler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is in great shape to merge.

There are some functionality questions I have, but I think they would be better addressed in smaller follow-up PRs.

Comment on lines +12 to +19
def pytest_bdd_apply_tag(tag, function):
"""Modify scenario tags to be pytest-compatible."""
if tag == "xdist-build":
marker = pytest.mark.xdist_group("build")
marker(function)
return True
else:
return None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💜

@ianpittwood ianpittwood merged commit 442d210 into main Aug 25, 2025
3 checks passed
@ianpittwood ianpittwood deleted the single-yaml-config-cli branch August 25, 2025 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants