Skip to content

Conversation

@mkleene
Copy link
Contributor

@mkleene mkleene commented May 31, 2024

The previous API was logically incorrect and the only reason that we hadn't uncovered it was because we
only had payloads that had one segment.

Previously we'd write a manifest entry for each segment. Now we return a stream that clients can write into
for the payload. So the API changes to

var writer = new ZipWriter(out);
writer.data("entry one", new byte[] { ... });
try (var out = writer.stream("entry two")) {
  var fi = new FileInputStream("...");
  fi.transferTo(out);
}
writer.finish(); // write the zip manifest

Also change things so that we can handle streams that don't return all of the bytes requested during
a given read. Previously we assumed that a read would always return all of the bytes that might be available
in a stream.

Address #66

@mkleene mkleene requested review from a team as code owners May 31, 2024 15:22
@mkleene mkleene changed the title fix: zip streaming fix: create TDFs larger than a single segment Jun 3, 2024
@mkleene mkleene merged commit e1da325 into main Jun 4, 2024
@mkleene mkleene deleted the fix/zip-streaming branch June 4, 2024 15:48
ttschampel pushed a commit that referenced this pull request Jun 13, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>0.1.0</summary>

## 0.1.0 (2024-06-13)


### Features

* add code to create services for SDK
([#35](#35))
([28513e6](28513e6))
* add logging ([#49](#49))
([9d20647](9d20647))
* Add NanoTDF E2E Tests
([#75](#75))
([84f9bd1](84f9bd1))
* **codegen:** Generate and publish Java Proto generated artifacts
([#2](#2))
([2328fd2](2328fd2))
* crypto API ([#33](#33))
([b8295b7](b8295b7))
* NanoTDF Implementation
([#46](#46))
([6485326](6485326))
* **PLAT-3087:** zip reader-writer
([#23](#23))
([3eeb626](3eeb626))
* SDK Encrypt (with mocked rewrap)
([#45](#45))
([d67daa2](d67daa2))
* **sdk:** add CLI and integration tests
([#64](#64))
([df20e6d](df20e6d))
* **sdk:** add ssl context
([#58](#58))
([80246a9](80246a9))
* **sdk:** get e2e rewrap working
([#52](#52))
([fe2c04b](fe2c04b))
* **sdk:** Issue [#60](#60) -
expose SDK ([#61](#61))
([ddef62a](ddef62a))
* **sdk:** provide access tokens dynamically to KAS
([#51](#51))
([04ca715](04ca715))
* **sdk:** update archive support
([#47](#47))
([29a80a9](29a80a9))


### Bug Fixes

* create TDFs larger than a single segment
([#65](#65))
([e1da325](e1da325))
* fix pom for release please
([#77](#77))
([3a3c357](3a3c357))
* Force BC provider use
([#76](#76))
([1bc9dd9](1bc9dd9))
* get rid of duplicate channel logic
([#59](#59))
([1edd666](1edd666))
* **sdk:** allow SDK to handle protocols in addresses
([#70](#70))
([97ae8ee](97ae8ee))
* **sdk:** make sdk auto closeable
([#63](#63))
([c1bbbb4](c1bbbb4))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
This was referenced Aug 27, 2024
pflynn-virtru pushed a commit that referenced this pull request Oct 8, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>0.7.2</summary>

## [0.7.2](v0.7.0...v0.7.2)
(2024-10-08)


### ⚠ BREAKING CHANGES

* move to single jar
([#160](#160))

### Features

* add code to create services for SDK
([#35](#35))
([28513e6](28513e6))
* add logging ([#49](#49))
([9d20647](9d20647))
* Add NanoTDF E2E Tests
([#75](#75))
([84f9bd1](84f9bd1))
* adds token exchange and general auth
([#176](#176))
([bb325c4](bb325c4))
* BACK-2316 add a simple method to detect TDFs
([#111](#111))
([bfbef70](bfbef70))
* **build:** maven refactor for maven central
([#174](#174))
([c640773](c640773)),
closes [#79](#79)
* **ci:** Add xtest workflow trigger
([#96](#96))
([bc54b63](bc54b63))
* **cmd:** Adds command `--mime-type` opt
([#113](#113))
([45a2c30](45a2c30))
* **cmdline:** Adds --ecdsa-binding and help
([#164](#164))
([ed6e982](ed6e982))
* **codegen:** Generate and publish Java Proto generated artifacts
([#2](#2))
([2328fd2](2328fd2))
* **core:** Add attributes client
([#118](#118))
([98ba6a9](98ba6a9))
* **core:** Add autoconfigure for key splitting
([#120](#120))
([7ecbf23](7ecbf23))
* **core:** Adding key cache, tests for specificity
([#126](#126))
([a149887](a149887))
* **core:** Handle split keys on tdf3 encrypt and decrypt
([#109](#109))
([943751f](943751f))
* **core:** KID in NanoTDF
([#112](#112))
([33b5982](33b5982))
* **core:** NanoTDF resource locator protocol bit mask
([#107](#107))
([159d2f1](159d2f1))
* crypto API ([#33](#33))
([b8295b7](b8295b7))
* **lib:** add fallback to namespace kas
([#166](#166))
([4368840](4368840))
* NanoTDF Implementation
([#46](#46))
([6485326](6485326))
* **PLAT-3087:** zip reader-writer
([#23](#23))
([3eeb626](3eeb626))
* SDK Encrypt (with mocked rewrap)
([#45](#45))
([d67daa2](d67daa2))
* **sdk:** add CLI and integration tests
([#64](#64))
([df20e6d](df20e6d))
* **sdk:** add mime type.
([#108](#108))
([6c4a27b](6c4a27b))
* **sdk:** add ssl context
([#58](#58))
([80246a9](80246a9))
* **sdk:** expose GRPC auth service components
([#92](#92))
([2595cc5](2595cc5))
* **sdk:** get e2e rewrap working
([#52](#52))
([fe2c04b](fe2c04b))
* **sdk:** Issue [#60](#60) -
expose SDK ([#61](#61))
([ddef62a](ddef62a))
* **sdk:** provide access tokens dynamically to KAS
([#51](#51))
([04ca715](04ca715))
* **sdk:** the authorization service is needed for use by gateway
([#85](#85))
([73cac82](73cac82))
* **sdk:** update archive support
([#47](#47))
([29a80a9](29a80a9))
* **sdk:** Update the assertion support to match go sdk
([#117](#117))
([f9badb3](f9badb3))
* support key id in ztdf key access object
([#84](#84))
([862460a](862460a))
* update README.md
([#142](#142))
([198d335](198d335))


### Bug Fixes

* Align identifier bytes correctly in ResourceLocator
([#148](#148))
([2efe226](2efe226))
* **core:** Add support for certs
([#131](#131))
([2f98a3a](2f98a3a))
* **core:** Revert "feat(core): Add attributes client"
([#124](#124))
([3d1ef2b](3d1ef2b))
* create TDFs larger than a single segment
([#65](#65))
([e1da325](e1da325))
* fix pom for release please
([#77](#77))
([3a3c357](3a3c357))
* Force BC provider use
([#76](#76))
([1bc9dd9](1bc9dd9))
* get rid of duplicate channel logic
([#59](#59))
([1edd666](1edd666))
* GitHub packages snapshot repo
([#178](#178))
([713cb2b](713cb2b))
* GPG key and Maven credentials in release workflow
([#171](#171))
([864e9ce](864e9ce))
* Issue [#115](#115) - fix for
SSL Context for IDP and plaintext platform
([#116](#116))
([36a29df](36a29df))
* make sure we do not deserialize null
([#97](#97))
([9579c42](9579c42))
* **nano:** Store key ids if found
([#134](#134))
([94c672b](94c672b))
* passpharse ([#169](#169))
([8b3cbed](8b3cbed))
* policy-binding new structure
([#95](#95))
([b10a61e](b10a61e))
* **sdk:** allow SDK to handle protocols in addresses
([#70](#70))
([97ae8ee](97ae8ee))
* **sdk:** assertion support in tdf3
([#82](#82))
([c299dbd](c299dbd))
* **sdk:** give a test framework test scope
([#90](#90))
([b99de43](b99de43))
* **sdk:** make sdk auto closeable
([#63](#63))
([c1bbbb4](c1bbbb4))
* **sdk:** Mixed split fix
([#163](#163))
([649dac7](649dac7))
* ztdf support both base and handling assertions
([#128](#128))
([5f72e94](5f72e94))


### Documentation

* **sdk:** Adds brief usage code sample
([#26](#26))
([79215c7](79215c7))


### Miscellaneous Chores

* release 0.6.1 Release-As: 0.6.1
([#135](#135))
([09ec548](09ec548))
* release 0.7.2 ([#184](#184))
([ea6cf12](ea6cf12))


### Code Refactoring

* move to single jar
([#160](#160))
([ba9b2d5](ba9b2d5))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants