Skip to content

Preparing for vty multi-platform release #260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
40 tasks done
jtdaugherty opened this issue Jul 18, 2023 · 44 comments
Closed
40 tasks done

Preparing for vty multi-platform release #260

jtdaugherty opened this issue Jul 18, 2023 · 44 comments
Assignees

Comments

@jtdaugherty
Copy link
Owner

jtdaugherty commented Jul 18, 2023

This issue is for tracking the status of numerous tasks as part of the recent/ongoing work to support Windows in Vty. These tasks are now aimed at preparing for a public release. This is mostly for me to track my own work but also to provide a place where others can see what's planned and discuss it.

Planned release/announcement date: October 30, 2023

Related issues:

People involved: @jtdaugherty @chhackett @ShrykeWindgrace

Relevant repositories and branches:

Relevant pull requests:

Tasks, roughly in order:

  • General
    • Figure out where the vty demo applications (in demos and examples) should live (probably vty-unix)
    • Move the demo applications and get them building
      • Rogue and EventEcho demos moved from vty to vty-crossplatform; build with cabal build -f demos.
    • Decide on one or more public forums to announce the change in Vty's packaging
  • vty
    • Get all (remaining) tests building
    • Get all (remaining) tests passing
    • Write changelog with details of the package split (including what changed, what moved to vty-unix, and what went away entirely and how to deal with that)
      • Easy-to-miss change: vty no longer implicitly loads the user config from disk. That is now an application responsibility (to call the library routine that does so).
    • Do full Haddock audit, looking for references to things that changed/moved/went away
    • Write a migration guidance document
    • Write a platform package implementer's guidance document
    • Update the README describing how vty needs to be used now
    • Set version to 6.0
    • Merge to master
  • vty-unix
    • Get all (remaining) tests building
    • Get all (remaining) tests passing
    • Do full Haddock audit, looking for references to things that changed/moved/went away
    • Update vty dependency to >= 6.0
    • Port the stdin flushing fix patch to vty-unix
  • vty-windows
  • vty-crossplatform
    • Add a README
    • Update vty bound to >= 6.0
  • Coordinate with @glguy on trying out changes with irc-core
  • Get all-clear on CI build(s)
  • Hackage releases
    • Update brick's latest release metadata to set Vty upper bound to < 6.0
    • Tag and release vty 6.0 to Hackage
    • Tag and release vty-unix 0.1.0.0 to Hackage
    • Work with @chhackett to get vty-windows 0.1.0.0 released to Hackage
    • Tag and release vty-crossplatform 0.1.0.0 to Hackage (depends on vty-windows and vty-unix releases)
  • Prepare public release announcement text
  • Do public announcement
  • Update brick
    • Remove unix dependency
    • Merge refactor/vty-crossplatform to master
    • Check User Guide for text and code that needs to get updated
    • Check README for text that needs to get updated
    • Add note about Windows support to README
    • Set version to 2.0
    • Cut a release to Hackage

Testing your application

Want to test your application before the release? Here’s a high-level summary of what is changing and how to test (full details to follow before the release):

  • vty will still provide the core rendering functionality and data types and your application will still need to depend on it.
  • But Vty’s initialization of the terminal (called mkVty) has moved to one of three packages:
    • If you only want to support Unix terminals, depend on vty-unix and import mkVty from that package.
    • If you only want to support Windows terminals, depend on vty-windows and import mkVty from that package.
    • If you want your application to support both platforms and choose the right implementation at build time based on the build environment, depend on vty-crossplatform and import mkVty from that package.

There are some other changes that will likely be necessary, but the amount of changes will depend on how much of the Vty API your application uses. I’m happy to answer questions if you try out the package dependency changes and then run into API issues. For most applications, the upgrade will not involve very many changes, though.

To test your application, git clone the various repositories (and check out the branches mentioned above) and then put the following cabal.project file into your application source directory during testing, updating it to point to the paths to your Git clones:

packages: .
          ../vty
          ../vty-unix
          ../vty-windows
          ../vty-crossplatform
@jtdaugherty jtdaugherty self-assigned this Jul 18, 2023
@chhackett
Copy link

@jtdaugherty
Regarding the first item in your list:

  • Figure out where the vty demo applications (in demos and examples) should live (probably vty-unix)

I'm interested in your thoughts whether it would be feasible to put the tests, demos, and examples into the crossplatform project instead. My reasoning is that I would like to maintain feature parity across windows and linux. And if I have to try and maintain a separate copy of the tests, that goal is just harder.

I've already got a copy of the demos building and running in the crossplatform project. I'll work on the examples and tests if you are agreeable to this kind of approach.

@jtdaugherty
Copy link
Owner Author

'm interested in your thoughts whether it would be feasible to put the tests, demos, and examples into the crossplatform project instead.

I think this could work nicely for the demos and examples since they are end-user programs that depend on a mkVty implementation.

I think the tests are another matter, though; I already split up the tests into those that only test the Vty core internals (those went in vty itself) and those that somehow tested Unix-specific functionality (and those now live in vty-unix). For example, some of the vty-unix tests do some things with terminfo, and I wouldn't assume that those make sense in vty-windows necessarily so I definitely wouldn't want them in vty-crossplatform.

With that said, there could also be tests that make sense in vty-crossplatform if such tests could be written to ensure that the same functionality works as desired on whatever platforms get supported by that package. I don't know what those tests would be at the moment, but I'd be all for adding a test suite to that package in the event that such cases could be devised.

So, I guess what I'm saying is that it seems to me that there could legitimately be tests that belong in each of the four packages, depending on what is being tested.

@ShrykeWindgrace
Copy link

Hi!

That's a great write-up, I'll see where and when I can help with these steps this summer.

Regarding announcements, I would say reddit and discourse are two most obvious places to promote the upcoming release. I can also cover Russian-speaking telegram channels. I follow neither IRC nor mailing lists, so I can not say whether they have a significant audience not covered by previous options.

@jtdaugherty
Copy link
Owner Author

Thanks, @ShrykeWindgrace!

@jtdaugherty
Copy link
Owner Author

@chhackett FYI the Rogue and EventEcho demo programs are now in vty-crossplatform.

@ShrykeWindgrace
Copy link

By the way, where do we report issues for current set of packages and integrations? One centralized repository (vty and/or vty-crossplatform) and then redispatch them accordingly? Or dispatch them across the four repositories?

@jtdaugherty
Copy link
Owner Author

@ShrykeWindgrace I'd say that if it isn't clear, report on vty. If an issue turns out to be a problem with a different package, we can just migrate it accordingly.

@chhackett
Copy link

Just an update for you, @jtdaugherty - I am working on bringing tests into the vty-windows plugin now, analogous to the vty-unix tests. Once I get those working I believe we will be in a position to work on the last few points of your task list here.

I did add a README to vty-windows btw. If you could give it a read and provide some feedback I'd appreciate it. I just started by copying vty README...

@hasufell
Copy link

Is there already a way to test this? Does anyone have a cabal.project configuration for this?

@chhackett
Copy link

@hasufell
Yes, you can certainly test this yourself. And I would appreciate the help. I checked in the test code yesterday.

To run the tests, check out vty and change the branch to refactor/vty-unix-split.
Then check out vty-windows (from https://github.com/chhackett/vty-windows.git).
The unit tests are in a subproject under the tests directory.
My cabal.project config just has the current directory, vty project directory and the vty-windows project directory. So something like ". ../../vty .."

The problem I'm having is with a test that tries to look up and change the console mode to support VT sequences, and is failing with an exception on the GetConsoleMode windows call.

In other contexts, such as running the demo applications (which are provided in the vty-crossplatform project) this all works fine.

@hasufell
Copy link

I want to try it in an existing package that depends on vty at the moment. I don't understand the purpose of all those new packages. Can you give a quick migration guide?

@jtdaugherty
Copy link
Owner Author

Hi @hasufell, there will definitely be a migration guide before the releases happen, but I have not gotten to that particular task yet. However, here’s a summary:

  • vty will still provide the core rendering functionality and data types and your application will still need to depend on it.
  • But Vty’s initialization of the terminal (called mkVty) has moved to one of three packages:
    • If you only want to support Unix terminals, depend on vty-unix and import mkVty from that package.
    • If you only want to support Windows terminals, depend on vty-windows and import mkVty from that package.
    • If you want your application to support both platforms and choose the right implementation at build time based on the build environment, depend on vty-crossplatform and import mkVty from that package.

There are some other changes that will likely be necessary, but the amount of changes will depend on how much of the Vty API your application uses. I’m happy to answer questions if you try out the package dependency changes and then run into API issues. For most applications, the upgrade will not involve very many changes, though.

As far as a cabal.project file, I recommend putting this in your application source directory during testing:

packages: .
          ../vty
          ../vty-unix
          ../vty-windows
          ../vty-crossplatform

And then git clone the various repositories into the parent directory accordingly, being sure to check out the refactor/vty-unix-split branch in the vty clone.

@hasufell
Copy link

Is there a patch/branch for brick yet?

@chhackett
Copy link

Writing a migration guide is still on our TODO list, but I can give you a quick and dirty version.
First, you'll need to add vty-crossplatform as a dependency to your cabal file.
Also make sure that your project does not depend on any packages that don't build on Windows, of course (like unix).
In your project main module where you construct the 'vty' instance, you'll need to import the new module Graphics.Vty.CrossPlatform.

If your project builds vty with the default configuration, then your project should build and run on Windows as-is.

For an example, see: https://github.com/jtdaugherty/vty-crossplatform/blob/main/programs/ModeDemo.hs

@jtdaugherty
Copy link
Owner Author

@chhackett - thanks for your update, that’s great news! The README you added looks good to me.

@jtdaugherty
Copy link
Owner Author

@hasufell No, there is not a branch for brick yet, but I’m happy to make one. I’ll get it put up some time today and then I’ll update here.

@chhackett
Copy link

chhackett commented Aug 20, 2023

@jtdaugherty
I do have a locally patched version (of brick) that fixes the one module that breaks on Windows, btw (FileBrowser). Would you like me to submit a PR for that?

@jtdaugherty
Copy link
Owner Author

@chhackett yes, please do. I had a note to myself about checking for other Unix dependencies in Brick; you might have fixed the only one.

@jtdaugherty
Copy link
Owner Author

@chhackett actually, let me create the Brick branch and then you can submit it against that.

@jtdaugherty
Copy link
Owner Author

@chhackett
Copy link

chhackett commented Aug 20, 2023

PR is up.

@jtdaugherty
Copy link
Owner Author

@chhackett Thanks!

@ShrykeWindgrace
Copy link

For reference: jtdaugherty/brick#445 =)

@hasufell
Copy link

hasufell commented Aug 21, 2023

I tried the following cabal.project section:

source-repository-package
  type: git
  location: https://github.com/jtdaugherty/brick.git
  tag: 250c8b471b439950ea4b3eefe45a4cc205755d85

source-repository-package
  type: git
  location: https://github.com/jtdaugherty/vty.git
  tag: 40b2a01d431afdef62a84b8f731cb5846583a088

source-repository-package
  type: git
  location: https://github.com/jtdaugherty/vty-crossplatform.git
  tag: 7beefd86af3386dfbf352563f2211fee94549752

if os(mingw32)
	source-repository-package
		type: git
		location: https://github.com/chhackett/vty-windows.git
		tag: 4cda7fe6f5b92049838e7084795591e09c201131
else
	source-repository-package
		type: git
		location: https://github.com/jtdaugherty/vty-unix.git
		tag: ad31125fe40eb9212fdea8011f34728429d4d213

But it didn't work and vty-unix failed to build:

Configuring library for vty-unix-0.1.0.0..
Preprocessing library for vty-unix-0.1.0.0..
Building library for vty-unix-0.1.0.0..
[ 1 of 18] Compiling Data.Terminfo.Parse ( src/Data/Terminfo/Parse.hs, dist/build/Data/Terminfo/Parse.o, dist/build/Data/Terminfo/Parse.dyn_o )
[ 2 of 18] Compiling Data.Terminfo.Eval ( src/Data/Terminfo/Eval.hs, dist/build/Data/Terminfo/Eval.o, dist/build/Data/Terminfo/Eval.dyn_o )
[ 3 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Classify.Types ( src/Graphics/Vty/Platform/Unix/Input/Classify/Types.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Classify/Types.o, dist/build/Graphics/Vty/Platform/Unix/Input/Classify/Types.dyn_o )
[ 4 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Classify.Parse ( src/Graphics/Vty/Platform/Unix/Input/Classify/Parse.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Classify/Parse.o, dist/build/Graphics/Vty/Platform/Unix/Input/Classify/Parse.dyn_o )
[ 5 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Focus ( src/Graphics/Vty/Platform/Unix/Input/Focus.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Focus.o, dist/build/Graphics/Vty/Platform/Unix/Input/Focus.dyn_o )
[ 6 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Mouse ( src/Graphics/Vty/Platform/Unix/Input/Mouse.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Mouse.o, dist/build/Graphics/Vty/Platform/Unix/Input/Mouse.dyn_o )
[ 7 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Paste ( src/Graphics/Vty/Platform/Unix/Input/Paste.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Paste.o, dist/build/Graphics/Vty/Platform/Unix/Input/Paste.dyn_o )
[ 8 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Classify ( src/Graphics/Vty/Platform/Unix/Input/Classify.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Classify.o, dist/build/Graphics/Vty/Platform/Unix/Input/Classify.dyn_o )
[ 9 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Terminfo.ANSIVT ( src/Graphics/Vty/Platform/Unix/Input/Terminfo/ANSIVT.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Terminfo/ANSIVT.o, dist/build/Graphics/Vty/Platform/Unix/Input/Terminfo/ANSIVT.dyn_o )
[10 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Terminfo ( src/Graphics/Vty/Platform/Unix/Input/Terminfo.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Terminfo.o, dist/build/Graphics/Vty/Platform/Unix/Input/Terminfo.dyn_o )
[11 of 18] Compiling Graphics.Vty.Platform.Unix.Output.Color ( src/Graphics/Vty/Platform/Unix/Output/Color.hs, dist/build/Graphics/Vty/Platform/Unix/Output/Color.o, dist/build/Graphics/Vty/Platform/Unix/Output/Color.dyn_o )
[12 of 18] Compiling Graphics.Vty.Platform.Unix.Output.TerminfoBased ( src/Graphics/Vty/Platform/Unix/Output/TerminfoBased.hs, dist/build/Graphics/Vty/Platform/Unix/Output/TerminfoBased.o, dist/build/Graphics/Vty/Platform/Unix/Output/TerminfoBased.dyn_o )

src/Graphics/Vty/Platform/Unix/Output/TerminfoBased.hs:209:39: warning: [-Wdeprecations]
    In the use of ‘tiGetStr’
    (imported from System.Console.Terminfo, but defined in System.Console.Terminfo.Base):
    Deprecated: "use tiGetOutput instead."
    |
209 |     = case Terminfo.getCapability ti (Terminfo.tiGetStr capName) of
    |                                       ^^^^^^^^^^^^^^^^^

src/Graphics/Vty/Platform/Unix/Output/TerminfoBased.hs:215:39: warning: [-Wdeprecations]
    In the use of ‘tiGetStr’
    (imported from System.Console.Terminfo, but defined in System.Console.Terminfo.Base):
    Deprecated: "use tiGetOutput instead."
    |
215 |     = case Terminfo.getCapability ti (Terminfo.tiGetStr capName) of
    |                                       ^^^^^^^^^^^^^^^^^
[13 of 18] Compiling Graphics.Vty.Platform.Unix.Output.XTermColor ( src/Graphics/Vty/Platform/Unix/Output/XTermColor.hs, dist/build/Graphics/Vty/Platform/Unix/Output/XTermColor.o, dist/build/Graphics/Vty/Platform/Unix/Output/XTermColor.dyn_o )
[14 of 18] Compiling Graphics.Vty.Platform.Unix.Settings ( src/Graphics/Vty/Platform/Unix/Settings.hs, dist/build/Graphics/Vty/Platform/Unix/Settings.o, dist/build/Graphics/Vty/Platform/Unix/Settings.dyn_o )
[15 of 18] Compiling Graphics.Vty.Platform.Unix.Output ( src/Graphics/Vty/Platform/Unix/Output.hs, dist/build/Graphics/Vty/Platform/Unix/Output.o, dist/build/Graphics/Vty/Platform/Unix/Output.dyn_o )
[16 of 18] Compiling Graphics.Vty.Platform.Unix.Input.Loop ( src/Graphics/Vty/Platform/Unix/Input/Loop.hs, dist/build/Graphics/Vty/Platform/Unix/Input/Loop.o, dist/build/Graphics/Vty/Platform/Unix/Input/Loop.dyn_o )
[17 of 18] Compiling Graphics.Vty.Platform.Unix.Input ( src/Graphics/Vty/Platform/Unix/Input.hs, dist/build/Graphics/Vty/Platform/Unix/Input.o, dist/build/Graphics/Vty/Platform/Unix/Input.dyn_o )
[18 of 18] Compiling Graphics.Vty.Platform.Unix ( src/Graphics/Vty/Platform/Unix.hs, dist/build/Graphics/Vty/Platform/Unix.o, dist/build/Graphics/Vty/Platform/Unix.dyn_o )
Warning: the following files would be used as linker inputs, but linking is not being done: cbits/gwinsz.h
ghc: no input files
Usage: For basic information, try the `--help' option.
Error: cabal: Failed to build vty-unix-0.1.0.0 

This is with cabal-install-3.10.1.0.

@hasufell
Copy link

It seems to work with cabal-install-3.6.2.0 though, so it looks like a cabal regression.

@jtdaugherty
Copy link
Owner Author

@hasufell thanks for your testing. Were there any other issues once Cabal build vty-unix successfully?

@hasufell
Copy link

@hasufell thanks for your testing. Were there any other issues once Cabal build vty-unix successfully?

Doesn't seem so. But I have not built on windows yet.

@jtdaugherty
Copy link
Owner Author

@chhackett on your comment above about the vty-windows README, I took a look. I think the only thing I'd recommend is to mention that vty-crossplatform might be a preferable dependency if someone wants to support multiple platforms. vty-windows itself would only be recommended for developers who only want to support Windows. So maybe add a note about that with a link to the repo. (By the time you release your package, I'll be about to release vty-crossplatform so it'll be on Hackage by that time.)

@jtdaugherty
Copy link
Owner Author

jtdaugherty commented Oct 23, 2023

My release announcement text draft:

I’m happy to announce the release of vty version 6.0. The highlight
of this release is that vty now works on Windows, thanks to hard work
by Chris Hackett and Timofey Zakrevskiy! For years, many people in the
Haskell community have requested Windows support in vty. Thanks to
Chris and Timofey for the time and energy they put into this, and their
patience as we worked out an approach! Thanks also to Eric Mertens
and Kevin Quick for early feedback on the release.

In this release, the core "vty" package was reorganized so that support
for specific platforms now lives in platform-specific packages that need
to be used alongside the core "vty" package. This release includes the
release of three aditional new packages:

* vty-unix (Unix terminal support previously provided by "vty")
* vty-windows (the Windows backend)
* vty-crossplatform (for when you want to support both)

The core package still provides all of the important library
functionality and is still a necessary dependency. Applications will
need to depend on an additional package for the platform that they need
to support. When in doubt, use vty-crossplatform!

The release changelog update has a detailed listing of changes as well
as instructions for how to update your application to work with Vty 6:

https://github.com/jtdaugherty/vty/blob/master/CHANGELOG.md#60

To get help, please open an issue on the Vty repository.

While I'm at it, Brick has been updated to use vty-crossplatform in version 2.0!

Thanks!

@hasufell
Copy link

Cool, ghcup might be the first to use it for next release.

@jtdaugherty
Copy link
Owner Author

@hasufell I'm glad to hear it! I'm currently working with @chhackett to choose a release date. I'll update here when we have a date in mind.

@jtdaugherty
Copy link
Owner Author

jtdaugherty commented Oct 23, 2023

@chhackett I'm wondering if you could stress-test ghcup on a Windows system. Do you have time to give that a try?

@ShrykeWindgrace
Copy link

I tried building ghcup (without tui) on windows with stack and failed. Maybe cabal can manage that.

@ShrykeWindgrace
Copy link

My release announcement text draft:

Hurrah!

@hasufell
Copy link

hasufell commented Oct 23, 2023

I tried building ghcup (without tui) on windows with stack and failed. Maybe cabal can manage that.

Use cabal --project-file=cabal.project.release build -w ghc-9.2.8 (with cabal-3.10.1.0)

@ShrykeWindgrace
Copy link

I tried building ghcup (without tui) on windows with stack and failed. Maybe cabal can manage that.

Use cabal --project-file=cabal.project.release build -w ghc-9.2.8 (with cabal-3.10.1.0)

I'll test that, thanks=)

@jtdaugherty
Copy link
Owner Author

FYI: @chhackett and I are planning on October 30 for the releases.

@jtdaugherty
Copy link
Owner Author

As of around 3 PM Pacific time today, vty 6.0, vty-crossplatform, and vty-unix have been published to Hackage. Once vty-windows is uploaded, I'll put up the announcements.

@chhackett
Copy link

chhackett commented Oct 30, 2023

So I ran into a permission problem. Apparently I need to be added to the 'upload group' of hackage. I did not realize there was more to the process beyond getting an account.

So I sent out the email to request that, but I guess I don't know how long it will take to get that issue resolved.

I guess I'm a little surprised by this. I read the page about uploading (https://hackage.haskell.org/upload) and it just said I needed a username/password, which I got.

@jtdaugherty
Copy link
Owner Author

@chhackett ah yeah, it's been so long since I had to do that that I forgot that it was a thing. No problem!

@chhackett
Copy link

I'll let you know here as soon as I get permission, and get vty-windows uploaded.

@chhackett
Copy link

vty-windows is uploaded. Thank you @jtdaugherty for orchestrating all the pieces of this project, and @ShrykeWindgrace for all your help with the prototyping, investigation and problem solving skills.

@jtdaugherty
Copy link
Owner Author

Announcements are now up!

Thanks, everyone!

And with that, I will now close this ticket!

@simonmichael
Copy link

simonmichael commented Oct 31, 2023

Huge thanks to all who made this happen! While I'm not using Windows myself, I very much appreciate that this allows my vty- and brick-based software to also run on Windows with little effort on my part. 🙏🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants