Allow extensions to publish beta releases and users to opt-in to them

[DanTup]

Apologies if this has come up before; I'm unable to find any related issues.

It would be cool to be able to publish extensions in a beta form (similar to Code Insiders) so that they can be run "in production" with real users that have opted-in. This would reduce the disk in publishing an extension because you can push it to a smaller number of users for testing (I probably don't need to sell this to you, I guess you have Insiders for very similar reasons).

I know we can package and distribute extensions ourselves for now, but it's not as easy (eg. updating is not automatic).

I originally thought the "Preview" flag might've provided this functionality but unless I've overlooked something, it does not seem to.

[ramya-rao-a]

I would very much like this myself.

[joaomoreno]

Why not putting those experimental features behind a configuration setting and have the opt-in mechanism be toggling that setting to true? Once the feature is stable enough just remove the setting.

[DanTup]

Why not putting those experimental features behind a configuration setting and have the opt-in mechanism be toggling that setting to true?

This is an option, but I think it has some drawbacks

1. If you want to do a beta release every day, your "stable" users will get the update notifications too
2. It doesn't eliminate all the risk (if you're shipping bugs in the new features, maybe your feature detection is buggy, or how you're dealing with two code paths)
3. For things that aren't just new features, it's more work to maintain/switch between the branches
4. Users have to keep opting-in to each feature, rather than choosing to be a beta tester once; this could really limit the testing you get
5. Having the feature toggles show up in the config auto-complete isn't ideal (this might not be an issue, but I didn't get a response to Allow extensions to set configuration options. #1396 (comment) so it's unclear if "hidden options" are supposed to be used)

None of these issues are huge, but since the Insiders builds work so well and it seems like a similar scenario it seemed like a good request to raise for discussion :)

[davidanthoff]

I agree, this would be a really useful feature. Right now I'm distributing our beta releases via VSIX downloads, but that is cumbersome and I don't reach many folks that way.

Hiding things behind a feature flag wouldn't really help. For example, we are about to release a new version that changes a lot of code, the whole architecture of the extension. There is no way that we could have had that behind a feature flag.

[eamodio]

I would love to see this as well -- while feature flags work in certain cases, it doesn't cover them all. It also provides a sanity check and limits the "damage" of a bad release. And installing via vsix is just too user unfriendly to really be viable imo -- as it is completely manual.