Skip to content

Add DAW Controller support for various Arturia controllers #743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from

Conversation

soyersoyer
Copy link
Contributor

@soyersoyer soyersoyer commented Nov 5, 2024

This will be available here:
https://github.com/soyersoyer/MiniDexed

This feature can be enabled in the minidexed.ini with:

DAWControllerEnabled=1

Features

Controller Display, Main Encoder

The controller display in DAW mode shows the MiniDexed menu.
With the main encoder you can navigate in the menu.
The Home is the shift+Main click

Supported:

  • MiniLab 3 (tested)
  • KeyLab mkII (tested)
  • Keylab Essential
  • Keylab Essential 3

Energy efficiency

This version uses a little less power so it runs cooler.

Extra MIDI CC

  • Sostenuto
  • Hold2
  • Portamento

MiniLab 3 features:

Encoders

By holding down shift will bring up an overlay menu, where you can see and adjust what the encoders do.
You can use the main encoder to select the current encoding page.
Short pressing the shift will show the actual values.
Holding down the shift will goes back to the normal menu.
This overlay menu is context-aware, it is different if you are in the effect menu, or in a TG menu.
If you are in a TG menu, it only affects the selected TG.
If you are in the main menu, it affects the TGs on the first TG's channel.

  • Main Overlay 1 (default)
    Cutoff, Resonance, ReverbSend, None
    PortamentoTime, Program, Vol, None
    Faders: ChG 1 Vol, ChG 2 Vol, ChG 3 Vol, ChG 4 Vol

  • Effect Overlay
    CompressorEnable, ReverbEnable, ReverbSize, ReverbHighDamp
    ReverbLowDamp, ReverbLowPass, ReverbDiffusion, ReverbLevel

  • Main Overlay 2
    Pan1, Pan2, Pan3, Pan4
    Det1, Det2, Det3, Det4
    Faders: TG1 Vol, TG2 Vol, TG3 Vol, TG4 Vol

  • Main Overlay 3
    Pan5, Pan6, Pan7, Pan8
    Det5, Det6, Det7, Det8
    Faders: TG5 Vol, TG6 Vol, TG7 Vol, TG8 Vol

  • TG Overlay 1
    Cutoff, Resonance, ReverbSend, MasterTune
    PortamentoTime, Program, Volume, Pan
    Faders: Cutoff, Resonance, ReverbSend, Volume

  • TG Overlay 2
    MIDIChannel, None, None, PitchBendRange
    PortamentoGlissando, MonoMode, None, PitchBendStep

  • TG Overlay 3
    MWRange, MWPitch, FCRange, FCPitch
    MWEGBias, MWAmplitude, FCEGBias, FCAmplitude

  • TG Overlay 4
    BCRange, BCPitch, ATRange, ATPitch
    BCEGBias, BCAmplitude, ATEGBias, ATAmplitude

  • Voice Overlay 1
    ALGORITHM, FEEDBACK, TRANSPOSE, None,
    None, None, None, None

  • Voice Overlay 2
    PITCH_EG_R1, PITCH_EG_R2, PITCH_EG_R3, PITCH_EG_R4
    PITCH_EG_L1, PITCH_EG_L2, PITCH_EG_L3, PITCH_EG_L4
    Faders: PITCH_EG_L1, PITCH_EG_L2, PITCH_EG_L3, PITCH_EG_L4

  • Voice Overlay 3
    OSC_KEY_SYNC, LFO_SPEED, LFO_DELAY, LFO_PITCH_MOD_DEP
    LFO_SYNC, LFO_WAVE, LFO_PITCH_MOD_SENS, LFO_AMP_MOD_DEP

  • OP Overlay 1
    OP_OUTPUT_LEV, OP_FREQ_COARSE, OP_FREQ_FINE, OP_OSC_DETUNE,
    OP_OSC_MODE, OP_ENABLE, None, None

  • OP Overlay 2
    OP_EG_R1, OP_EG_R2, P_EG_R3, OP_EG_R4
    OP_EG_L1, OP_EG_L2, OP_EG_L3, OP_EG_L4,
    Faders: OP_EG_L1, OP_EG_L2, OP_EG_L3, OP_EG_L4,

  • OP Overlay 3
    OP_LEV_SCL_BRK_PT, OP_SCL_LEFT_DEPTH, OP_SCL_RGHT_DEPTH, OP_AMP_MOD_SENS
    OP_OSC_RATE_SCALE, OP_SCL_LEFT_CURVE, OP_SCL_RGHT_CURVE, OP_KEY_VEL_SENS

In the Main overlay you can reach the TG and voice overlays also.
In the voice overlay there is a page for every OP control also, with the 6 operator and with an encoder which sets all.

Pads on Bank A

  • Mono/Poly mode
  • Portamento enable/disable, hard press to enable/disable Portamento Glissando
  • Sostenuto
  • Sustain
  • All Sound Off
  • Hold2
  • None
  • Channel AfterTouch Pad

Pads on Bank B

  • Short press a TG will enable/disable that TG
  • Long press will enable/disable all the TG on the TG's channel
  • Hard press will enable only that TG (it uses the pad's AT messsages)

KeyLab mkII features

Faders

  • The faders adjust the volume of the MIDI channels of the current performance

DAW buttons

  • Mono/Poly mode
  • Portamento enable/disable, hard press to enable/disable Portamento Glissando
  • Sostenuto
  • Sustain
  • Hold2

Track select buttons

  • Short press a TG will enable/disable that TG
  • Long press will enable only that TG

Copy link

github-actions bot commented Nov 5, 2024

Build for testing:
MiniDexed_2024-11-05-1ad2394
Use at your own risk.

@probonopd
Copy link
Owner

Thank you very much for this contribution @soyersoyer.

Unfortunately I don't have the matching hardware to test this with, but I am sure many users will find this extremely useful.

Semi off topic: Do you know any good/straightforward documentation how the display and encoder on the Keystep communicates with the host over USB? It just so happens that recently I was thinking about building something the other way around (unrelated to MiniDexed), a simple microcontroller + display + encoder combination that does what a Keystep can do (e.g., to browse presets of the free Analog Lab Play), but without needing an actual Keystep.

@diyelectromusic
Copy link
Collaborator

Unfortunately I don't have the matching hardware to test this with, but I am sure many users will find this extremely useful.

Well users of this specific MIDI controller maybe, but I'm really not sure about hardcoding support for just one vendor's device in MiniDexed... In that SysEx message, 0x00, 0x20, 0x6B is the Arturia SysEx Vendor ID - right? It's a little hard to tell as I didn't seem to be able to find any mention of the MIDI implementation of Arturia devices from the manufacturer themself. Do you have a link?

If we were to go down this route, then I think we'd have to have some way of querying a connected device and mapping MiniDexed features to the features of the device itself rather than hard-code in specific SysEx messages in my view.

Do we know what other controllers support displays and how they do it?

@probonopd you weren't keen on me adding bespoke SysEx to MiniDexed to support this kind of thing between MiniDexeds, so I'm a little surprised you seem quite happy to be considering adding bespoke vendor MIDI messages other than Yamaha...?

Kevin

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Nov 6, 2024

how the display and encoder on the Keystep communicates with the host over USB?

Do you think KeyLab? KeyStep doesn't have a display. In the MiniLab 3 there is an 'Arturia' program for Analog Lab, but I don't know how it communicates. Probably after an initial model query, the main Encoder sends relative CC-s, the other encoders send absolute CCs, the Analog lab sets the encoder values on preset change and updates the display with sysex. If you think, I can capture it.

For the DAW mode, I found this:
https://github.com/PrzemekBarski/arturia-keylab-essential-mk3-programming-guide

hardcoding support for just one vendor's device in MiniDexed...

This is a very popular controller, so I think it might be worth adding.

Additional vendors may be added later.

In that SysEx message, 0x00, 0x20, 0x6B is the Arturia SysEx Vendor ID - right?

Yes

Do you have a link?

From Arturia, No.
I found the link above, and some scripts like these:
https://github.com/gluon/AbletonLive11_MIDIRemoteScripts/blob/main/MiniLab_3/midi.py
https://github.com/bitwig/bitwig-extensions/blob/api-18/src/main/java/com/bitwig/extensions/controllers/arturia/minilab3/SysExHandler.java

We can send generic device inquiry with
F0 7E 7F 06 01 F7
So in theory we can match the model, but we don't have a good spec or access to devices, so we'll probably limit the users by only enabling features for certain models without them being able to control the features.

You are right, the DAWDisplayEnabled config option is too broad and should enable some auto-detection mode.
So I think for this we should use the DAWDisplayArturiaEnabled=1 or DAWDisplay=arturia option.

@soyersoyer soyersoyer force-pushed the dawdisplay branch 2 times, most recently from 787cbe1 to 0211e16 Compare November 7, 2024 13:17
Copy link

github-actions bot commented Nov 7, 2024

Build for testing:
MiniDexed_2024-11-07-361a5ce
Use at your own risk.

@omersiar
Copy link

omersiar commented Nov 8, 2024

@soyersoyer You beat me to it while planning to do the same. It works with my MiniLab3 (firmware 1.1.1). The only issue is the Dumping MIDI to console which seems not reliable.

Also you don't configure the any of the available sliders, knobs to actual functions like Pan, Volume etc. Let me take a look.

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Nov 8, 2024

I refined it a bit. Now it does a sysex inquiry and only sends sysex if it is MiniLab3 or KeyLab 3 Essential. Does this fix your MIDI issue?
Do you know what is the preferred layout for the sliders and knobs? And how to set them up?

Copy link

github-actions bot commented Nov 8, 2024

Build for testing:
MiniDexed_2024-11-08-2fa8b3a
Use at your own risk.

Copy link

github-actions bot commented Nov 8, 2024

Build for testing:
MiniDexed_2024-11-08-5eb53ac
Use at your own risk.

Copy link

github-actions bot commented Nov 8, 2024

Build for testing:
MiniDexed_2024-11-08-74ef13c
Use at your own risk.

Copy link

github-actions bot commented Nov 8, 2024

Build for testing:
MiniDexed_2024-11-08-8601f20
Use at your own risk.

@soyersoyer soyersoyer marked this pull request as draft November 10, 2024 15:10
@omersiar
Copy link

Screenshot 2024-11-10 213302
I currently use like this

And how to set them up?
Pad RGB can be controlled also you can assign what knobs do and how to represent them.
https://github.com/mhugo/sysex
https://gist.github.com/Janiczek/04a87c2534b9d1435a1d8159c742d260

@soyersoyer
Copy link
Contributor Author

How should they work? Should they send CC-s to channel 1?

Copy link

Build for testing:
MiniDexed_2024-11-11-cff7fd1
Use at your own risk.

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Nov 11, 2024

Now, it maps the encoders. Also updates the encoders values at performance load from TG0.
The order is not the same as you suggested yet.
Isn't the volume better on fader?
I'm a bit unsure about which TG it should set and where to get the values ​​from.

Unfortunately the pads cannot be configured to send CC-s in DAW mode, they send notes only on channel 10.

What I want also:

  • If a new value appears from MIDI CC, it would be good to display it for a little time.
  • Remap the pad notes to MIDI CC-s.
  • Map the MIDI CC buttons automatically.

@soyersoyer soyersoyer changed the title Add DAW Display support for newer Arturia keyboards Add DAW Controller support for newer Arturia keyboards Nov 11, 2024
Copy link

Build for testing:
MiniDexed_2024-11-11-9c2a1bb
Use at your own risk.

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Nov 12, 2024

I think the encoders should set the parameters based on the menu, so if the display is on the TG1, it should set the TG1 parameters, if on the performance, it should set all TG-s parameters, if it is on the effects it should set the effect parameters. I will try to make this later on a different branch.

Copy link

Build for testing:
MiniDexed_2024-11-12-453f37e
Use at your own risk.

@probonopd
Copy link
Owner

probonopd commented Jan 18, 2025

What is "long press on shift the sub-menu"? @soyersoyer do you understand why the two displays might not behave identically in some situations?

@Spacepadrille
Copy link

@probonopd : lol my sentence was badly formulated. I mean activating the display of the sub menu by a long press on shift.

MiniLab 3
Keylab Essential
KeyLab Essential mk3
KeyLab mkII

based on https://github.com/PrzemekBarski/arturia-keylab-essential-mk3-programming-guide

Tested on a Arturia MiniLab 3 and KeyLab mkII

Keylab Essential and Keylab Essential mk3 is not tested
Copy link

Build for testing:
MiniDexed_2025-01-19-78141bb
Use at your own risk.

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Jan 19, 2025

But I must admit I experienced a lot of glitches and delay in the display when I turn the knobs to kraft the sound.

Sorry, I didn't test a change, and it updated the color of the pads too many times. That's what caused the glitches. If you experience this still, please let me know.

What is "long press on shift the sub-menu"?

Holding down the Shift key (CC27) will bring up an overlay menu where you can select control layouts for what the 2x4 encoders change.

The available overlay menus change depending on where you are in the menu. The Main, TG, Effect, Voice, Op menus have different overlay layers. But all voice and operator settings are routed to Main / TG / Voice also, so that on one page, for example, you can set OP_EG_R1 of the 6 operators (encoder 1-6) and all of them with encoder 8.
You can see the control values ​​by briefly pressing the shift.

I think it's a good idea to have the overlay menu appear on the default screen as well. However, it would have to be integrated into the MiniDexed menu and I don't know if others would think it's a good idea.

@Spacepadrille
Copy link

Thank you for your hard work !

@Spacepadrille
Copy link

I think it's a good idea to have the overlay menu appear on the default screen as well. However, it would have to be integrated into the MiniDexed menu and I don't know if others would think it's a good idea.

Maybe this can be "transparent", I mean not activated on minidexed without Arturia mod ? If you select "DAWControllerEnabled=1" you have the display of the sub menus on the MiniDexed screen, otherwise not. I'm not a coder, so maybe I tell sh** ?

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Jan 20, 2025

I was thinking that it would be possible to set a MIDI CC in the config for the shift button and 8 for the encoders, and it could work with any configurable controller that has 1 button and 8 encoders.

@soyersoyer
Copy link
Contributor Author

This functionality will be available here:
soyersoyer#2

@soyersoyer soyersoyer closed this Jan 20, 2025
@GertSchepens
Copy link

Oh. What does this closing mean for moving the controler code into the main MiniDexed?

@soyersoyer
Copy link
Contributor Author

#767 couldn't make it to MiniDexed in 2 months. Maybe probonopd doesn't want to see any vendor specific stuff in it? I don't know.

#767 is a small change compared to this, so if this PR could be merged, I think it would take about 1 year to get there.

Since these output zips are only available for 3 days, I'd have to do something with the code every 3 days for the next year so people can try it out here.

This makes no sense.

There's also no point in asking people to try the change #800, but that's not available (for most people).

I don't see the reason for changing the retention days to 3 anymore, because it seems like actions from public github projects wouldn't count towards storage. e9751e6

The linked PR doesn't have a 3-day limit. soyersoyer#2

I'll always try to rebase on the main from here.

@GertSchepens
Copy link

Thanks for the insight :)
I haven't been able to test for a while, but I'll check-in on the other one asap then ^^

@diyelectromusic
Copy link
Collaborator

#767 couldn't make it to MiniDexed in 2 months. Maybe probonopd doesn't want to see any vendor specific stuff in it? I don't know.

Well, to be fair - just /one week/ ago there will still some updates required for this "simple" change.

The hazard of having a larger user-base now for code like this is that changing code and testing new functionality is relatively easy, but reviewing those changes and testing for no side effects for existing platforms (remember this supports Pi V1,2,3,4,5,Zero, Zero 2, 400) takes time. People always come forward to test new functionality, especially when it meets an unmet need they had. But very few people come forward and confirm that everything is still fine without side effects on existing platforms.

And you may recall it was I who was concerned about vendor specific code (#743 (comment)) not @probonopd who has only been very supportive of these more experimental changes.

But my main concern is about longevity of support for these large changes, hence wanting to understand the implications if asked for my opinions on them.

As I say, initial implementation is one thing, but it took quite a while to build up knowledge of the code base after the initial creators left, so when faced with a large body of changes (#743, #791, #790, #778, #782, #779, #777, #747, #783, #755, #750, #758), some of which didn't seem to be solving an obvious known issue, some of which were updating core libraries, and some of which were adding a significant amount of code, then we must collectively forgive the fact that volunteer maintainers (who also do this for fun) might take quite a while to catch up with what is going on.

Kevin

@probonopd
Copy link
Owner

probonopd commented Jan 21, 2025

I am very sorry for the 3 days limit. I have been running out of space that GitHub grants. Still would very much like to see these contributions to be merged!

@soyersoyer
Copy link
Contributor Author

What do public github projects count towards?

@probonopd
Copy link
Owner

probonopd commented Jan 21, 2025

Good question, I need to check. I just got a warning from GitHub toward end of December that I was over 90%.

Edit: I think it is 1 GB across all repos of a user. And I have many repos...

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Jan 25, 2025

News:

  • Better KeyLab mkII support
  • Fewer unnecessary display/encoder/pad color updates
  • Uses less power

It turned out that it doesn't work well with the MiniLab 3 firmware v1.2.0. Please use version v1.1.1.

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Jan 27, 2025

@soyersoyer
Copy link
Contributor Author

soyersoyer commented Jan 29, 2025

News:

  • Minilab 3 firmware v1.2.0 fixes

@probonopd
Copy link
Owner

probonopd commented Jan 29, 2025

You can get a usage report.

Not entirely sure how to read this, but my understanding is that across all my repositories, I only have 0.5 GB-months in the free tier but am at 0.79 already for January.

{8098E217-BC4F-456A-A58F-169EBF379FD1}

I interpret the sentence "Removing stored artifacts will not reduce this number, but it will lower its rate of growth." as "In order to reduce this number, you need to wait until all already-uploaded artifacts have expired, and set a shorter retention time for future artifacts" - but that's just my interpretation and might well be wrong.

@soyersoyer
Copy link
Contributor Author

For me it's 0, even though I already have a few MiniDexed artifacts. I think this doesn't include public projects. If you request the usage report, you'll get a CSV where you can see how much each project is using.

@soyersoyer
Copy link
Contributor Author

News:

  • MiniLab3 Main encoder pages: add Pan/Detune pages (TG 1-4, TG 5-8) with Volume faders
  • MiniLab3 TG encoder pages: add Cutoff/Resonance/ReverbSend/Volume faders to the first page
  • MiniLab3 Voice encoder pages: add Pitch EG Level faders
  • MiniLab3 OP encoder pages: add EG level faders

@soyersoyer
Copy link
Contributor Author

Will the 3-day limit remain or will there be something else?

@probonopd
Copy link
Owner

Maybe we can find some alternative service where one can temporarily upload files?
There used to be transfer.sh but it seems to be down...

@soyersoyer
Copy link
Contributor Author

How much space does MiniDexed take up based on the CSV?

@probonopd
Copy link
Owner

I never managed to see the CSV. They don't give it for direct download but send it to some mail address (I don't even know which one, I seem to have too many).

@soyersoyer
Copy link
Contributor Author

@soyersoyer
Copy link
Contributor Author

And if you click on the "Get Usage Report" button, it will show you where it will send the report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants