Optimize Cohere2-MoE graph parallel

[ikawrakow]

The Cohere2 architecture is different from other architectures in that the input activations for a repeating layer are used as input to self-attention and also to the FFN portion of the network (while typically the self-attention output serves as the input to the FFN part). Because of this property one can save one AllReduce operation when using split mode graph. This is implemented for dense Cohere2 models, but not for the MoE variant just added in PR #1945. This PR adds the optimization of avoiding AllReduce on the result of self-attention also for Cohere2-MoE.

Here is a sweep=bench comparison for North-Mini-Code-1.0 quantized with Q8_0 on a 2x3090 system using split mode graph and -muge

Main branch

PP	TG	N_KV	T_PP s	S_PP t/s	T_TG s	S_TG t/s
2048	128	0	0.354	5789.33	0.861	148.68
2048	128	2048	0.290	7059.56	0.919	139.32
2048	128	4096	0.317	6470.49	0.974	131.46
2048	128	6144	0.327	6268.50	0.980	130.64
2048	128	8192	0.333	6157.44	0.988	129.52
2048	128	10240	0.340	6021.26	0.997	128.35
2048	128	12288	0.348	5888.02	1.011	126.59
2048	128	14336	0.354	5777.87	1.015	126.08
2048	128	16384	0.364	5626.44	1.019	125.64
2048	128	18432	0.368	5558.07	1.029	124.44
2048	128	20480	0.376	5440.99	1.033	123.90
2048	128	22528	0.383	5344.13	1.044	122.63
2048	128	24576	0.391	5231.80	1.047	122.25
2048	128	26624	0.401	5106.76	1.056	121.25
2048	128	28672	0.410	4997.07	1.069	119.79
2048	128	30720	0.416	4920.37	1.071	119.51

This PR

PP	TG	N_KV	T_PP s	S_PP t/s	T_TG s	S_TG t/s
2048	128	0	0.276	7424.97	0.729	175.65
2048	128	2048	0.261	7856.95	0.787	162.72
2048	128	4096	0.286	7162.32	0.836	153.19
2048	128	6144	0.299	6850.76	0.843	151.80
2048	128	8192	0.306	6694.15	0.853	150.04
2048	128	10240	0.315	6494.87	0.868	147.51
2048	128	12288	0.322	6369.84	0.881	145.34
2048	128	14336	0.329	6226.44	0.882	145.16
2048	128	16384	0.336	6093.21	0.887	144.31
2048	128	18432	0.342	5982.55	0.893	143.36
2048	128	20480	0.350	5855.81	0.899	142.42
2048	128	22528	0.358	5718.71	0.912	140.34
2048	128	24576	0.367	5581.96	0.921	139.00
2048	128	26624	0.375	5459.95	0.927	138.13
2048	128	28672	0.383	5351.90	0.929	137.72
2048	128	30720	0.392	5229.52	0.935	136.96