-
Notifications
You must be signed in to change notification settings - Fork 475
chore(ci_visibility): test new pytest plugin on the ddtrace CI #15742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…w plugin tests; itr:noskip
|
|
Performance SLOsComparing candidate vitor-de-araujo/test-new-pytest-plugin (6c09b9e) with baseline main (f8b72bc) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 17.961µs (SLO: <20.000µs 📉 -10.2%) vs baseline: 📈 +21.4% Memory: ✅ 42.920MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.5% ✅ add_inplace_aspectTime: ✅ 14.964µs (SLO: <20.000µs 📉 -25.2%) vs baseline: +0.7% Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.9% ✅ add_inplace_noaspectTime: ✅ 0.336µs (SLO: <10.000µs 📉 -96.6%) vs baseline: -0.1% Memory: ✅ 43.116MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +5.2% ✅ add_noaspectTime: ✅ 0.547µs (SLO: <10.000µs 📉 -94.5%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ bytearray_aspectTime: ✅ 17.951µs (SLO: <30.000µs 📉 -40.2%) vs baseline: -0.3% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ bytearray_extend_aspectTime: ✅ 23.844µs (SLO: <30.000µs 📉 -20.5%) vs baseline: -0.5% Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.9% ✅ bytearray_extend_noaspectTime: ✅ 2.736µs (SLO: <10.000µs 📉 -72.6%) vs baseline: -0.2% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ bytearray_noaspectTime: ✅ 1.469µs (SLO: <10.000µs 📉 -85.3%) vs baseline: -0.9% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ bytes_aspectTime: ✅ 16.590µs (SLO: <20.000µs 📉 -17.1%) vs baseline: -0.5% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ bytes_noaspectTime: ✅ 1.408µs (SLO: <10.000µs 📉 -85.9%) vs baseline: -1.2% Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.0% ✅ bytesio_aspectTime: ✅ 55.243µs (SLO: <70.000µs 📉 -21.1%) vs baseline: -0.3% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ bytesio_noaspectTime: ✅ 3.281µs (SLO: <10.000µs 📉 -67.2%) vs baseline: -0.5% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.1% ✅ capitalize_aspectTime: ✅ 14.678µs (SLO: <20.000µs 📉 -26.6%) vs baseline: ~same Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ capitalize_noaspectTime: ✅ 2.591µs (SLO: <10.000µs 📉 -74.1%) vs baseline: -0.4% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ casefold_aspectTime: ✅ 14.635µs (SLO: <20.000µs 📉 -26.8%) vs baseline: ~same Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.1% ✅ casefold_noaspectTime: ✅ 3.157µs (SLO: <10.000µs 📉 -68.4%) vs baseline: -0.5% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ decode_aspectTime: ✅ 15.633µs (SLO: <30.000µs 📉 -47.9%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.5% ✅ decode_noaspectTime: ✅ 1.601µs (SLO: <10.000µs 📉 -84.0%) vs baseline: +0.4% Memory: ✅ 43.018MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +5.0% ✅ encode_aspectTime: ✅ 18.155µs (SLO: <30.000µs 📉 -39.5%) vs baseline: 📈 +22.5% Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.2% ✅ encode_noaspectTime: ✅ 1.496µs (SLO: <10.000µs 📉 -85.0%) vs baseline: -0.4% Memory: ✅ 43.018MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.9% ✅ format_aspectTime: ✅ 171.004µs (SLO: <200.000µs 📉 -14.5%) vs baseline: ~same Memory: ✅ 43.155MB (SLO: <43.250MB 🟡 -0.2%) vs baseline: +4.8% ✅ format_map_aspectTime: ✅ 191.393µs (SLO: <200.000µs -4.3%) vs baseline: +0.2% Memory: ✅ 43.175MB (SLO: <43.500MB 🟡 -0.7%) vs baseline: +4.9% ✅ format_map_noaspectTime: ✅ 3.825µs (SLO: <10.000µs 📉 -61.7%) vs baseline: -0.4% Memory: ✅ 43.037MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +5.1% ✅ format_noaspectTime: ✅ 3.149µs (SLO: <10.000µs 📉 -68.5%) vs baseline: -1.4% Memory: ✅ 42.979MB (SLO: <43.250MB 🟡 -0.6%) vs baseline: +4.9% ✅ index_aspectTime: ✅ 15.319µs (SLO: <20.000µs 📉 -23.4%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <43.250MB 🟡 -0.7%) vs baseline: +4.7% ✅ index_noaspectTime: ✅ 0.464µs (SLO: <10.000µs 📉 -95.4%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ join_aspectTime: ✅ 16.949µs (SLO: <20.000µs 📉 -15.3%) vs baseline: -1.0% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.7% ✅ join_noaspectTime: ✅ 1.558µs (SLO: <10.000µs 📉 -84.4%) vs baseline: -0.3% Memory: ✅ 42.920MB (SLO: <43.250MB 🟡 -0.8%) vs baseline: +4.6% ✅ ljust_aspectTime: ✅ 20.792µs (SLO: <30.000µs 📉 -30.7%) vs baseline: ~same Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.8% ✅ ljust_noaspectTime: ✅ 2.706µs (SLO: <10.000µs 📉 -72.9%) vs baseline: +0.2% Memory: ✅ 42.959MB (SLO: <43.250MB 🟡 -0.7%) vs baseline: +4.9% ✅ lower_aspectTime: ✅ 17.832µs (SLO: <30.000µs 📉 -40.6%) vs baseline: ~same Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ lower_noaspectTime: ✅ 2.434µs (SLO: <10.000µs 📉 -75.7%) vs baseline: +0.5% Memory: ✅ 43.037MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.9% ✅ lstrip_aspectTime: ✅ 17.514µs (SLO: <20.000µs 📉 -12.4%) vs baseline: -1.0% Memory: ✅ 43.077MB (SLO: <43.250MB 🟡 -0.4%) vs baseline: +5.1% ✅ lstrip_noaspectTime: ✅ 1.862µs (SLO: <10.000µs 📉 -81.4%) vs baseline: -0.1% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ modulo_aspectTime: ✅ 165.870µs (SLO: <200.000µs 📉 -17.1%) vs baseline: -0.1% Memory: ✅ 43.234MB (SLO: <43.500MB 🟡 -0.6%) vs baseline: +5.0% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 174.355µs (SLO: <200.000µs 📉 -12.8%) vs baseline: ~same Memory: ✅ 43.214MB (SLO: <43.500MB 🟡 -0.7%) vs baseline: +4.8% ✅ modulo_aspect_for_bytesTime: ✅ 168.806µs (SLO: <200.000µs 📉 -15.6%) vs baseline: ~same Memory: ✅ 43.096MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +4.5% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 171.415µs (SLO: <200.000µs 📉 -14.3%) vs baseline: -0.3% Memory: ✅ 43.175MB (SLO: <43.500MB 🟡 -0.7%) vs baseline: +4.9% ✅ modulo_noaspectTime: ✅ 3.679µs (SLO: <10.000µs 📉 -63.2%) vs baseline: -0.1% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ replace_aspectTime: ✅ 220.064µs (SLO: <300.000µs 📉 -26.6%) vs baseline: +3.7% Memory: ✅ 43.175MB (SLO: <44.000MB 🟡 -1.9%) vs baseline: +4.8% ✅ replace_noaspectTime: ✅ 2.907µs (SLO: <10.000µs 📉 -70.9%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ repr_aspectTime: ✅ 1.419µs (SLO: <10.000µs 📉 -85.8%) vs baseline: +0.5% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ repr_noaspectTime: ✅ 0.533µs (SLO: <10.000µs 📉 -94.7%) vs baseline: +1.8% Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.0% ✅ rstrip_aspectTime: ✅ 23.058µs (SLO: <30.000µs 📉 -23.1%) vs baseline: 📈 +21.6% Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.1% ✅ rstrip_noaspectTime: ✅ 1.944µs (SLO: <10.000µs 📉 -80.6%) vs baseline: +0.4% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ slice_aspectTime: ✅ 15.748µs (SLO: <20.000µs 📉 -21.3%) vs baseline: ~same Memory: ✅ 42.900MB (SLO: <43.500MB 🟡 -1.4%) vs baseline: +4.8% ✅ slice_noaspectTime: ✅ 0.592µs (SLO: <10.000µs 📉 -94.1%) vs baseline: -1.5% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ stringio_aspectTime: ✅ 53.804µs (SLO: <80.000µs 📉 -32.7%) vs baseline: -0.4% Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +5.0% ✅ stringio_noaspectTime: ✅ 3.656µs (SLO: <10.000µs 📉 -63.4%) vs baseline: -0.3% Memory: ✅ 43.018MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +5.0% ✅ strip_aspectTime: ✅ 17.631µs (SLO: <20.000µs 📉 -11.8%) vs baseline: -0.3% Memory: ✅ 43.018MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +5.0% ✅ strip_noaspectTime: ✅ 1.865µs (SLO: <10.000µs 📉 -81.3%) vs baseline: +0.8% Memory: ✅ 42.939MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.5% ✅ swapcase_aspectTime: ✅ 18.528µs (SLO: <30.000µs 📉 -38.2%) vs baseline: +0.2% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.7% ✅ swapcase_noaspectTime: ✅ 2.808µs (SLO: <10.000µs 📉 -71.9%) vs baseline: ~same Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ title_aspectTime: ✅ 18.368µs (SLO: <30.000µs 📉 -38.8%) vs baseline: +0.4% Memory: ✅ 42.900MB (SLO: <43.000MB 🟡 -0.2%) vs baseline: +4.6% ✅ title_noaspectTime: ✅ 2.865µs (SLO: <10.000µs 📉 -71.4%) vs baseline: +7.1% Memory: ✅ 43.018MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.9% ✅ translate_aspectTime: ✅ 20.448µs (SLO: <30.000µs 📉 -31.8%) vs baseline: -0.7% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ translate_noaspectTime: ✅ 4.317µs (SLO: <10.000µs 📉 -56.8%) vs baseline: -0.6% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.1% ✅ upper_aspectTime: ✅ 17.900µs (SLO: <30.000µs 📉 -40.3%) vs baseline: -0.1% Memory: ✅ 42.841MB (SLO: <43.500MB 🟡 -1.5%) vs baseline: +4.4% ✅ upper_noaspectTime: ✅ 2.441µs (SLO: <10.000µs 📉 -75.6%) vs baseline: -0.3% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.7% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.095µs (SLO: <10.000µs 📉 -49.0%) vs baseline: 📈 +19.4% Memory: ✅ 41.425MB (SLO: <43.500MB -4.8%) vs baseline: +4.9% ✅ ospathbasename_noaspectTime: ✅ 4.305µs (SLO: <10.000µs 📉 -56.9%) vs baseline: +0.3% Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9% ✅ ospathjoin_aspectTime: ✅ 6.235µs (SLO: <10.000µs 📉 -37.6%) vs baseline: -0.6% Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +5.0% ✅ ospathjoin_noaspectTime: ✅ 6.278µs (SLO: <10.000µs 📉 -37.2%) vs baseline: -0.7% Memory: ✅ 41.504MB (SLO: <43.500MB -4.6%) vs baseline: +5.0% ✅ ospathnormcase_aspectTime: ✅ 3.556µs (SLO: <10.000µs 📉 -64.4%) vs baseline: +0.2% Memory: ✅ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.9% ✅ ospathnormcase_noaspectTime: ✅ 3.606µs (SLO: <10.000µs 📉 -63.9%) vs baseline: -1.3% Memory: ✅ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 4.883µs (SLO: <10.000µs 📉 -51.2%) vs baseline: -0.3% Memory: ✅ 41.484MB (SLO: <43.500MB -4.6%) vs baseline: +5.1% ✅ ospathsplit_noaspectTime: ✅ 5.026µs (SLO: <10.000µs 📉 -49.7%) vs baseline: +1.0% Memory: ✅ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 3.759µs (SLO: <10.000µs 📉 -62.4%) vs baseline: -1.0% Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 0.746µs (SLO: <10.000µs 📉 -92.5%) vs baseline: -1.0% Memory: ✅ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9% ✅ ospathsplitext_aspectTime: ✅ 4.615µs (SLO: <10.000µs 📉 -53.8%) vs baseline: -0.5% Memory: ✅ 41.484MB (SLO: <43.500MB -4.6%) vs baseline: +5.0% ✅ ospathsplitext_noaspectTime: ✅ 4.650µs (SLO: <10.000µs 📉 -53.5%) vs baseline: +1.1% Memory: ✅ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +4.8% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.396µs (SLO: <20.000µs 📉 -83.0%) vs baseline: 📈 +14.1% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.2% ✅ 1-count-metrics-100-timesTime: ✅ 200.403µs (SLO: <220.000µs -8.9%) vs baseline: +0.6% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.8% ✅ 1-distribution-metric-1-timesTime: ✅ 3.324µs (SLO: <20.000µs 📉 -83.4%) vs baseline: -0.4% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 1-distribution-metrics-100-timesTime: ✅ 214.704µs (SLO: <230.000µs -6.7%) vs baseline: +0.8% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.3% ✅ 1-gauge-metric-1-timesTime: ✅ 2.173µs (SLO: <20.000µs 📉 -89.1%) vs baseline: -0.8% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +5.0% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.096µs (SLO: <150.000µs -9.3%) vs baseline: -0.3% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +5.4% ✅ 1-rate-metric-1-timesTime: ✅ 3.144µs (SLO: <20.000µs 📉 -84.3%) vs baseline: +0.8% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.1% ✅ 1-rate-metrics-100-timesTime: ✅ 213.314µs (SLO: <250.000µs 📉 -14.7%) vs baseline: -0.6% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 100-count-metrics-100-timesTime: ✅ 19.977ms (SLO: <22.000ms -9.2%) vs baseline: +0.2% Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.3% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.214ms (SLO: <2.550ms 📉 -13.2%) vs baseline: -1.8% Memory: ✅ 35.271MB (SLO: <35.500MB 🟡 -0.6%) vs baseline: +5.2% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.407ms (SLO: <1.550ms -9.2%) vs baseline: +0.5% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ 100-rate-metrics-100-timesTime: ✅ 2.191ms (SLO: <2.550ms 📉 -14.1%) vs baseline: +0.9% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.626µs (SLO: <20.000µs 📉 -76.9%) vs baseline: +0.2% Memory: ✅ 35.134MB (SLO: <35.500MB 🟡 -1.0%) vs baseline: +4.6% ✅ flush-100-metricsTime: ✅ 173.540µs (SLO: <250.000µs 📉 -30.6%) vs baseline: -0.4% Memory: ✅ 35.291MB (SLO: <35.500MB 🟡 -0.6%) vs baseline: +4.7% ✅ flush-1000-metricsTime: ✅ 2.173ms (SLO: <2.500ms 📉 -13.1%) vs baseline: +0.2% Memory: ✅ 36.058MB (SLO: <36.500MB 🟡 -1.2%) vs baseline: +4.7% 🟡 Near SLO Breach (15 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
…raujo/test-new-pytest-plugin
…raujo/test-new-pytest-plugin
Description
Testing
Risks
Additional Notes