Skip to content

Set up a benchmark for our trampolines.#519

Merged
adelbertc merged 3 commits into
masterfrom
topic/eval-call-bench
Sep 15, 2015
Merged

Set up a benchmark for our trampolines.#519
adelbertc merged 3 commits into
masterfrom
topic/eval-call-bench

Conversation

@non
Copy link
Copy Markdown
Contributor

@non non commented Sep 9, 2015

This commit adds a new benchmark to the bench project.
It updates the version of JMH that we are using, and
also adds a new, hidden Eval subtype which improves
the performance of Eval.defer.

This commit adds a new benchmark to the bench project.
It updates the version of JMH that we are using, and
also adds a new, hidden Eval subtype which improves
the performance of Eval.defer.
@non non added the in progress label Sep 9, 2015
@non
Copy link
Copy Markdown
Contributor Author

non commented Sep 9, 2015

Here's the benchmark result running on my machine:

Benchmark                    Mode  Cnt      Score      Error  Units
TrampolineBench.eval        thrpt    3  23049.351 ± 3891.576  ops/s
TrampolineBench.stdlib      thrpt    3  11827.608 ± 1919.654  ops/s
TrampolineBench.trampoline  thrpt    3   5743.017 ±  197.309  ops/s

(Run with bench/jmh:run -i 3 -wi 3 -f1 -t1 cats.bench.TrampolineBench)

It seems like TailCalls in 2.10 lacks .flatMap and .map,
meaning that our benchmark won't work. If we comment it
out for now, we can eventually use cats.macros.Platform
to compile this benchmark only on 2.11.
@ceedubs
Copy link
Copy Markdown
Contributor

ceedubs commented Sep 10, 2015

Are the build failures random or related to this?

@non
Copy link
Copy Markdown
Contributor Author

non commented Sep 10, 2015

I'm looking into it. It doesn't seem like Travis flakiness.

@non
Copy link
Copy Markdown
Contributor Author

non commented Sep 10, 2015

So validate seems to reliably pass locally. It could be an issue of memory/resources available.

@ceedubs
Copy link
Copy Markdown
Contributor

ceedubs commented Sep 10, 2015

I restarted the builds and this time they worked in 2.10 but not 2.11. It does seem like a quirky sporadic/resource issue, but if we aren't seeing this outside of this branch, I'm also a little hesitant to merge this PR simply because I don't want us to start completely ignoring build statuses. @inthenow do you have any ideas?

@non
Copy link
Copy Markdown
Contributor Author

non commented Sep 10, 2015

Maybe we should set up Contained-Based builds? I wonder if that would help: http://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

EDIT: This article looks old -- maybe we are already doing this. I need to investigate further.

@codecov-io
Copy link
Copy Markdown

Current coverage is 65.16%

Merging #519 into master will increase coverage by +0.07% as of b4a9df2

@@            master    #519   diff @@
======================================
  Files          156     156       
  Stmts         2418    2420     +2
  Branches        68      68       
  Methods          0       0       
======================================
+ Hit           1574    1577     +3
  Partial          0       0       
+ Missed         844     843     -1

Review entire Coverage Diff as of b4a9df2

Powered by Codecov. Updated on successful CI builds.

@codecov-io
Copy link
Copy Markdown

Current coverage is 65.16%

Merging #519 into master will increase coverage by +0.07% as of f537fec

@@            master    #519   diff @@
======================================
  Files          156     156       
  Stmts         2418    2420     +2
  Branches        68      68       
  Methods          0       0       
======================================
+ Hit           1574    1577     +3
  Partial          0       0       
+ Missed         844     843     -1

Review entire Coverage Diff as of f537fec

Powered by Codecov. Updated on successful CI builds.

@non
Copy link
Copy Markdown
Contributor Author

non commented Sep 15, 2015

Awesome. We seem to have overcome the previous build flakiness!

@ceedubs
Copy link
Copy Markdown
Contributor

ceedubs commented Sep 15, 2015

👍

1 similar comment
@adelbertc
Copy link
Copy Markdown
Contributor

👍

adelbertc added a commit that referenced this pull request Sep 15, 2015
Set up a benchmark for our trampolines.
@adelbertc adelbertc merged commit 14b5772 into master Sep 15, 2015
@non non deleted the topic/eval-call-bench branch April 28, 2016 03:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants