Skip to content

Benchmarking tool suggestion. #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
fbernier opened this issue Mar 28, 2013 · 1 comment
Closed

Benchmarking tool suggestion. #2

fbernier opened this issue Mar 28, 2013 · 1 comment

Comments

@fbernier
Copy link

I don't know WeigHTTP enough to make any comparison but I I've been recommanded this one often and it does work quite well:

https://github.com/wg/wrk
@TechEmpower
Copy link
Collaborator

Thanks for the suggestion @fbernier! We're currently in the process of moving from WeigHTTP to wrk. We wanted a tool that gave us latency and standard deviation, and this fits the bill.

pfalls1 pushed a commit that referenced this issue Apr 23, 2013
Merge pull request #2 from pfig/feature/mojolicious
e101d0c 
Mojolicious (Perl) app
mirob2005 pushed a commit that referenced this issue Oct 12, 2013
Merge pull request #2 from pfig/feature/mojolicious
b56ccf4 
Mojolicious (Perl) app
msmith-techempower pushed a commit that referenced this issue Jan 16, 2014
@raphaelbauer
Merge pull request #2 from kpacha/doc-url-fix
0a20f2d 
fix the test urls
hamiltont pushed a commit that referenced this issue Aug 24, 2014
@ikkez
fix db raw test return values, #2
fdb4682
hamiltont pushed a commit that referenced this issue Aug 24, 2014
@ikkez
fix fortunes test, #2
2ec6396
hamiltont pushed a commit that referenced this issue Aug 24, 2014
@ikkez
fix additional fortune item, #2
c8d1396
@andregmoeller andregmoeller mentioned this issue Feb 17, 2016
Merged
NateBrady23 pushed a commit that referenced this issue Jul 6, 2016
@sanjoydesk
Merge pull request #2 from TechEmpower/master
5468f3d 
Merging base branch
michaelhixson added a commit to michaelhixson/FrameworkBenchmarks that referenced this issue Jan 30, 2018
@michaelhixson
Try to fix PHP/hhvm tests by working around an issue with SET NAMES
8856c48 
I think we are running into this problem:

  facebook/hhvm#8111

We don't solve that problem, but we avoid it by specifying utf8 in a
different fashion.  I also took the opportunity to use the "TFB-database"
hostname instead of the DBHOST environment variable because it's simpler
that way.

Here are the steps I took in a local TFB environment that make me think
we're running into the same issue as that GitHub issue:

 - Run toolset/run-tests.py --mode verify --test hvm, see that it fails
   all tests with error messages that look like the application server
   simply isn't there
 - Enable logging in PHP/hhvm/deploy/config.hdf (basically copy the
   "Log" section over from config-debug.hdf), run the test again
 - Look at the log file it spit out, PHP/hhvm/error.log, notice messages
   like this:

    Core dumped: Segmentation fault
    Stack trace in /tmp/stacktrace.28653.log

 - /tmp is cleared by the TFB toolset each run, so I comment out that
   part of benchmarker.py that clears /tmp
 - Run the test again, look at the stacktrace file in /tmp, notice a
   stack trace referring to our implementation code like this:

    #0  PDO->__construct() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/once.php.inc:14]
    TechEmpower#1  Benchmark->setup_db() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/once.php.inc:31]
    TechEmpower#2  Benchmark->bench_db() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/db.php:10]
    TechEmpower#3  main() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/db.php:13]

 - Google a bit, come across a GitHub issue talking about a similar
   problem when they used "SET NAMES"
 - Comment the part of PHP/hhvm/once.php.inc that uses SET NAMES, run
   test again and it worked, except for fortunes which failed because of
   character encoding problems
 - Google for a different way to specify utf8 in the PDO constructor,
   try it again, all tests pass locally
@michaelhixson michaelhixson mentioned this issue Jan 30, 2018
Merged
NateBrady23 pushed a commit that referenced this issue Jan 30, 2018
@michaelhixson @NateBrady23
Try to fix PHP/hhvm tests by working around an issue with SET NAMES (#…
903997f 
…3220)

I think we are running into this problem:

  facebook/hhvm#8111

We don't solve that problem, but we avoid it by specifying utf8 in a
different fashion.  I also took the opportunity to use the "TFB-database"
hostname instead of the DBHOST environment variable because it's simpler
that way.

Here are the steps I took in a local TFB environment that make me think
we're running into the same issue as that GitHub issue:

 - Run toolset/run-tests.py --mode verify --test hvm, see that it fails
   all tests with error messages that look like the application server
   simply isn't there
 - Enable logging in PHP/hhvm/deploy/config.hdf (basically copy the
   "Log" section over from config-debug.hdf), run the test again
 - Look at the log file it spit out, PHP/hhvm/error.log, notice messages
   like this:

    Core dumped: Segmentation fault
    Stack trace in /tmp/stacktrace.28653.log

 - /tmp is cleared by the TFB toolset each run, so I comment out that
   part of benchmarker.py that clears /tmp
 - Run the test again, look at the stacktrace file in /tmp, notice a
   stack trace referring to our implementation code like this:

    #0  PDO->__construct() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/once.php.inc:14]
    #1  Benchmark->setup_db() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/once.php.inc:31]
    #2  Benchmark->bench_db() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/db.php:10]
    #3  main() called at [/home/techempower/FrameworkBenchmarks/frameworks/PHP/hhvm/db.php:13]

 - Google a bit, come across a GitHub issue talking about a similar
   problem when they used "SET NAMES"
 - Comment the part of PHP/hhvm/once.php.inc that uses SET NAMES, run
   test again and it worked, except for fortunes which failed because of
   character encoding problems
 - Google for a different way to specify utf8 in the PDO constructor,
   try it again, all tests pass locally
@msmith-techempower msmith-techempower mentioned this issue Feb 23, 2018
Merged
NateBrady23 pushed a commit that referenced this issue Apr 14, 2018
@msmith-techempower @NateBrady23
[ci skip] Attempt #2 at .gitattributes (#3564)
bb783a0
michaelhixson added a commit to michaelhixson/FrameworkBenchmarks that referenced this issue Apr 23, 2018
@michaelhixson
Fix an uncaught exception during dart startup
2e2fec3 
Here's an example of the error output from before this change:

Unhandled exception:
Bad state: Stream was already listened to
#0      _RawServerSocket.listen (dart:io-patch/socket_patch.dart:1106)
TechEmpower#1      new _ForwardingStreamSubscription (dart:async/stream_pipe.dart:123)
TechEmpower#2      _ForwardingStream._createSubscription (dart:async/stream_pipe.dart:91)
TechEmpower#3      _ForwardingStream.listen (dart:async/stream_pipe.dart:86)
TechEmpower#4      _ServerSocket.listen (dart:io-patch/socket_patch.dart:1351)
TechEmpower#5      _HttpServer.listen (dart:io/http_impl.dart:2278)
TechEmpower#6      _startServer.<anonymous closure> (file:///server.dart:88:12)
TechEmpower#7      _RootZone.runUnary (dart:async/zone.dart:1371)
TechEmpower#8      _FutureListener.handleValue (dart:async/future_impl.dart:129)
TechEmpower#9      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:636)
TechEmpower#10     _Future._propagateToListeners (dart:async/future_impl.dart:665)
TechEmpower#11     _Future._completeWithValue (dart:async/future_impl.dart:478)
TechEmpower#12     Future.wait.<anonymous closure> (dart:async/future.dart:362)
TechEmpower#13     _RootZone.runUnary (dart:async/zone.dart:1371)
TechEmpower#14     _FutureListener.handleValue (dart:async/future_impl.dart:129)
TechEmpower#15     _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:636)
TechEmpower#16     _Future._propagateToListeners (dart:async/future_impl.dart:665)
TechEmpower#17     _Future._completeWithValue (dart:async/future_impl.dart:478)
TechEmpower#18     Future.wait.<anonymous closure> (dart:async/future.dart:362)
TechEmpower#19     _RootZone.runUnary (dart:async/zone.dart:1371)
TechEmpower#20     _FutureListener.handleValue (dart:async/future_impl.dart:129)
TechEmpower#21     _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:636)
TechEmpower#22     _Future._propagateToListeners (dart:async/future_impl.dart:665)
TechEmpower#23     _Future._completeWithValue (dart:async/future_impl.dart:478)
TechEmpower#24     _Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:510)
TechEmpower#25     _microtaskLoop (dart:async/schedule_microtask.dart:41)
TechEmpower#26     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50)
TechEmpower#27     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:99)
TechEmpower#28     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:152)
NateBrady23 pushed a commit that referenced this issue Apr 24, 2018
@michaelhixson @NateBrady23
Fix an uncaught exception during dart startup (#3640)
04dd98d 
Here's an example of the error output from before this change:

Unhandled exception:
Bad state: Stream was already listened to
#0      _RawServerSocket.listen (dart:io-patch/socket_patch.dart:1106)
#1      new _ForwardingStreamSubscription (dart:async/stream_pipe.dart:123)
#2      _ForwardingStream._createSubscription (dart:async/stream_pipe.dart:91)
#3      _ForwardingStream.listen (dart:async/stream_pipe.dart:86)
#4      _ServerSocket.listen (dart:io-patch/socket_patch.dart:1351)
#5      _HttpServer.listen (dart:io/http_impl.dart:2278)
#6      _startServer.<anonymous closure> (file:///server.dart:88:12)
#7      _RootZone.runUnary (dart:async/zone.dart:1371)
#8      _FutureListener.handleValue (dart:async/future_impl.dart:129)
#9      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:636)
#10     _Future._propagateToListeners (dart:async/future_impl.dart:665)
#11     _Future._completeWithValue (dart:async/future_impl.dart:478)
#12     Future.wait.<anonymous closure> (dart:async/future.dart:362)
#13     _RootZone.runUnary (dart:async/zone.dart:1371)
#14     _FutureListener.handleValue (dart:async/future_impl.dart:129)
#15     _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:636)
#16     _Future._propagateToListeners (dart:async/future_impl.dart:665)
#17     _Future._completeWithValue (dart:async/future_impl.dart:478)
#18     Future.wait.<anonymous closure> (dart:async/future.dart:362)
#19     _RootZone.runUnary (dart:async/zone.dart:1371)
#20     _FutureListener.handleValue (dart:async/future_impl.dart:129)
#21     _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:636)
#22     _Future._propagateToListeners (dart:async/future_impl.dart:665)
#23     _Future._completeWithValue (dart:async/future_impl.dart:478)
#24     _Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:510)
#25     _microtaskLoop (dart:async/schedule_microtask.dart:41)
#26     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50)
#27     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:99)
#28     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:152)
greenlaw110 added a commit to greenlaw110/FrameworkBenchmarks that referenced this issue Jun 9, 2018
@greenlaw110
[ci fw-only Java/act] Act: tune TechEmpower#2 for pgsql update tests
8cf2878
NateBrady23 pushed a commit that referenced this issue Jun 12, 2018
@greenlaw110 @NateBrady23
[ci fw-only Java/act] Act: tune #2 for pgsql update tests (#3850)
9b0368d 
* [ci fw-only Java/act] Act: tune #2 for pgsql update tests

* use HikariCP with Eclipselink; use druid with ebean
@simonhf simonhf mentioned this issue Jun 26, 2018
Closed
roberthusak referenced this issue in roberthusak/FrameworkBenchmarks Nov 6, 2018
@greenlaw110 @roberthusak
[ci fw-only Java/act] Act: tune #2 for pgsql update tests (TechEmpowe…
a35083a 
…r#3850)

* [ci fw-only Java/act] Act: tune #2 for pgsql update tests

* use HikariCP with Eclipselink; use druid with ebean
@ww9 ww9 mentioned this issue Feb 4, 2019
Closed
@jcheron jcheron mentioned this issue Oct 16, 2019
Merged
@NateBrady23 NateBrady23 mentioned this issue Oct 21, 2019
Closed
@matt-42 matt-42 mentioned this issue Apr 27, 2020
Closed
@synopse synopse mentioned this issue Sep 4, 2023
Open
@joanhey joanhey mentioned this issue Aug 24, 2024
Closed
msmith-techempower pushed a commit that referenced this issue Jan 16, 2025
@smthing
Merge pull request #2 from TechEmpower/master
6e51629 
aa
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@fbernier