balancer_by_lua by splitice · Pull Request #7 · openresty/stream-lua-nginx-module

splitice · 2016-02-04T05:01:28Z

Here is my first attempt at porting balancer_by_lua from the http module to stream.

TODO:

Implement ngx_stream_lua_ffi_balancer_get_last_failure, if its possible?

If you have any comments, please dont hesitate. This is an early pull request to facilitate that discussion. I have tried to make it a direct conversion, and used a few string replacements where possible, some comments are hence a bit silly.

I have also thrown #if (NGX_DEBUG) around the unused ngx_connection_t's that where annoying me with errors (and with the right flags, warnings). I can take them out, but I guess they are a plus?

agentzh · 2016-02-04T05:18:04Z

@splitice Thanks for looking into this. Will you port the tests too?

splitice · 2016-02-04T05:24:02Z

I dont have an environment to run them, but a bit of regex would probably do the configuration changes necessary. The error messages would need to be done manually I guess.

I'd be happy to do that much.

agentzh · 2016-02-04T07:26:57Z

@splitice There are perl scripts under util/ that can do most of the modifications needed when porting the tests.

splitice · 2016-02-04T08:41:06Z

I'll take a look, its probably not dissimilar to what I whipped up.

Currently looking for the cause of a bug, s->connection->log is null in the balancer context causing a segfault when any error occurs or with ngx.log.

splitice · 2016-02-04T23:21:57Z

Fix stream session session passing, now it works. Successfully set the current peer to a dynamic value.

agentzh · 2016-02-04T23:45:59Z

src/ngx_stream_lua_balancer.c

+{
+    lua_State                          *L;
+    ngx_int_t                           rc;
+    ngx_stream_session_t                 *r;


Need to run the ngx-releng script to catch obvious NGINX coding style issues like this :) See

https://github.com/openresty/nginx-devel-utils

Doesn't tell me anything for some reason.

# ./ngx-releng ======================================= # ls -las src 0 lrwxrwxrwx 1 root root 30 Feb 4 19:37 src -> ../stream-lua-nginx-module/src # ls -las src/ | head -n 6 total 1064 4 drwxr-xr-x 3 root root 4096 Feb 3 19:14 . 4 drwxr-xr-x 6 root root 4096 Feb 3 19:21 .. 4 drwxr-xr-x 2 root root 4096 Feb 3 19:14 api 4 -rw-r--r-- 1 root root 771 Feb 3 19:13 ddebug.h 16 -rw-r--r-- 1 root root 15790 Feb 4 18:15 ngx_stream_lua_balancer.c

@splitice Found a lot of things on your PR's branch on my side:

agentzh@fedora64 ~/git/stream-lua-nginx-module (master)$ which ngx-releng ~/git/nginx-devel-utils/ngx-releng gentzh@fedora64 ~/git/stream-lua-nginx-module (master)$ g hub pr-fetch 7 remote: Counting objects: 42, done. remote: Compressing objects: 100% (42/42), done. g remote: Total 42 (delta 22), reused 0 (delta 0), pack-reused 0 Unpacking objects: 100% (42/42), done. From github.com:openresty/stream-lua-nginx-module * [new ref] refs/pull/7/head -> PR/7 agentzh@fedora64 ~/git/stream-lua-nginx-module (master)$ g co PR/7 Switched to branch 'PR/7' agentzh@fedora64 ~/git/stream-lua-nginx-module (PR/7)$ ngx-releng ======================================= src/ngx_stream_lua_balancer.h 14:ngx_int_t ngx_stream_lua_balancer_handler_inline(ngx_stream_session_t *s, ngx_log_t *r, 17:ngx_int_t ngx_stream_lua_balancer_handler_file(ngx_stream_session_t *s, ngx_log_t *r, src/ngx_stream_lua_balancer.c 174: p = ngx_copy(p, NGX_STREAM_LUA_INLINE_TAG, NGX_STREAM_LUA_INLINE_TAG_LEN); 220: bp = ngx_pcalloc(r->connection->pool, sizeof(ngx_stream_lua_balancer_peer_data_t)); 333:ngx_stream_lua_balancer_by_chunk(lua_State *L, ngx_log_t *log, ngx_stream_session_t *s) src/ngx_stream_lua_common.h 118:typedef ngx_int_t (*ngx_stream_lua_srv_conf_handler_pt)(ngx_stream_session_t *s, ngx_log_t *log, src/ngx_stream_lua_balancer.c 547: //ngx_stream_upstream_state_t *state; src/ngx_stream_lua_phase.c 60: 61: src/ngx_stream_lua_balancer.c 532: src/ngx_stream_lua_module.c 107: src/ngx_stream_lua_common.h 117: 165: 232: src/ngx_stream_lua_balancer.c 533: *err = NULL; 544: /* NOT PORTED: IS IT POSSIBLE? */ src/ngx_stream_lua_common.h 233: struct { t/123-lua-path.t 19:master_on(); t/servroot/logs/error.log t/cert/equifax.crt t/cert/startcom.crt src/api/ngx_stream_lua_api.h 22:#define ngx_stream_lua_version 1 src/ngx_stream_lua_phase.c: found unnecessary tail space 60: found unnecessary tail space 61: src/ngx_stream_lua_balancer.c: incorrect front space, unclosed bracket 60: lscf->balancer.src.data, incorrect front space, unclosed bracket 80: lscf->balancer.src.data, variable name should align with the previous line 121: ngx_stream_lua_srv_conf_t *lscf = conf; incorrect front space, unclosed bracket 144: value[1].len); variable name should align with the previous line 249: ngx_stream_session_t *r; variable name should align with the previous line 413: ngx_stream_lua_ctx_t *ctx; found unnecessary tail space 532: src/ngx_stream_lua_module.c: found unnecessary tail space 107: src/ngx_stream_lua_common.h: found unnecessary tail space 117: found unnecessary tail space 165: found unnecessary tail space 232: incorrect front space, level indent 234: ngx_str_t src; done.

@splitice Are you sure your nginx-devel-utils clone is up to date? Also, better add the path of your nginx-devel-utils directory to your PATH environment.

@splitice The git master is clean BTW:

agentzh@fedora64 ~/git/stream-lua-nginx-module (master)$ ngx-releng ======================================= t/123-lua-path.t 19:master_on(); t/servroot/logs/error.log t/cert/equifax.crt t/cert/startcom.crt src/api/ngx_stream_lua_api.h 22:#define ngx_stream_lua_version 1 done.

Its the latest, I just cloned it now.

I just created a symlink to src in the directory out of lazyness so the scripts are in the cwd/pwd.

…ithout --with-debug. thanks Mathew Heard for the report in #7.

Rebase against upstream

…rn status 0 always.

splitice · 2016-02-05T01:14:53Z

There is a bunch of style fixes

agentzh · 2016-02-05T05:15:42Z

@splitice The ngx-releng tool does not follow symlinks BTW.

agentzh · 2016-02-05T05:16:44Z

@splitice Thanks for your quick updates.

@doujiang24 Please have a look at this PR when you have a chance. Thanks!

daxiong380380 · 2016-06-28T07:21:34Z

@splitice
Could you give me your balancer_by_lua source code,I'd like to have a look.
Thank you.

splitice · 2016-06-28T07:24:53Z

This is a pull request for the code.

daxiong380380 · 2016-06-28T07:37:07Z

@splitice
The pull request has not been accepted,has it ?
How can I get your source code?

splitice · 2016-06-28T08:15:57Z

Its in my branch - https://github.com/splitice/stream-lua-nginx-module

daxiong380380 · 2016-06-28T11:16:52Z

@splitice
In the page of https://github.com/splitice/stream-lua-nginx-module , there is no description to tell us how to use this function,also we'd like to know the detail introduction .

daxiong380380 · 2016-06-30T01:34:04Z

@splitice
Hi,
Cloud you give me some advises to use the function of balance_by_lua,I have been trapped here for long time,please help me.
One more question,In the balancer_by_lua_block,can I set the proxy address .

agentzh · 2016-06-30T03:01:26Z

@daxiong380380 balancer_by_lua* is officially documented in lua-nginx-module:

https://github.com/openresty/lua-nginx-module#balancer_by_lua_block

What is implemented here should be (mostly) compatible.

rshriram · 2016-07-20T13:39:21Z

@agentzh any ETA on when this PR will be merged? What needs to be done to get this merged into upstream? I can try to help, if I manage to get my head around the nginx module stuff.

agentzh · 2016-07-20T16:33:21Z

src/ngx_stream_lua_balancer.c

+    int *status, char **err)
+{
+
+	// This is not yet implemented.


Style: bad indentation and the C++-style line comments should be avoided.

agentzh · 2016-07-20T16:37:25Z

This needs to be rebased to the latest master branch before it can get merged.

Also it has various coding style issues that need to be fixed before the merge.

agentzh · 2016-07-20T16:39:46Z

@rshriram No ETA yet since it has unresolved issues (see my comments above).

agentzh · 2016-07-20T16:40:42Z

We need to pass as many test cases as possible from ngx_http_lua_module for balancer_by_lua* and ngx.balancer here.

splitice · 2016-07-21T02:15:18Z

I currently do not have any time to spend on this patch. PR's welcome to my fork.

I am happy to re-base, but without style work its frivolous currently. I just don't have time for that currently.

rshriram · 2016-08-09T17:43:06Z

@splitice @agentzh I have fixed all the styling issues.. and my branch is here: https://github.com/rshriram/stream-lua-nginx-module
Should I send a PR to @splitice so that the issue history is maintained, or shall I open up a new PR ? All of @splitice 's commit history is maintained in either case. Just want to avoid the extra hop.

agentzh · 2016-08-09T18:57:18Z

@rshriram Creating your own PR is more convenient given that @splitice is busy atm.

splitice · 2016-08-10T04:07:46Z

No concerns on my end, do whichever is easiest / Github allows :)

splitice · 2016-09-23T04:31:34Z

Closed in favour of further work by @rshriram

splitice added 3 commits February 4, 2016 14:40

Initial Commit of balancer_by_lua_* port

35e6cd7

add balancer to FFI context and fix get_phase

559a9b4

fix segfault due to miss-matched types

0729016

first part of porting tests, the configuration

14a545c

splitice mentioned this pull request Feb 4, 2016

FFI ngx_http_lua_set_req equivilent #9

Closed

splitice added 2 commits February 5, 2016 09:39

fix log passing

463a8eb

pass the stream session correctly

485ec80

splitice mentioned this pull request Feb 4, 2016

Balancer by lua stream openresty/lua-resty-core#25

Open

agentzh reviewed Feb 4, 2016
View reviewed changes

agentzh added a commit that referenced this pull request Feb 5, 2016

bugfix: fixed compilation errors in gcc and etc when nginx is built w…

36f2388

…ithout --with-debug. thanks Mathew Heard for the report in #7.

splitice added 6 commits February 5, 2016 11:25

Merge remote-tracking branch 'refs/remotes/openresty/master'

e72e37f

Merge remote-tracking branch 'refs/remotes/openresty/master'

169a8cf

Rebase against upstream

request/session r -> s

a969b82

FFI: get_last_failure comment the reason for non-implementation. Retu…

70942f9

…rn status 0 always.

Style: remove extra whitespace / newline

1527ee1

style fixes

9d3efbf

splitice added 2 commits February 5, 2016 20:14

missed semicolon

1477f6d

Merge remote-tracking branch 'refs/remotes/openresty/master'

d6252a2

Merge remote-tracking branch 'refs/remotes/openresty/master'

8d47c5c

agentzh mentioned this pull request Jun 30, 2016

UDP downstream api #22

Closed

agentzh reviewed Jul 20, 2016
View reviewed changes

rshriram mentioned this pull request Aug 10, 2016

balancer_by_lua for streams #30

Closed

splitice added 2 commits September 23, 2016 13:52

Merge remote-tracking branch 'refs/remotes/openresty/master'

32eb884

attempt 1.11.4 support

7471fdf

splitice closed this Sep 23, 2016

Conversation

splitice commented Feb 4, 2016

Uh oh!

agentzh commented Feb 4, 2016

Uh oh!

splitice commented Feb 4, 2016

Uh oh!

agentzh commented Feb 4, 2016

Uh oh!

splitice commented Feb 4, 2016

Uh oh!

splitice commented Feb 4, 2016

Uh oh!

agentzh Feb 4, 2016

Choose a reason for hiding this comment

Uh oh!

splitice Feb 5, 2016

Choose a reason for hiding this comment

Uh oh!

agentzh Feb 5, 2016

Choose a reason for hiding this comment

Uh oh!

agentzh Feb 5, 2016

Choose a reason for hiding this comment

Uh oh!

agentzh Feb 5, 2016

Choose a reason for hiding this comment

Uh oh!

splitice Feb 5, 2016

Choose a reason for hiding this comment

Uh oh!

splitice commented Feb 5, 2016

Uh oh!

agentzh commented Feb 5, 2016

Uh oh!

agentzh commented Feb 5, 2016

Uh oh!

daxiong380380 commented Jun 28, 2016

Uh oh!

splitice commented Jun 28, 2016

Uh oh!

daxiong380380 commented Jun 28, 2016

Uh oh!

splitice commented Jun 28, 2016

Uh oh!

daxiong380380 commented Jun 28, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daxiong380380 commented Jun 30, 2016

Uh oh!

agentzh commented Jun 30, 2016

Uh oh!

rshriram commented Jul 20, 2016

Uh oh!

agentzh Jul 20, 2016

Choose a reason for hiding this comment

Uh oh!

agentzh commented Jul 20, 2016

Uh oh!

agentzh commented Jul 20, 2016

Uh oh!

agentzh commented Jul 20, 2016

Uh oh!

splitice commented Jul 21, 2016

Uh oh!

rshriram commented Aug 9, 2016

Uh oh!

agentzh commented Aug 9, 2016

Uh oh!

splitice commented Aug 10, 2016

Uh oh!

splitice commented Sep 23, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

daxiong380380 commented Jun 28, 2016 •

edited

Loading