Control Nginx SSL/SNI with Lua

[miracle2k]

I'd like to run some LUA to support multiple frontends with different SSL certificates, ideally via SNI. I will need to execute my Lua code before the SSL is decoded (and make it use the correct certificate).

I gather this is not supported?

[agentzh]

@miracle2k No. This requires patching nginx I guess :)

[miracle2k]

I was hoping that nginx might have a plugin hook that lua-nginx could expose.

Of course, if this isn't a thing that realistically is going to happen, we can close the ticket.

[bearnard]

I'd really have liked this to be possible without patching nginx.

@agentzh could you elaborate a bit?
I'm assuming there is no exposed "phase" during ssl negotiations that nginx-lua could tap into?

There are alternative "ways" of doing similar things outside of nginx e.g: https://github.com/dlundquist/sniproxy

[agentzh]

@bearnard Yeah, it's good to have. Maybe we could ask the Nginx team to add a hook for us :)

[rkearsley]

Hi
I'm very interested in this too. The current solution of having to have a separate server {} in the nginx config is very tedious to automate since nginx has to be restarted for every change.
I would much prefer to automate this with Lua!

I will start a thread on nginx mailing list to hopefully get an nginx hook added.. It will just highlight the problem, hopefully you guys can add your support and suggestions in response to it..

Thanks
Richard

[agentzh]

@rkearsley yeah we need more voices from the community. I actually talked about it with some guys on the nginx team in person a few weeks ago. But to change their priority, we really need more voices.

[Shananra]

I need this feature as well. It would be nice to just drop the certificate and key files into a specified directory and have lua just start using them without using scripts to rebuild the config and restart nginx, which is what I'm currently doing.

[agentzh]

@Shananra I'll try to patch the nginx core to have the hooks required by ngx_lua (and other nginx C modules) :)

[agentzh]

I do have plans to add hooks to the nginx core and implement directives like ssl_verify_by_lua and ssl_verify_by_lua_file in ngx_lua in the next 6 months or so.

[Shananra]

@agentzh Awesome. Thanks a ton for doing that.

[agentzh]

The directive names are going to be ssl_certificate_by_lua and ssl_certificate_by_lua_file. I've got my first nontrivial test case working locally. I'll continue maturing it in the next 5 months or so.

[bearnard]

\o/

[bearnard]

@agentzh let me know if I can help out with testing etc...

[agentzh]

@bearnard Sure, I'll let you know.

[chrisboulton]

@agentzh We're very excited by the possibility of this too. I'd definitely be interested in helping out or testing in any way I can.

[agentzh]

@chrisboulton Thanks! I'll post things here when I'm ready.

[jperla]

Would it be possible to integrate a keyless oracle too? as in http://arstechnica.com/information-technology/2014/09/in-depth-how-cloudflares-new-web-service-promises-security-without-the-key/

[agentzh]

@jperla That's another story and requires a patched version of OpenSSL. Whether CloudFlare is going to opensource the keyless stuff is not determined yet AFAIK.

[andrea-spoldi]

+1

[agentzh]

FYI, the feature that adds the ssl_certificate_by_lua and ssl_certificate_by_lua_file directives will be opensourced in ngx_lua in early March, as originally planned. It has already been driving CloudFlare's global SSL gateway for a few months now.

[chrisboulton]

@agentzh Still on track for March? We're very keen to start digging in and utilizing ssl_certificate_by_lua when the changes are released. It's going to make our lives so much easier. ✨

[agentzh]

@chrisboulton Yes, sure. I'm trying to opensource it on 1 March.

[petethepig]

+1

[andrea-spoldi]

@agentzh any updates ?

[agentzh]

@andrea-spoldi It's taking longer than I wanted. Still working on it.

[agentzh]

Okay, I'm very sorry that I don't have enough time to polish up our internal branch for ssl_certificate_by_lua to make it ready for merging into master. So I've decided to opensource the internal branch ssl-cert-by-lua directly on GitHub for the public review and testing for the time being:

https://github.com/openresty/lua-nginx-module/tree/ssl-cert-by-lua

And there's no documentation but we have comprehensive test cases for ssl_certificate_by_lua:

https://github.com/openresty/lua-nginx-module/blob/ssl-cert-by-lua/t/130-ssl-cert-by.t

Check the test cases for various usage. They're declarative and precise :)

A lot of existing Lua APIs are available in this new context, including ngx.timer.at, ngx.sleep, ngx.exit, and the cosocket API. Everything is nonblocking :)

BTW, this ssl_certificat_by_lua hook is for the SSL certificate serving phase only. It cannot be used to control the SSL ID-based session management (like implementing a memcached-based global-sharing SSL session cache), which will be fulfilled by the ssl_session_fetch_by_lua and ssl_session_store_by_lua directives in the future (already in CloudFlare's internal use, expected to be opensourced in another 6 months).

To make it work, you need to

1. use the GitHub branch ssl-cert-by-lua of ngx_lua,

2. build with at least OpenSSL 1.0.2. Older versions of OpenSSL will not even compile.

3. apply the following patch to your NGINX core (tested with 1.7.7+):

   https://github.com/openresty/lua-nginx-module/blob/ssl-cert-by-lua/patches/nginx-ssl-cert.patch

   Eventually we'll try to eliminate this nginx patch by upstreaming the changes.

4. Pass the ./configure option --with-http_ssl_module while building nginx.

5. Add the path of the lua/ directory to your lua_package_path setting:

   https://github.com/openresty/lua-nginx-module/tree/ssl-cert-by-lua/lua/

   This is because the Lua API is implemented there as the ngx.ssl Lua module in FFI only (for now). The Lua API allows one to

   1. inspect the SNI name and the server IP address being accessed,
   2. set OCSP stapling for the current connection being (SSL) handshaked,
   3. set SSL certificate chain and private key for the current connection being (SSL) handshaked,
   4. construct OCSP request and parse/validate OCSP response.

Let me know if you have any further problems.

There's still a lot of work before this branch can be merged into master and become part of a release. Anyway, hopefully you'll find this useful and helpful.

Feedback welcome!

[ajayk]

:+1

[dol]

Much appreciated

[andrea-spoldi]

@agentzh thanks for it.
Though build is failing for me, anyone else tried already ?

this is what I got
/home/ubuntu/test_nginx_ssl/ngx_lua-0.9.15/src/ngx_http_lua_sslcertby.h:33:35: error: unknown type name ‘ngx_ssl_conn_t’
int ngx_http_lua_ssl_cert_handler(ngx_ssl_conn_t *ssl_conn, void *data);
^

[petethepig]

@andrea-spoldi Make sure you're compiling nginx with --with-http_ssl_module

[syzh]

编译ssl-cert-by-lua分支下的lua-module报错

objs/ngx_modules.o
-L/usr/local/lib -Wl,-E -lpthread -lcrypt -L/usr/local/lib -lluajit-5.1 -lm -ldl /var/work/test/soft/pcre-8.30/.libs/libpcre.a -lssl -lcrypto -ldl /var/work/test/soft/zlib-1.2.8/libz.a
objs/addon/src/ngx_http_lua_module.o: In function ngx_http_lua_merge_srv_conf': /var/work/lua-nginx-module/src/ngx_http_lua_module.c:837: undefined reference toSSL_CTX_set_cert_cb'
objs/addon/src/ngx_http_lua_sslcertby.o: In function ngx_http_lua_ffi_ssl_clear_certs': /var/work/lua-nginx-module/src/ngx_http_lua_sslcertby.c:481: undefined reference toSSL_certs_clear'
objs/addon/src/ngx_http_lua_sslcertby.o: In function ngx_http_lua_ffi_ssl_set_der_certificate': /var/work/lua-nginx-module/src/ngx_http_lua_sslcertby.c:542: undefined reference toSSL_add0_chain_cert'
collect2: ld returned 1 exit status
make[1]: *** [objs/nginx] Error 1
make[1]: Leaving directory `/var/work/test/soft/nginx-1.9.8'
make: *** [build] Error 2

openssl 已经安装OpenSSL 1.0.1 14 Mar 2012
/patches/nginx-ssl-cert.patch 补丁已经打了
configure 选项
--prefix=/usr/local/nginx-1.9.8 --with-ipv6 --with-http_ssl_module --with-http_secure_link_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_gzip_static_module --with-http_stub_status_module --with-mail --with-mail_ssl_module --with-http_mp4_module --with-pcre=/var/work/test/soft/pcre-8.30 --with-zlib=/var/work/test/soft/zlib-1.2.8 --add-module=/var/work/lua-nginx-module --with-http_v2_module

[agentzh]

@syzh Please, no Chinese here. This place is considered English only. Regarding your problem, you need at least OpenSSL 1.0.2, as specified in my previous comment above. BTW, please do not cross-post. Thanks for your cooperation.

[agentzh]

Just a quick update: I've just enabled the coroutine.* and ngx.thread.* API in the context of ssl_certificiate_by_lua* in the ssl-cert-by-lua branch mentioned above.

I'm currently working on the ssl-cert-by-lua-2 branch for cleanups and modifications before merging into master. The Lua API will change slightly in that

1. the ngx.ssl Lua module is now splitted into two modules: ngx.ssl and ngx.ocsp.
2. the ssl_certificiate_by_lua directive is gone; we introduced the new ssl_certificate_by_lua_block {} directive for inlining Lua source inside nginx.conf directly.
3. the Lua modules ngx.ssl and ngx.ocsp will move into the lua-resty-core Lua library.

The Lua API in the ssl-cert-by-lua branch will stay intact, however, to help existing users migrate slowly.

I'll also clean up the nginx core patches and propose them to the nginx-devel mailing. Hopefully we will no longer to patch the nginx core for this feature soon.

[agentzh]

Okay, this feature has just been merged into master. It will get included in the next OpenResty formal release (1.9.7.2). I'm closing this.

[chrisboulton]

Bravo! Thanks for all of the hard work you've put into this @agentzh! 👏

[leandromoreira]

thanks 👍

[subnetmarco]

I couldn't ask for a better start to the year, thanks @agentzh 👌

[Shananra]

Thank you so much for this @agentzh

[moseleymark]

Sorry, running a bit late :)

I got tied up trying to get an openssl 1.0.2e package built for Precise.

So far so good here on testing on Precise with nginx 1.9.9.

Out of curiosity, why is openssl 1.0.2e required over 1.0.2a?

[moseleymark]

Oh, and a huge thanks! :)

[agentzh]

@moseleymark Just because I didn't want to bother testing compatibility with earlier versions of OpenSSL 1.0.2.

[dol]

@moseleymark IMHO 1.0.2a should work just fine. It's always recommended to use the latest version when using from https://openssl.org/.
Btw: Instead of creating an OpenSSL distro package you could also build nginx/OpenResty with a static linked OpenSSL.
The configure options are:

--with-openssl=<path to openssl source code>
--with-openssl-opt=<optional configuration for OpenSSL>

Take a look at https://github.com/jgautheron/dockerfiles/blob/master/openresty/Dockerfile for an example.

[agentzh]

FYI, this ssl_certificate_by_lua* feature has been included in the OpenResty 1.9.7.2 RC1 version:

https://groups.google.com/d/msg/openresty-en/SZsbhGJc6D0/vJ0eHSIbCwAJ

The ssl_session_fetch/store_by_lua* feature will be opensourced soon. And we'll announce on the openresty-en mailing list when it's out :) Stay tuned.

[leandromoreira]

Thanks once more @agentzh

[favorinfo]

the test file of ssl-cert-by has changed to page https://github.com/openresty/lua-nginx-module/blob/ssl-cert-by-lua/t/139-ssl-cert-by.t @agentzh Could you change the page link in #331 (comment)

[agentzh]

@favorinfo The ssl-cert-by-lua branch is now obsolete. Switch over to the master branch (or better, the latest OpenResty formal release) instead.

[jarl-dk]

A question regarding this feature: In #331 (comment) @agentzh writes

1. apply the following patch to your NGINX core (tested with 1.7.7+):

   https://github.com/openresty/lua-nginx-module/blob/ssl-cert-by-lua/patches/nginx-ssl-cert.patch

   Eventually we'll try to eliminate this nginx patch by upstreaming the changes.

Is this still applicable? or has the patch been upstreamed to nginx core?

[agentzh]

@jarl-dk Pleas read the manual:

https://github.com/openresty/lua-nginx-module#ssl_certificate_by_lua_block

The info there is always up to date and it already answers your question. BTW, this ticket is already closed and obsolete.

[agentzh]

FYI, we've just opensourced the ssl_session_fetch_by_lua* and ssl_session_store_by_lua* directives via the following pull requests for lua-nginx-module and lua-resty-core:

• feature: implemented ssl_session_fetch_by_lua* and ssl_session_store_by_lua* #822
• feature: added the ngx.ssl.session module lua-resty-core#53

The rendered documentation for these new features can be browsed here:

• https://github.com/openresty/lua-nginx-module/tree/ssl-session#ssl_session_fetch_by_lua_block
• https://github.com/openresty/lua-resty-core/blob/ssl-session/lib/ngx/ssl/session.md#readme

[trevortao]

Hello, I have a problem when using Kong dynamic SSL:
When I follow the guide of https://getkong.org/plugins/dynamic-ssl/, and did the configuration for my api as it said:

curl -X POST http://localhost:8001/apis/cdnaas_stage1/plugins --form "name=ssl" --form "config.cert=@/home/trevor/frontdoor/kong/server.crt" --form "config.key=@/home/trevor/frontdoor/kong/server.key"

and the configuration seems to be successful, but when I access it from:
curl -i -v -k -X GET https://localhost:8443/healthcheck --header "Host:cdnaas.stage1.foo.com"

the certificate returned to me is still the certificate from Kong, not my own certificate:

[root@localhost kong]# curl -i -v -k -X GET https://localhost:8443/healthcheck --header "Host:cdnaas.stage1.foo.com"

• About to connect() to localhost port 8443 (#0)
• Trying ::1...
• Connection refused
• Trying 127.0.0.1...
• Connected to localhost (127.0.0.1) port 8443 (#0)
• Initializing NSS with certpath: sql:/etc/pki/nssdb
• skipping SSL peer certificate verification
• SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
• Server certificate:

•   subject: CN=localhost,OU=IT Department,O=Kong,L=San Francisco,ST=California,C=US
  

•   start date: Jul 12 13:19:13 2016 GMT
  

•   expire date: Aug 11 13:19:13 2016 GMT
  

•   common name: localhost
  

•   issuer: CN=localhost,OU=IT Department,O=Kong,L=San Francisco,ST=California,C=US
  

  GET /healthcheck HTTP/1.1

Could you tell me what's the problem?

my Email address: [email protected]
Thank you!

[subnetmarco]

@trevortao this is a Kong issue - I would open an issue at https://github.com/Mashape/kong/issues

[moseleymark]

Just curious if there's a roadmap for extending the dynamic SSL certificate stuff to streaming (i.e. the ngx_stream_* stuff) too.

Not being impatient :) Just trying to get a feel, since there was a project I was thinking of doing that would use that. Thanks!

[agentzh]

@moseleymark Yes, we'll get there eventually in ngx_stream_lua_module.

[WangBaoling]

Is ngx.ssl supports setting ssl_client_certificate?
I want to add multi certificates to ssl_client_certificate from database

[agentzh]

@WangBaoling No, it does not support client certificates yet.

[agentzh]

@WangBaoling Patches welcome :)

[jacobnrlto]

@bearnard Yeah, it's good to have. Maybe we could ask the Nginx team to add a hook for us :)

Hi I have some problem on this. I would like to run ssl on my docker container but it always glitch on port 80 and 443.