[Experimental] Luals rust-based luajit port

[CppCXY]

The current LuaJIT porting is complete. jit.on is not enabled because jit.off performs better. Feel free to try it if you're interested.

see: https://github.com/LuaLS/lua-language-server-rust

[tomlau10]

Here are some old issues which suggested porting to luajit, maybe participants inside will be interested to test this out

• Consider using LuaJIT instead of Lua54 #261
• LuaJIT port #2482
• Disappointing speed 😭 #2781

[lewis6991]

How did you test with LuaJIT? Did you remove all <close> variables?

[CppCXY]

How did you test with LuaJIT? Did you remove all <close> variables?

I replace close to defer ：

[tomlau10]

I just did a preliminary benchmark here: #2482 (comment)

• The compile time of a large lua file (~8K lines) is 40% faster 🚀
• But the textDocument/documentSymbol of that file is 100% slower 🐌 ... and currently I have no idea why 😕

[CppCXY]

I just did a preliminary benchmark here: #2482 (comment)

• The compile time of a large lua file (~8K lines) is 40% faster 🚀
• But the textDocument/documentSymbol of that file is 100% slower 🐌 ... and currently I have no idea why 😕

I have already fixed many issues related to the APIs. Please check the problems again

[tomlau10]

I have already fixed many issues related to the APIs. Please check the problems again

I checked out the latest commit LuaLS/lua-language-server-rust@6c7bab8 and rebuilt the luals-rust, now textDocument/documentSymbol is much faster than before 👍

[18:17:04.943][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [1.401]sec. {
[18:17:05.041][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.684]sec. {
[18:17:27.900][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.801]sec. {
[18:17:28.008][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.903]sec. {
[18:17:28.149][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.906]sec. {
[18:17:44.449][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.643]sec. {
[18:17:44.617][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.762]sec. {
[18:17:45.654][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.844]sec. {
[18:17:55.249][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.662]sec. {
[18:17:55.695][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.569]sec. {
[18:18:03.385][warn] [#0:resources/override_script/proto\proto.lua:189]: Method [textDocument/documentSymbol] takes [0.593]sec. {

• avg 0.789s
• but it's just on par with regular luals 0.71s 🤔 (a little bit slower to be exact)

[CppCXY]

• but it's just on par with regular luals 0.71s 🤔 (a little bit slower to be exact)

Is there any comprehensive performance report, such as a comparison of memory usage and startup time for large codebases?

[tomlau10]

Is there any comprehensive performance report, such as a comparison of memory usage and startup time for large codebases?

I don't have that currently, I only tested it on a large file (the large_file.lua generated by the python script mentioned in the above comment). Because with the compile time benchmark of a large file, we may directly see the performance gain (if any) by switching to luajit.

I might try to do a more comprehensive performance comparison on some of my team's projects later 🤔

[CppCXY]

By conducting startup tests on the luals project itself, the manually measured results are roughly as follows. It seems to be getting slower and slower, possibly due to my own limited skills. I look forward to others' tests. I will go back to working on my own language server.

language server	memory	start time
luals	300mb	10s
luals-rust-lua5.4	315mb	10s
luals-rust-luajit	330mb	13.86s

[tomlau10]

By conducting startup tests on the luals project itself

May I ask how did you conduct the start time test? By checking the timestamp in the log file? 😕

• I tried to open a new vscode windows with luals project workspace (without opening any file),
• then open the main.lua the project to let luals startup in this workspace.
• Afterwards measure the timestamp between the following 2 logs:

[09:24:19.327][info] [#0:script\workspace\workspace.lua:38]: Workspace init root: 	file:///c%3A/Users/TomLau/lua-language-server
...
[09:24:21.968][warn] [#0:script\proto\proto.lua:189]: Method [textDocument/didOpen] takes [2.414]sec. {

---

Doing 5 runs on each server setup:

Start up time (in second)

server	1	2	3	4	5	avg
luals-3.11	2.422	2.62	3.13	3.134	2.691	2.7994
luals-rust-luajit	3.856	3.749	3.367	3.542	3.852	3.6732
luals-rust-luajit-jitoff	2.3	2.456	2.95	2.114	2.404	2.4448

Memory usage (in MB)

server	1	2	3	4	5	avg
luals-3.11	152	152	156	155	142	151.4
luals-rust-luajit	175	173	179	179	185	178.2
luals-rust-luajit-jitoff	129	127	127	131	128	128.4

• I have tried to do the above test with jit.off() as well. I added it in the main.lua in luals-rust project

require "luajitCompact"
local fs      = require 'bee.filesystem'
local util    = require 'utility'
local version = require 'version'
require 'config.env'

jit.off()
jit.flush()

• ‼️ surprisingly it runs FASTER when jit.off() 🤔 maybe the startup logic of luals is not jit-friendly...
• ref for turning off jit may improve performance: https://www.cnblogs.com/zhaolaosan/p/16185249.html