Memory Allocation

We haven't gotten to the point of this mattering yet, but it could potentially matter as we get something more substantial working. The default memory management for Lua is dynamic without any limit. For things where the user is inputting arbitrary Lua, this makes it pretty easy to muck up memory. We might want to consider a custom static allocation strategy so we can at least fail gracefully if the user puts something crazy in the device.

The process for doing that seems pretty straight forward. Just have to define a single function and pass it to the lua_newstate() call.

https://www.lua.org/manual/5.4/manual.html#lua_Alloc

@AnalogWeapon
Yeah thats really good thinking.

I was aware of this problem but kinda just have been ignoring it. Technically only a problem if it actually happens lol. I mean if you think about it this is really also try for desktop Lua but there you usually have plenty of ram.

Anyway, I do think that handling the Lua memory allocation differently in this context makes a lot of sense.

As a rough proof on concept I think it should just kill the lua script without crashing the main program. If that's not what already happens.

I think ideally there would be some way or warning the user:

• Before the file is even sent to the device. (This isnt foolproof but we can at least make an attempt at this point)
• Before they actually run out of memory

This is getting very theoretical but we might even be able to do some static analysis on the Lua files. Kinda hard to say how much memory a given program might use when it is dynamically allocating. But in this case we actually have 100% control over all runtime "inputs". Meaning the user cant really do anything in the script that isnt evident just by reading the script. Whereas with a desktop Lua script the user could ask for input and get a number like 9 billion. In our case we can know and control the minimum and maximum ranges of all inputs. So we maybe can actually take a pretty good guess about how much memory a given script might use before it even runs.

@AnalogWeapon
Also plz feel free to make PRs to the picolua project if this is the kind of thing you want to try tackling. We can move it over to the magpie github at some point. I just wasnt sure if it was even going to work when I started it.

Yeah, it would be cool to be able to look at a user script and be able to tell if it's going to use too much memory. But to start out with, I think just a simple log message of "You ran out of memory" would be good enough.

I'm not totally sure, but it seems like the way to customize how much memory Lua uses is all just about that one allocation function. I was about to say it doesn't even need to be in the repo you're making, but then I remembered it's called pico-lua, so I guess it would make sense to include a basic allocator. Like even if it's one that uses like 50% of the memory or something. Then we just document how to tweak it.