lua - Why is L->l_G->_defaultmeta.value.gc always NULL?

Question

I am currently trying to hack the Lua implementation of a game in order to extend the built-in methods for game modders.

In order to do so, I try to hijack a the pointer to a valid lua_State struct and register new libraries with it.

I have now tried several places / stages of the target game to intercept the program and steal lua_State from it. My first try was callin in luaL_openlib() at the very end of base_open(). This was the first time I got this null-pointer exception:

Exception thrown: read access violation.

L->l_G->_defaultmeta.value.gc was nullptr.

From the comments you can see, that Egor Sktiptunoff suggested to me moving the entry point of my hack into a user-level function. Since I know, that one of the first functions getting called is dofile(), I stole the lua_State struct from there and passed it to my DLL.

What you see here is the actual code from my injected DLL which I tried to execute at the end of base_open() and dofile() (user-level):

EXTERN_DLL_EXPORT void initialize(lua_State *L)
{
    if (initialized == true) {
        return;
    }

    initialized = true;

    lua_pushvalue(L, LUA_GLOBALSINDEX);          // Works
    luaL_openlib(L, "ext", extension_funcs, 0);  // Crashes with "L->l_G->_defaultmeta.value.gc was nullptr"
}

Below you can find the screenshot of a debug session and the location where the exception gets thrown. The lua_State object is the one that I stole and was passed to e.g. dofile. How can it be that L->l_G->_defaultmeta.value.gc is NULL at this point in time? Is there anything I can do here or is there any explanation for this?

I know that the game which I try to hack here uses a "slightly different version of Lua 5.0", but could it be that they changed the way how garbage collection works or something? Because there is ..

---

One more thing to keep in mind:

The game has Lua compiled into it. The DLL I created has its own compilation of Lua 5.0.1. There is of course a chance, that the game developers back then decided to not just "sligtly" change Lua, but instead change it a lot. I am always just assuming that all the developers did was removing some default libraries and added some other built-in functions like LOG(), WARN(), etc. It would be strange if they changed code in Lua's core - but I tell you that just so somebody who has an idea about Lua might consider this as an explanation for the exception I am getting here.

Answer 1

It really looks like your sizeof(lua_TObject) doesn't match what the game's Lua is using. A common reason for this to be different is using floats (or ints) as numbers instead of the default double. I haven't personally made this change in Lua 5.0, but it looks like it can be done by defining LUA_NUMBER as float (or int). With int/float the size should be 8, but with double it is 16 (not 12 because the double needs to be aligned).

lua_TObject consists of a type tag (tt) along with a value union. The largest thing in that union is usually the 8-byte double (or an 8-byte pointer if 64-bit, but this is 32-bit). Everything else is typically 4 bytes. Most of the stuff in your watch window looks pretty valid, but the lua_Objects look sketchy. And _defaultmeta comes directly after another lua_TObject, so if the size of lua_TObject is different, your code and the game code will disagree about the location of this member. tt for _defaultmeta should be 5 (LUA_TTABLE), but yours looks like it is probably a pointer. Also, the tt fields for top, base, and _registry indicate other types that will have a pointer type value, but the gc field indicates your code is seeing small integers instead of pointers. These are probably tt fields of adjacent lua_TObjects. On top of all that, top and base don't have the same 16-byte alignment, and these are supposed to be pointers into the same lua_TObject array.

If you continue having trouble, try examining the lua_State object you get from the game before you have modified it, and compare it to the lua_State you see in a cleanly initialized lua_State you have made yourself. Obviously, pointer addresses may differ, and the game may be setting up some contents you are not, but some inconsistencies may still jump out.

If you have access to compiled Lua chunks that the game uses, you could also look at the header information of these chunks. This will contain some information that could be helpful in matching your Lua configuration including sizeof(lua_Number).

(But I think there is some chance that changing doubles to floats could be sufficient.)

It may also help you to add debugging checks to your own Lua build that may warn you sooner in the case of any funny business, at least initially. It looks like Lua 5.0 allows for both lua_assert and api_check to be #defined for such purposes.

Good luck!