Hook on lines is not enough. This code is a single line, but loops forever:
while true do end
You need to set an instruction count hook. If I'm not mistaken, that would be
lua_sethook(Lua, &LineHookFunc, LUA_MASKCOUNT, NUM_INSTRUCTIONS);
You want to pick NUM_INSTRUCTIONS so it is not too small (or you get a performance overhead) and not too large (or you'll wait too long until stop).
For related Lua-only implementation please see this library.
If you want to add more protection from untrusted Lua code, google for "Lua sandboxing" -- there is more to it than just infinite loop prevention.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…