Remote Debugging for Lua on the iPhone

The biggest problem with Lua, and this probably goes for a lot of scripting languages, is that it is difficult to debug remotely. If you’re running under Windows then you have Decoda, which hooks into a local Lua process to give you a powerful debugger. Outside of that, you’re on your own.

My needs are simple so far:

1. Ability to log data from device or simulator with or without GDB running
2. Ability to trace script execution real-time
3. Minimal native code- try to keep it all in the script
4. Minimally invasive in the app

Number two is critical for dynamic languages. Compilation only catches syntax errors so scripts can die in unexpected ways at runtime, and when one does you’re left with the option of adding print statements to trace execution and figure out why/where it died. We’ve all done this before in one language or another but it’s far from a state-of-the-art technique.

The requirement for the system to work without GDB and on the actual device points to a network-based solution. The minimally invasive requirement means that we don’t want to add any UI on the client and that we need auto-discovery/configuration.

Autodiscovery

For autodiscovery the obvious choice is Bonjour. It is supported on the iPhone, Mac, and Windows and is very stable. That said, I’m going to go ahead and ignore it. My thinking is this- if I keep this all in standard TCP/UDP socket code then I can share some of the code between the client and server and do the whole thing in Lua. This means I don’t need to write a native OSX app, I only need to run a Lua script. The key getting this done quickly is luasocket, which encapsulates the standard C socket interface into a clean Lua library.

My protocol for autodiscovery is shown below:

The purpose of this sequence of events is:

1. Broadcast a PING to give the server the IP of the iPhone
2. Receive a PONG to give the iPhone the IP of the server
3. Initiate a reliable TCP connection back to the server for the bulk data

Obviously, if the script is running in a simulator then there is no need to discover the remote IP so this applies only to scripts running on the device.

To support the protocol above, only two additional functions are required in Lua, one to determine if the script is running in the simulator and one to find the Wifi IP so the initial broadcast packet can be sent.

These two functions are implemented with the following code:

// code taken from:
// http://stackoverflow.com/questions/1448411/how-to-check-for-local-wi-fi-not-just-cellular-connection-using-iphone-sdk
// http://zachwaugh.com/2009/03/programmatically-retrieving-ip-address-of-iphone/
// and a few other places

static int sfGetWifiIP(lua_State * l)
{
    struct ifaddrs *addresses;
    struct ifaddrs *cursor;
    BOOL wiFiAvailable = NO;
    NSString *address = @"error";

    if (getifaddrs(&addresses) != 0) return NO;

    cursor = addresses;
    while (cursor != NULL) {
        if (cursor -> ifa_addr -> sa_family == AF_INET) 
        {
            // Check for WiFi adapter
            if (strcmp(cursor -> ifa_name, "en0") == 0 ) {
                wiFiAvailable = YES;
                address = [NSString stringWithUTF8String:inet_ntoa(((struct sockaddr_in *)cursor->ifa_addr)->sin_addr)];
                break;
            }
        }
        cursor = cursor -> ifa_next;
    }

    freeifaddrs(addresses);
    if(wiFiAvailable){
        lua_pushstring(l, [address UTF8String]);
    } else {
        lua_pushnil(l);
    }

    return 1;
}

static int sfIsRunningInSimulator(lua_State * l)
{
    NSString *model= [[UIDevice currentDevice] model];
    NSString *iPhoneSimulator = @"iPhone Simulator";
    if([model compare:iPhoneSimulator] == NSOrderedSame){
        lua_pushboolean(l, true);
    } else {
        lua_pushboolean(l, false);
    }
    return 1;
}

These have been added to the scriptutils.mm file in the included project.

Lua Debugging Hooks

The standard Lua distribution includes a debug library to allow a script to collect basic debugging information. Callback functions can be defined that are called by several events including function calls, and line execution. We can use the “line” callback to build a basic trace function. The code below outputs a trace to stdout:

function trace(event, line)
    local s = debug.getinfo(2).short_src        
    print("TRACE:"..s..":"..line.."\n")
end
debug.sethook(trace,"l")

The documentation for the debug module can be found int he Lua docs or in Programming in Lua. It's worth noting that the code above works well as-is if you just want to dump a trace to GDB when running the app with a debugger running. For many this will be useful enough.

Server Code

The server for this system is also written in Lua. As with the rest of my Lua work, I’ve tried to keep it to the minimum code that will do anything useful. It will only allow one client to connect and runs from a terminal window. It should be cross-platform since it only requires a working Lua installation and the luasocket module.

The included project includes a zip file for CompleteLua. This is just a build of Lua with the luasocket module compiled in by default. It does not require installation of any type- it can be run in-place.

Integration

If you’ve been following the past few posts here then you already have a working Lua installation running in an app. To enable the functionality here the following changes have been made:

Download luasocket and add to project as a new target. Remove the wsocket.c and wsocket.h files as these are only used for Windows

Update the test lua target to depend on luasocket and link against it.

Add the scriptutils.h and scriptutils.mm to test lua to add the two new functions for simulator detection and Wifi IP.

Modify mcLua to initialize the new luasocket library and enable the included debug library

Modify the HelloWorld layer to initialize the script utils functions.

After creating the mcLuaManager object, run the debugsupport.lua script to add the debugging functions to the global namespace. The script object can be destroyed after running it to save memory- the functions will remain in the globals table.

Now you can add the following code to the top of the script to be debugged:

How to Use

To turn on the remote tracing in a script the following lines need to be added:

local ip,port = GetRemoteAddress()
if ip and port then
print("Remote Server: "..ip.." , "..port)
EnableTracing(ip,port)
end

The server is run on the Mac by entering the command ./CompleteLua debugserver.lua . At that point you can run your app on the device or a simulator.

The demo will show the following:

The numbers after the colons represent the current line number of the script being executed.

Conclusion

The project can be downloaded here. The zip file includes a directory called "server." This directory contains the CompleteLua binary, and a zip with the source, as well as the debugserver.lua file.

As-is, the trace functionality is pretty useful but there are obvious improvements that could be made. I'm going to try this for a while and see what ends up being the most obvious missing element. I'm not entirely sure that keeping the whole thing in Lua on the client is the correct thing to do but it's functional and it will let me work now and see if I need to move it back to the C/C++ realm. My gut tells me that more C/C++ code may be required to get all of the functionality I want in the future but I'd rather avoid a premature decision.

As always, I'd appreciate any comments or suggestions.

Comments

[Youen]

Nice article, I agree that lua clearly lacks debugging tools for portable platforms. That's why I've written a remote lua debugger too (http://cushy-code.com/grld) that should work on iPhone (never tried it on this platform so far, but it's socket based). It lacks the discovery feature you have here (the client has to be configured with the server IP and connects to it on startup), but I think it has a lot more features for debugging itself (grahical interface, etc.) I plan to integrate a performance profiler, but don't have any date for this.

For my experience, keeping the client purely written in lua was a bad idea for performance reasons (I began doing it that way). Tested on a real-time game that uses lua a lot, it was really slow (hooks are executed very often). The current version uses a hook written in C and is a lot faster (no noticeable slowdown in the game, excepted when using profiling tools that show an unavoidable overhead).

[Robert]

Youen-

Your project looks nice. A gui based on wxLua was in my plans if the need occurred but it looks like I don't need to do that now- I'm going to try yours out next time I need one.

Regarding the Lua-based debugged shown here, it works well only because the whole game isn't based solely on Lua the way I've laid it out here. If that were the case, I'd expect some pretty poor performance.

-Robert