Here’s a tiny “post-mortem” on porting Freesynd to the latest Native Client SDK. This was about a days worth of tinkering and covers one of the most troublesome issues when moving a traditional application to a locked-down browser environment – mandatorily asynchronous file access, usually involving threading hilarity. All those blocking fopen()s won’t work here, matey.

This work was done on Windows, so it also covers the fiddles requires to get printf-debugging working and the joy of cygwin when working with naclports packages. If you want to just port a natty little SDL app without going near cygwin, a download of the patched SDL libraries pre-built for use in NaCl 0.5 is available from my space on GitHub - source available via naclports.

The SDK

… is pretty excellent. The Windows installer both exists and works – I say this like it’s a surprise – plus, the only other dependency is Python (for scons, the build system they’re using). Wins points for being tidy, documented and easy to use – one can have examples running within minutes.

Envvars & Chrome Canary

I recommend setting the NACL_SDK_ROOT environment variable, pointing it at your install directory. This is used in the NaClPorts build system, mentioned next.

While you’re messing with envvars (can I recommend RapidEE for the job? It’s excellent.) take a moment to set NACL_EXE_STDOUT and NACL_EXE_STDERR too. These are required if you wish to get access to any stdout (printf) or stderr output from your application on Windows. Given the presently limited reach of NaCl debugging, you are most definitely going to want to do this. Set them to be absolute paths to files and remember they are not deleted or purged between Chrome sessions, output is simply appended.

There are a few other variables that enable extremely verbose debug output from the NaCl runtime itself – in practice I didn’t find this very useful and it mounts up quickly. After an hour or so of running some tests, I saw that the log had grown to 1.5GB. Turned it off.

And then there is Chrome itself – I chose to install Chrome Canary, the absolute bleeding edge. Because this version is considered experimental and volatile, it installs side-by-side with your existing browser so you don’t have to switch your main Chrome install to the Dev branch, or change any NaCl-y settings in the one you use day-to-day.

This is particularly important on Windows, as to complete the process of getting stdout / stderr output from the runtime you have to run Chrome with the ‘--no-sandbox‘ command line, which, as it tells you every time it runs :

SDL and NaClPorts

The naclports repository has a sizeable array of pre-ported packages ready to build and use – from libpng, zlib, gmock and SDL all the way to Boost, the patch file for which is 240KB big. Good times.

Things fall apart slightly here, at least they do on Windows with SDK 0.5. I had problems getting the package builder script to run successfully and ended up having to make a few small changes – adding ${NACL_SDK_ROOT} to convert a few paths to absolutes and short-circuiting the Check() function which compares SHA1 hashes to downloaded packages. I guess perhaps the hashes haven’t been updated?

Whatever the case, with a bit of persistence I ended up with a lengthy configure process and, finally, SDL building. Luckily for me it was one of the first libraries to be tackled, as the build script failed immediately afterwards. As I had my prize already, I haven’t investigated further.

I then repeated the build with the NACL_PACKAGES_BITSIZE variable (in build_tools/common.sh) set to “64″ so some nacl64 libraries were also created. I couldn’t see the build script building both library versions itself, so I think this is a necessary step; note you can also set that as an environment variable, should you wish.

Or download my pre-built ones above.

The Main Thread

There is no main() function in a NaCl module; instead, it follows a traditional plug-in model. A factory method produces an Instance for each appropriate <embed> block in a page and this instance then responds to inputs, resize events, and so on. At the point of instantiation, your code is running on the ‘Main Thread’ and there are a few rules governing that environment that need to be adhered to.

If you tie up the main thread – that is, if you sit and run a gameloop, or spend an hour raytracing – Chrome will assume the plugin has hung and alert the user to kill it. Operations on the Main Thread need to be simple or asynchronous, using threads and callbacks to work on more complex tasks.

In my case, once my instance is ready, I initialize SDL and kick off a single thread that jumps straight to what was Freesynd’s main() entrypoint.

static void* freesynd_thread(void* param)
{
  // we pass the calling instance into the app, it will need it to communicate back with the browser
  SyndicateInstance* ii = (SyndicateInstance*)param;
  freesynd_main(ii);

  return 0;
}

bool SyndicateInstance::Init(uint32_t argc, const char* argn[], const char* argv[])
{
  // these SDL setup functions must happen on the main thread. apparently.
  SDL_NACL_SetInstance(pp_instance(), GAME_SCREEN_WIDTH, GAME_SCREEN_HEIGHT);
  SDL_Init(SDL_INIT_VIDEO);
  ...
  pthread_create(&(data.freesynd_main_thread), NULL, freesynd_thread, this);
  ...

This lets the synchronous game code tick along in the background. The Main Thread pushes any received events (mouse moves, key presses) into SDL which enqueues them for processing at the next update tick.

The next restriction is callback execution – specifically, the callbacks used when fetching file data can only execute on the Main Thread. Code running on the worker thread that is relying on blocking fopen() calls will need to be changed, which brings us to…

File Access

Unless you’re already merrily streaming data into your application asynchronously, it’s likely you’ll run into some problems when porting to NaCl. Your code is now running in a web browser sandbox – it has no local directory full of files, no ‘default’ set of content, data is retrieved by the browser and can take time to arrive.

The path I took was the following, more or less. Brace for diagrams.

On the game thread, a single static file request structure is used to communicate with the main thread. This holds the name of the file to open, the instance (as passed into the freesynd_main() function above), a buffer container to be filled and a semaphore. The semaphore is initialized to 0 as we need to immediately block on it until the request is filled.

I then use the NaCl Core CallOnMainThread function to kick off the request. The function passed creates a URLLoader (used to actually pull data from a given URL) and a ResponseHandler (which handles the callback called when the file has arrived) and initiates the download. At this point, the game thread is paused, blocked on the semaphore. The main thread is mostly idle, as the data streaming is done asynchronously.

When the response callback fires, the fetched data is copied into the request buffer container and the semaphore is incremented, causing the game thread to resume. Job done.

File Access – Bonus Round

There are two other things to note about loading files like this. If you have a manifest of all the files required to run your code (such as it is with the Quake demo and its single pak file) then an alternative to all this fuss is just pre-loading upfront. A file load in the game thread simply becomes a look-up into a map of existing data.

The other extra feature that is now possible with the latest NaCl SDK is caching data with the HTML5 File System API, pretty much next on my list to look at.

The Rest

Things were getting quite far along by this point; other problems arrived when SDL_Image was used, as it uses its own file access functions which needed to be patched too. Spoiler: I didn’t do anything nice – seeing that it was only loading a single PNG, I just turned the image into a header (BIN2C) and pushed it through as a SDL memory resource.

I created an array of buttons on the page to supply cheat codes on-demand. To communicate from the browser to the game, a simple lockless queue was used – JavaScript functions on the page send messages (as strings) to the instance, which in turn post those messages into a queue shared with the game thread.

Addendum

Jamie Fristrom over at GameDevBlog brought this to my attention: use ‘--jobs #‘ when running scons to help parallelize those NaCl builds.

Finally, don’t forget to strip! The scons build process doesn’t do this by default and it took about 4MB off my SyNaCl executables, so definitely worth it before deploying live. Hopefully the NaCl team will also add the ability to deploy compressed executables too, as zipping up the stripped results shrunk them down by another 3 or 4 times.

nacl-strip <path to .nexe>

will do it, assuming the toolchain\win_x86\bin directory is in your path.