August 2021 Update#
It’s been more than a month since the 8bitworkshop 3.8.0 release, and I’ve been doing lots of prototyping behind the scenes. Some of this work will end up in dead ends, but we might learn a thing or two along the way. So I thought I’d document everything here.
Code Refactoring / ES6 Modules#
ECMAScript (aka JavaScript) did not have a standardized module system until ES6, which was published in 2015. The ecosystem is still catching up. For example, some browsers still don’t support ES6 modules in web workers, and Node.js only recently added module support. TypeScript has always supported ES6 modules, but good luck getting it to consume old-style IIFE (“immediately-invoked function expression”) modules, or getting it to play nicely with your Node unit tests … etc.
To handle this hodge-podge of module formats, “bundlers” were developed. These tools eat a pile of source code, images, CSS, whatever – and regurgitate it back in whatever format you need. The downside is that they add another level of complexity to your build system. So I’d been avoiding bundlers, and just using the output from the TypeScript compiler directly from the index.html file.
That is, until I found ESBuild. It’s simpler than other bundlers – there are no config files, everything is passed on the command line. It runs a build faster than the TypeScript compiler, which I now just use for a final type-check and to generate modules for unit tests.
Embedded IDE Mode#
In a future release, you’ll be able to embed the IDE into an IFrame on an external web page, loading and running whatever code you want. It’s handy for blog posts, documentation, etc. All your edits are sandboxed and won’t affect the rest of your saved IDE code.
I even rolled up my sleeves to code a custom Sphinx tag, which makes it easy to embed a live code editor inside of any Sphinx documentation or blog. I’ll document the API later, but for now here’s a quick demo:
View Code
#include "neslib.h"
//#link "chr_generic.s"
void main(void) {
pal_col(0,0x02); // set screen to dark blue
pal_col(1,0x14); // fuchsia
pal_col(2,0x20); // grey
pal_col(3,0x30); // white
vram_adr(NTADR_A(2,2)); // set address
vram_write("HELLO, WORLD!", 13); // write bytes to video RAM
ppu_on_all();
while (1) ;
}
WASI Support#
Almost all of 8bitworkshop’s backend tools are compiled to WebAssembly (WASM) with the Emscripten compiler. This compiler is feature-rich, but it is not designed for ABI (Application-Binary Interface) stability – that is, the JavaScript glue that binds the WASM to your program and simulates system calls may change between Emscripten releases. This makes it more difficult to keep backend tools up-to-date.
WASI is an emerging standard ABI for WebAssembly, which allows running command-line programs and modules with a standard runtime and ABI. You can run WASI executables on the command line like any other binary, or embed them in web/Node apps.
Any language that uses LLVM for code generation can in theory support WASI. You can compile C/C++ programs with wasi-sdk. Zig, Rust, and Go are all in various stages of adding support. There are a few limitations, for example C++ exceptions are not yet supported.
WASI may eventually replace the cobbled-together set of Emscripten modules we use to run C/C++ compilers and assemblers, and it’ll probably be used for any new native tools we integrate.
Hint
If you are a backend tool developer, adding a WASI target gives you a cross-platform binary that will run on almost any platform. It’ll also make it easier to integrate into 8bitworkshop!
libretro and other emulators#
We’ve talked before about compiling emulators to WebAssembly using standalone clang. But you can also compile emulators to the WASI target, allowing you to use standard library features like a virtual filesystem, memory allocation, etc.
It’s not always as simple as typing “make” however. For example, gameboycore can probably be made to work, but it’s slower than decade-old JavaScript emulators.
I looked at porting the atari800 emulator to WASI, but unfortunately it’s designed to run frame-by-frame, not cycle-by-cycle, which makes it difficult to support debugging. It’ll need big architectural changes to work.
Libretro is a standard API for emulators, and it may be possible to compile libretro modules to WASI to integrate into 8bitworkshop. I’ve done some promising experiments with MAME-2003 and VICE cores, but much work remains. It might be neccessary to add an API for debugging and cycle-by-cycle execution.
Collaborative Features#
I tried integrating the Yjs framework into the IDE, which gives you distributed data structures with peer-to-peer communication, so you can build Google Docs-like collaborative features.
It was silly easy to get a basic demo working with the CodeMirror editor, but a fully collaborative IDE is a bigger task.
I would like to hear about potential use cases – would it help in classroom settings? Please leave your comments below.
Native File System API#
An 8bitworkshop desktop app would compile to a 100MB+ downloadable bundle, and it’s only getting bigger. So I’m looking for alternatives.
Chrome has an API to access the local file system, which might satisfy many of the use cases for a desktop app. Service Workers would allow caching of the app for offline usage.
Scripting Notebook#
A missing piece of the 8bitworkshop IDE is asset creation – sprites, level maps, sounds, music, etc. The Asset Editor is very limited in what it can do, which is mostly basic pixel editing. To create full games, authors often want a workflow customized to their project.
I’m working on an interactive scripting environment that lets you build your own asset pipeline right inside the IDE. You can decode binaries, create editors, and dump the result into a C or .asm file. It’s my hope that it’ll allow you to create your own pixel editors, map editors, and more.
I don’t really want to create a new programming language, so the idea is to just use JavaScript with some libraries tacked on. You can run your scripts interactively in the browser, and get an instant preview of any exposed objects. You should also be able to run them via Node.js for command-line builds.
You’ll have bitmap and palette operations built-in, for example:
bmp = bitmap.indexed(40,192,8) // 40 x 192 indexed bitmap
bmp.palette = color.palette.from(VCS_RGB) // set palette
You can create simple UI controls like sliders, toolbars and toggle buttons:
tools = ui.select(['🖌️','🎨']) // toolbar
mirror = ui.toggle("mirror") // toggle button
You’ll build editors by building up a transaction log, and replaying the operations each time you evaluate the script. So it’s kind of a functional model.
// transaction log
log = io.mutable([])
// paint the pixels from the log
for (let e of log) {
bmp.set(e.x, e.y, fgcols[e.y]);
}
// update the log in response to mouse clicks
ui.interact(bmp, (e) => {
log.push({tool, x:e.x, y:e.y, color})
})
Then you convert your objects to binary data and write them to files:
// write bitmap to C array
output.file("bitmap.c", bmp.encode({bpp:1}))
Then your project just rebuilds automatically! That’s the idea, at least. There’s still a lot of prototyping to do, and limiting the scope is a goal – otherwise the thing becomes a general-purpose JavaScript web framework, and do we really need more of those?
Anyway, keep following this blog or @8bitworkshop for updates!