I want to learn more about how video codecs work so someday I can grow up to be a Jedi-master Video Engineer. To aid me on this epic quest, I'm doing the one week "mini-batch" at the Recurse Center. Here's my notes from Day 3.

Lesiurely morning

I'm getting really tired from all the late nights, so I more or less took the morning to start some laundry (ut-oh...need to shuffle that a few hours ag...) and write a checkin on the Recurse Center chat program describing what I've done and plan to do today.

Blogging about LLDB and Rust

Then I immediately started doing other things! Specifically, writing this blog post about debugging Rust with LLDB on a MacOS, but I think that was a great distillation of hard to find information into a succint and approachable format. I'll call that a success overall...just not my actual goal.

Learning how to write a lisp

Over falalel lunch, I had the best conversation about how one implements user-defined functions in a homemade lisp interpreter and then how to invoke that function later. Apparently, this requires a "special form" in the AST whenever the parser encounters the function keyword. This special form captures the function's:

name
arguments
body as an AST

The whole thing is stored inside some kind of symbol table so that we can invoke it later.

Okay, so we've parsed out this funciton into a usable form. Great! Now, let's evaluate our AST and handle a function invocation. (1) Lookup the function by name in our symbol table. (2) Bind the proper tokens to the function arguments. (3) Evaluate the function body's AST as normal. I think that mostly makes sense to me, though it'd take me a while to get that working.

Port simple_decoder.c to Rust

Okay, so I have this sample C program doing exactly what I want (almost), and there's a generated crate for libvpx, so I'm going projectto start by trying to compile the crate on my system...

Make pkg-config happy

No love, apparently I need to have a file name vpx.pc ("pc" means package config maybe?) to satisfy the needs of pkg-config. Its telling me to put that file into my PKG_CONFIG_PATH environment variable. Okay, let's go see if there's a vpx.pc somewhere in my libvpx build directory.... (Note to self: someday actually learn how to use find. For now using exa --tree | grep .pc to search recursively through the build dir. It's in the top level of build. Anyway, I found it -- time to look at PKG_CONFIG_PATH... env | grep PKG gives me nothing, so...maybe I try export PKG_CONFIG_PATH=[my build dir]. Hooray!

Make Rust type checker happy

That got to a whole new set of errors. All of these errors are about mismatched types:

$ cargo build
   Compiling libvpx-native-sys v4.0.2
   Compiling vp9_simple_decoder v0.1.0
error[E0308]: mismatched types
  --> src/main.rs:22:13
   |
22 |             ffi::VPX_CODEC_MEM_ERROR => Error::Mem,
   |             ^^^^^^^^^^^^^^^^^^^^^^^^ expected u32, found enum `ffi::vpx_codec_err_t`
   |
   = note: expected type `u32`
              found type `ffi::vpx_codec_err_t`
....
error: aborting due to 8 previous errors

error: Could not compile `vp9_simple_decoder`.

To learn more, run the command again with --verbose.

Hmm, literally all of the errors are about u32 being expected by the function signature, but instead my code is giving back ffi::vpx_codec_err_t typed data. The definition of vpx_codec_err_t is:

pub type vpx_codec_err_t = Enum_Unnamed3;

...and the definition of Enum_Unnamed3 (love that name by the way) is...

pub type Enum_Unnamed3 = ::libc::c_uint;

...and the definition of c_uint from libc is...on my platform...

type c_uint = u32;

Not sure what I was expecting, but I assumed it would in fact be the type u32 the compiler seems to want. So, perhaps I could try to cast the type in my crate and keep going, OR I could try to understand why this would happen. First, I'll try to cast to just get this working and maybe later come back to learning why this type alias isn't working the way it maybe should. Wait wait wait -- I don't need to cast, I'll change the code I "borrowed" from the rustvpx companion crate to expect ffi::vpx_codec_err_t types instead of u32 since I know that these should always be the same on my platform. (What could go wrong?!). That worked!

Make ld happy

New errors now:

$ cargo build
   Compiling vp9_simple_decoder v0.1.0
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-L" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/home/user/vp9_simple_decoder/target/debug/deps/vp9_simple_decoder-761a7810146c7575.vp9_simple_decoder0.rust-cgu.o" "/home/user/vp9_simple_decoder/target/debug/deps/vp9_simple_decoder-761a7810146c7575.vp9_simple_decoder1.rust-cgu.o" "/home/user/vp9_simple_decoder/target/debug/deps/vp9_simple_decoder-761a7810146c7575.vp9_simple_decoder2.rust-cgu.o" "/home/user/vp9_simple_decoder/target/debug/deps/vp9_simple_decoder-761a7810146c7575.vp9_simple_decoder3.rust-cgu.o" "-o" "/home/user/vp9_simple_decoder/target/debug/deps/vp9_simple_decoder-761a7810146c7575" "/home/user/vp9_simple_decoder/target/debug/deps/vp9_simple_decoder-761a7810146c7575.crate.allocator.rust-cgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/home/user/vp9_simple_decoder/target/debug/deps" "-L" "/usr/local/lib" "-L" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/home/user/vp9_simple_decoder/target/debug/deps/libvpx_sys-3b31031fb7744e6c.rlib" "/home/user/vp9_simple_decoder/target/debug/deps/liblibc-731e9e00d9493fb1.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-3bdc66d380ab3ae0.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-7d37db020b297d4d.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_system-6020283742edae37.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librand-a9c96b55be5fbf12.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-cea338bc19530d4f.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-ec9c6c2125b5c0ae.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-b32efaa792018eef.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-010885f4927e31a8.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd_unicode-d8d07f9be800bfd6.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-f4176b2f23d80db2.rlib" "/home/user/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-6e4af26e08893557.rlib" "-l" "vpx" "-l" "m" "-l" "System" "-l" "resolv" "-l" "pthread" "-l" "c" "-l" "m"
  = note: ld: library not found for -lvpx
          clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: aborting due to previous error
error: Could not compile `vp9_simple_decoder`.
To learn more, run the command again with --verbose.

Looks like the system linker ld can't find the library vpx ("ld: library not found for -lvpx"). That's probably a think I can duckduckgo.... The first few results are not super helpful. Okay, let's actually try to grok that command cargo spat out:

cc - I know this is a c compiler (no idea how I know this), probably an apple version of gcc..yep.
-L "/path/to/rust/library" - looks like we're telling cc about other libraries to link against by providing the full path. That is most of the monstruously large error message above.
-l vpx - I'd venture a guess this is a library but its one that the linker or compiler is supposed to just know where it is.

Okay, lets figure out how cc uses the -l flag for sure...nope, there's no entries for this in man cc. Apparently, this is undocumented feature. Okay, well cargo error says something about ld so lets try man ld and looking for something about where it looks for libraries (my hypothesis for what -l vpx means). Sure ensure, there's a description of how this works:

$ man ld
...

   Search paths
     ld maintains a list of directories to search for a library or framework to use.  The default library search path is /usr/lib then /usr/local/lib.  The -L
     option will add a new library search path.  The default framework search path is /Library/Frameworks then /System/Library/Frameworks.  (Note: previously,
     /Network/Library/Frameworks was at the end of the default path.  If you need that functionality, you need to explicitly add -F/Network/Library/Frameworks).
     The -F option will add a new framework search path.  The -Z option will remove the standard search paths.  The -syslibroot option will prepend a prefix to all
     search paths.

So, if I know which lib is the library file for vpx, I can stick that file in /usr/lib or one of the other directories mentioned. Okay, wow I hate not knowing how C works at all. Phew...the rust libraries mostly have an rlib extension. Maybe there's an rlib in the libvpx/build directory? No, not even one file. Okay, maybe "dylib"? Nope. What about just things that have "lib"? Yep -- there's a bunch .a files, like libvpx.a. Hmm, that sounds plausible. What does .a even mean? I know .o is for object files produced during a C compilation, whatever those are... Okay, well some guy named Mike has this on his blog about .a files and compiling C:

There are other kinds of files as well, notably libraries (".a" files) and shared libraries (".so" files), but you won't normally need to deal with them directly.

Oh, yeah! I've totally worked with .so files on linux before. A file ending in ".a" on MacOS is like a ".dll" for Windows or a ".so" for Linux. TIL!

Not pwning my own system directories

Okay, let's copy that libvpx.a into /usr/lib and see what happens...Uh, apparently thats not okay with Apple:

cp: /usr/lib/libvpx.a: Operation not permitted

There's a lot of symlinks in there, maybe I can just symlink libvpx.a? Nope, same result.

I really feel like I should be able to do this, so there's something specific about that directory I don't understand. Sure enough, there's a great explanation of how Apple has deliberately locked down that directory to keep us all safe. There's also an alternative here. Use the DYLD_LIBRARY_PATH environment variable instead. Let's give that a whirl. No difference whatsoever. Same result we started with.

Hmm, maybe let's try a different approach. Maybe I could get more info out of cargo by using the verbose flag. Let's try:

$ cargo build -v
       Fresh libc v0.2.35
       Fresh pkg-config v0.3.9
       Fresh semver-parser v0.7.0
       Fresh libvpx-native-sys v4.0.2
   Compiling vp9_simple_decoder v0.1.0 (file:///Users/bryce/programming/vp9_simple_decoder)
     Running `rustc ...`
error: linking with `cc` failed: exit code: 1
  |
  ...
  = note: ld: library not found for -lvpx
          clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: aborting due to previous error
error: Could not compile `vp9_simple_decoder`.
Caused by:
  process didn't exit successfully: `rustc --crate-name vp9_simple_decoder src/main.rs --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=761a7810146c7575 -C extra-filename=-761a7810146c7575 --out-dir /Users/bryce/programming/vp9_simple_decoder/target/debug/deps -L dependency=/Users/bryce/programming/vp9_simple_decoder/target/debug/deps --extern vpx_sys=/Users/bryce/programming/vp9_simple_decoder/target/debug/deps/libvpx_sys-3b31031fb7744e6c.rlib --extern libc=/Users/bryce/programming/vp9_simple_decoder/target/debug/deps/liblibc-731e9e00d9493fb1.rlib -L native=/usr/local/lib` (exit code: 101)

Okay, now that rustc invocation at the bottom also has -L native=/usr/local/lib in it....I can definitely put anything in that directory I want to. Let's try it!

$ ln -s /home/user/libvpx/build/libvpx.a /usr/local/lib/libvpx.a
$ cargo build
   Compiling vp9_simple_decoder v0.1.0 (file:///Users/bryce/programming/vp9_simple_decoder)
    Finished dev [unoptimized + debuginfo] target(s) in 0.66 secs

Woohoo! I compiled the thing. Oh man that's exciting. Time for pizza and human contact...I'll see what Andy's up to with his Rust lisp interpreter.

Porting a few lines of simple_decoder

Okay, I've tried to do a mechanical port of simple_decoder.c file directly using the low-level C bindings in crate libvpx-native-sys. However, the first interesting thing that happens in simple_decoder, is that we open a file and try to parse it using vpx_video_reader_open()...which isn't included in the crate I'm using.

There's some mention of bindgen in some of `the code, so I go and look at the infamous bindgen crate and its very accessible tutorial. Step 1 in the tutorial is:

Add bindgen as a Build Dependency

Declare a build-time dependency on bindgen by adding it to the [build-dependencies] section of our crate's Cargo.toml metadata file:
[build-dependencies]
bindgen = "0.26.3"

...Uh oh, there's no bindgen dependency inside Cargo.toml in libvpx_sys crate...instead there's a PNaCl helper create. What the flip is PNaCl and why does it sound so familiar? A few duckduckgo's later, I find the project description:

Native Client (NaCl)

Native Client enables the execution of native code securely inside web applications through the use of advanced Software Fault Isolation (SFI) techniques. Native Client allows you to harness a client machine’s computational power to a fuller extent than traditional web technologies. It does this by running compiled C and C++ code at near-native speeds, and exposing a CPU’s full capabilities, including SIMD vectors and multiple-core processing with shared memory.

While Native Client provides operating system independence, it requires you to generate architecture-specific executables (nexe) for each hardware platform. This is neither portable nor convenient, making it ill-suited for the open web.

Portable Native Client (PNaCl)

PNaCl solves the portability problem by splitting the compilation process into two parts:

compiling the source code to a bitcode executable (pexe), and

translating the bitcode to a host-specific executable as soon as the module loads in the browser but before any code execution.

Right, so like WebAssembly but that only works on Chrome and friends. Oh, and then I see the WebAssembly migration guide:

Given the momentum of cross-browser WebAssembly support, we plan to focus our native code efforts on WebAssembly going forward and plan to remove support for PNaCl in Q1 2018 (except for Chrome Apps).

Okay, why is this crate using PNaCl? I'm so confused. At any rate, I have to stop for the day and go to sleep.

Open Questions

Should I contribute patches on top of rustvpx crate to get the bits I need to decode? Or should I rewrite the example C program to only use functions that are available in libvpx-sys crate?
Do any of the functions inside libvpx actually work at all?
Could I get Doxygen to produce the documentation for libvpx from the C code? Or is there a copy of the docs floating around on the interwebs?
Should I go back and just work on nom / mkv.rs? I might ping that maintainer again and call it a night. We'll see what he says in the morning