Starting the Recurse Center Tomorrow

Ok, tomorrow I start the recurse center. Sooo exciting! Here are my current plans:

Write a lot of Rust! (Port existing projects first).

I’m going start with porting this Javascript Paint-By-Number app to Rust, and then wasm so it can be used from the browser.

This will be good Rust practice, and also very fun. I’m curious about the tricks in image processing to make this better (different kinds of smoothing, image flooding, etc), and also parallelizing this with Rayon.

Desktop AR

A general thing I want to explore while at Recurse is AR, specifically projector-on-a-desk AR. I have a 3d camera (the new Kinect) and want to pair it with a cheap short throw projector. Then I want to make:

AR Sudoku solver

Sudoku is a solved problem (I’ve even ported the Norvig sudoku solver while learning Rust), so this task will be detecting and tracking the puzzle, and then projecting the solution numbers. Also a little bit of UX, to make the app more than just “show the solved puzzle”, but maybe show single hints, or highlight particular numbers. I think this is a great task to get into the basics of AR while still making something interesting.

AR paint by numbers

First, I’ll re-use the Rust paint-by-numbers tool I’m porting. Then I’ll use object detection and tracking for the canvas to project the paint regions onto it. This will be about wiring my own code into an AR app, and getting more practice with AR apps.

AR jigsaw puzzle solver

This one is a long project that I’ve been yak-shaving and going in circles and rabbit holes on. I’ve been working on this on and off for over a year, and it’s too big.

I’d like to finish the small step I’ve been working on, which is wiring my TSP Solver to a generated puzzle’s outside edges, to see if it can solve that.

I’d like to gain the motivation to refocus the project into something useful, like using the depth camera to properly scan all the pieces, and then maybe suggest the 10 possible best matches.

Actually solving a jigsaw puzzle is kind of hard because of the noise in the pieces, and also because as a human you get tactile feedback into whether a piece fits with another. The computer only has a lossy heuristic, and it only takes one mismatch while building the solution to screw up the whole puzzle. But just suggesting the next piece would be a useful feature, and it would be in desktop-AR alley.

Part of the reason I’ve been struggling with this one is that I’ve been sidetracked into generating puzzles with D3 and SVG, which are both important tools to me. We’ll see what I do about that idea.

Approach

My focus throughout is going to be on small to medium applications, and finding ways to develop small pieces of functionality at a time in little bundles, to avoid doing anything “big-bang"ly. I can layer different projects and learnings onto one more functional application. For example, I’ll pick up simple object tracking with the AR sudoku solver, then combine the Rust paint by numbers written earlier with it to make the AR Paint by numbers app. So, layering. Lots of layering.

Next, I want to dig into postgres and database indexes a tiny bit. I want to do so by:

Implement a fractal Btree tree index in Rust

This a data structure closely related to the all-important B-tree. There is also c code out there that I could push off of. I could also start with regular btrees to get the feel for them.

Then,

Try to package a fractal btree as a postgres extension

This sounds really cool, but may be hard. This will make me look into the postgres source, which I’ve wanted to do for a while, and I’ll learn

How to build a custom index type in postgres
How to do C FFI with rust really well

Read “Performance Modeling and Design of Computer Systems: Queueing Theory in Action”

I’ve had it on my shelf for 2 years. It looks amazing, and I want to really get Queuing Theory, because there are sooo many queues in computers, and especially in distributed systems like the web. I read “Analyzing Computer System Performance with Perl::PDQ” which was great, but I don’t remember all of it. Also, it has simulation software written in perl that I may port to Rust to more fully understand.

I’m interested in GPU-ey things, like writing a shader, and maybe catching up on recent DL improvements like attention, transformers, GPT2, but this is lower down my list.

Word embeddings

I have these “dumb ideas” that might lead to something:

What are the widest gaps in word embedding (meaning) space? ie, the biggest empty spheres? (In different languages?) This is like, there would be concepts like hotcold, or lovehate that we don’t have words for.
- However, this is more of an algorithm question, finding the largest N-dimensional sphere in a region, as I wouldn’t be training the embeddings myself.
I also like the ideas of “debiasing” language by changing words to remove eg the “gender” axis in them, which is a paper I read a while ago, and would be curious to read new developments on.
Maybe develop a new embedding for something

Dataflow

As a data processing person, I’ve been watching Frank McSherry’s Differential Dataflow and Timely dataflow work, and I’d like to go understand it. The rust compiler depends (or may in the future) on it. There are open issues I might be able to contribute to.

Frontend

I want to get back to fully functional with the frontend space.

That means, finally understanding webpack.
I want to try sveltejs, which looks really interesting.
I’ve also wanted to master d3.js, though I’m not sure how I’ll do that, except a vague “write a lot of visualizations”

Systems programming

I’m also curious about systems topics:

Networking
Profiling
How memory exploits and Return Oriented Programming work

So, wow, that’s a lot, but I have 3 months. I’m also going to be ok “BFS"ing projects, as in, doing layer n of project A, and then moving to layer m of project B, just because I know my brain has motivation issues. I’ll still be working to finish a layer at a time, to always have something “finished”, but it’s ok if I switch categories to get a break every now and then.

Summary

Write a lot of Rust
Get better at Frontend
Scratch some backend itches
Learn systems to be better a programmer in general

As is encouraged at Recurse, I’m going to try to blog every day.

And just to get warmed up, I’ll recall you how I moved files over to my new laptop this afternoon. I had never done this before, and when it works it’s actual magic!

I moved from a Macbook Pro to an XPS 13, so I went from Mac to Windows (no easy way with AirDrop). I also happened not to have any USB drives or cables on me, but both computers were on the same Wifi, so I figured I should be able to network them together.

I went into the Mac’s system preferences, and under Sharing, turned on File Sharing, and explicitly allowed Windows fiile sharing, making sure both computers had the same workgroup.

On the receiving computer, I opened the network tab, but couldn’t find the sending computer! This is usually how it goes with networking and file sending, even in 2020.

But this time, I’m a grown human, and I went into the sending computer, typed ifconfig, found the ip address of the en0 (wifi) interface, and was able to find the sending computer folder by looking for //192.168.0.154/folder (the sending computer’s ip on the local network, you can tell because it starts with 192.168.) on the receiving computer!

And then, I didn’t really want to do the copy on the Windows GUI, so I went into WSL and installed smbclient a CLI utility for reading samba shares. Then I followed this tutorial to recursively move everything over.