I’ve been a professional software developer for almost 8 years now. I’ve been paid to write a lot of software in those years. Far more interesting to me has been the recurring themes that have come up in my side-projects and in the software I’ve been personally compelled to write.
When I wanted to learn programming, I always had to come to the keyboard with a purpose. I couldn’t just sit down and start writing code, I had to have built an idea of where I was going.
For that reason, I’ve always used side-projects as a mean of learning programming languages. When I wanted to learn QBasic, there were a number of games, one based in space, another was a fantasy game. When I wanted to learn Envelop Basic, I attempted making a Yahtzee Clone, a Space Invaders Clone, a Hotel running game, and made a MicroGame based on the ones I’d seen videos of in WarioWare DIY.
When I wanted to learn C#, I went through a book, but I also, at the same time, worked on building a Scientific Calculator. (put a pin in that idea). When I wanted to learn Go, I wrote the CMS for the blog you’re reading right now. To pick up Lua, I used Love2D in several game jams. The only reason I have more than a passing familiarity with Erlang is because I used it for idea.junglecoder.com
Most times, when I’ve tried to learn a programming technology without a concrete goal to get something built, it is hard for me to maintain interest. That hasn’t kept me from trying out a lot of things in the past, but it’s the ones that allowed me to build useful or interesting things that have stuck with me the most. Right now, for side-projects, that list includes Go, Lua, Tcl, and Bash.
Ever since I first started cutting my teeth on C#, the ideas of parsing have held a certain fascination to me. Like I said before, I started wanting to write a scientific calculator. But, because I was a new programmer, with no idea of what building a scientific calculator should look like, I did a lot of inventing things from first principles. It felt like a divine revelation when I reasoned out a add/multiply algorithm for parsing numbers. It also took me the better part of two weeks to puzzle it out.
I was so proud of that, in fact, that I copied that code into one of my work projects, a fact which really amuses me now that I know about the existance of Regex and Int.Parse().
Eventually, I worked out a very basic notion of doing recursive descent parsing, and some tree evaluation, so that, on a good day, I had a basic, but working, math expression evaluator.
Working on that calculator, however, set me on a course of wanting to understand how programming languages worked. In the process of wanting to understand them, I’ve looked over papers on compilers more than once, but never quite had the patience to actually write one out. In the process of wanting to make a programming language, I ended up writing two before PISC. One that was a “I want to write something in a night that is Turing Complete” langauge that was basically a bastard version of assembly. At the time, I called it SpearVM. I had intended it to be a compilation target for some higher level language, but it mostly just served as stepping stone for the next two projects.
The second one was a semester-long moonshot project where I wanted to try to make a visual programming language, using either Java Swing or JavaFX, inspired by Google’s Blockly environment. Unfortunately, I could not figure out nesting, so I giving the ideas I’d had in SpearVM a visual representation, and using that for my class assignment.
The combination of all of these experiences, and discovering the Factor language, set me thinking about trying to build a programming language that was stack-based, especially since parsing it seemed a far easier task than what I’d been trying to do until then. A couple late nights later, and I’d built out a prototype in Go.
I’ve had a number of co-workers impressed that I’ve written a scripting language. Thing is, it took me like 7 false starts to find a way to do it that made sense to me (and that was a stack-based language with almost 0 lexing). It’s only now, that I’m on the other side of that learning experience, that I’d feel comfortable approach writing a language with C-like syntax. PISC, as I’ve had time to work on it, has actually started to develop more things like parsing, lexing, and even compiling. In fact, I’ve got a small prototype of a langauge called Tinscript that isn’t nearly so post-fix oriented as PISC, though it’s still stack based.
And, PISC, to boot, is still what I’d consider easy-mode when it comes to developing a programming language. Factor, Poprc, or even a run-of-the mill C-like language all strike me as projects that take more tenacity to pull off.
If the early years of my programming side-projects often focused on how to build programming languages, and how to better understand computers, the more recent years have a had a much stronger focus on how to harness computers to augment my mind. A big focus here was for me to find ways to reduce the working set I needed in my mind at any given time. This has resulted in no less than 7 different systems for trying to help keep track of things.
These are all approaches I’ve invested non-trivial amounts of time into over the past three years, trying to figure out a way to organize my thoughts as a software developer, but none of them lasted much longer than a month or so.
All of this came to a head during Thanksgiving weekend of 2018. My work at Greenshades often involved diving deep into tickets and opening a lot of SQL scripts in SSMS, and I had found no good way to organize them all. So, in a move that felt rather desperate at the time, I wrote a C# program that was a simple journal, but one that had a persistent search bar, and stored all of its entries in a SQLite database. And I used a simple tagging scheme for the entries, of marking them with things like @ticket65334, and displaying the most recent 5 notes.
It was finally a system that seemed to actually work for how I liked to think about things. The UI was a fairly simple 3-column layout. In the leftmost column, I had a “scratchpad” where I kept daily notes of what I’d worked on, in the middle I had my draft pad, and on the right I had the feed of notes, based on the search I’d done. I also had a separate screen dedicated to searching through all the notes that had previously been recorded.
There were several benefits to how this system worked:
All of this added up to me feeling like I’d found a missing piece of my mind. Almost like I’d created a REPL for my thought process.
Unfortunately, I don’t have that C# version any more. I do have a Go/Lua version, which is webapp based, though I still need to put some time into making the feedback-loop for it tighter, since my first versions weren’t quite as tightly focused on that, as much as they were focused on replicating the UI layout of the C# version. I’d argue that the tight feedback loop that the C# version had would be more important now, and I’ve slowly been working on adding it back.
The nice thing about the Go/Lua journal is that it’s far more flexible than the C# version, due to being able to write pages in Lua. Which means I’ll be able to use it for more things, and hopefully get that tight feedback loop back in place.
Exhaustive string search of both code and notes has proven to be a surprisingly effective tool for understanding and cataloging large systems for me. To this end, Gills (my journaling software), Everything Search (search over the paths and file names on your laptop) and RipGrep have been extremely handy tools to have on hand. The nice thing about search as a tool is that it can be adapted into other things quite nicely. In fact, I would argue that fast search, both via Google, and via the tools I’d mentioned above, is one of the more influential changes we’ve seen in programming in the last 20 years.
8 years is a long time, and there are lot more ideas that I’d like to get into later. However, these are the ideas and things I’ve worked on that have proven to be surprisingly sticky. Perhaps they might help you, or give you some ideas of where to focus.
Published September 9th, 2019