A week that starts with brownies and board games with friends can't be bad - though I may have to lay off the baking for a while, it's taking a physical toll. In other hobbies, my P2P event stream project has hit an exploration phase, so results this week are hard to quantify. That's natural, there's a lot new here - I'll just try to summarise what I've found.
I decided on a simple format for the
dust client/server protocol. It's
basically JSON-RPC, but
using bencode instead of JSON - this avoids
an extra dependency, and makes binary data easy to encode. This
protocol is now implemented in a bare bones client program called
dust, which interacts with a
dustd daemon; similar to how the
docker program interacts with
dockerd. This client/server design
should make a user-interface easy to build in other languages.
I've also been planning for the peer discovery stage. One of the immediate problems I'll hit is working around NAT in home routers. There are several techniques for punching through, but it's a lot easier with UDP than TCP. Instead of implementing my own crappy streams on top of UDP, I've been investigating an alternative: the Stream Control Transmission Protocol (SCTP) - It's used for the data channel in WebRTC.
SCTP is a message-oriented protocol like UDP, but each message is assigned a stream and all data within each stream is delivered in order, like TCP - potentially a best of both worlds. It can also be encapsulated in UDP packets, which could be very useful for NAT hole-punching! I found friendly information on SCTP a little thin on the web, so I've picked up a copy of the book UNIX Network Programming: The Sockets Networking API, which includes several chapters on it.
After alluding to it last week, I've started my blog post on timestamps in distributed systems. I think it might spawn a second post on version numbers too - hopefully they'll be a useful reference to point people to.
I've been testing Dan Leslie's chicken-scheme package. It provides auto-complete and documentation support without a running Scheme REPL - I've found it useful when working on C integration modules, which need to be compiled.
A few links to keep you entertained:
- All File Systems are Not Created Equal: On the Complexity of Crafting Crash Consistent Applications - An old but good post I was reminded of this week - take a look to see how poorly your database of choice handles crashes (though SQLite continues to impress!).
- Acing the technical interview an amusing post by Aphyr
Enjoy your weekend!