One of the most important parts of any venture we take in our earthly bodies requires collaboration in some way or another.
Slack chats, Zoom calls, in-person meetings, and brainstorming sessions, it always comes back to collaboration. In teams, real-time collaboration on documents is also equally important.
I'm going to review Google Docs and a feature that makes it so useful compared to other applications. Specifically, I'm looking at change history and real-time collaborative editing.
I'm personally building a screenplay editing software ( for fun ), and I found some things rather interesting about how these background characters in life work.
The Parts
A basic online editor has several parts: The WYSIWYG ( What You See Is What You Get) editor on the frontend, an API that handles storing and retrieving the document, and a storage system that keeps a compiled version of your document, ready to go.
Maximize imageEdit imageDelete image
In the diagram, we have two storage buckets. One will store metadata: the document title, the location of its master file, the user, and other information. Secondly, the Storage bucket, which stores a binary format of the merged change history.
This binary will be loaded at startup, giving you immediate access to all changes.
Real-time collaboration introduces a new problem - merging changes in real time. What happens when User X and User Y edit the same line simultaneously, but their changes reach the server out of order?
For that, we need an algorithm to resolve the change conflicts. One approach is to have a central server handle change merging, and you put your brain to work implementing OT ( Operational Transform), or another is to shift the responsibility to the client's devices and use CRDT (Conflict-Free Replicated Data Type). I went with a CRDT because of offline support, upfront simplicity, and the fact that I'll use this with 2 to 4 friends.
CRDT assigns every character a unique identity, so operations reference content directly rather than positions that can shift. This does mean that when something is deleted, the character isn't removed — it's marked as deleted. That marker (tombstone) stays in the structure forever, which can balloon the file size over time.
One solution is to prune tombstones after a certain stage.
This article gives more details and is a good read about the different approaches (CRDTs vs. OT).
It's interesting to see the approaches and caveats of these technologies that fly under the radar
What's next
I'll continue to work on this over the week in my spare time. Plan to Dockerize and host it on an old laptop in the lab
I have a few other projects I want to complete too: a data visualizer for Premier League results over the last 20 years and a way to sync work across my laptops. No, don't want a NAT, nor an FTP client. Something similar to the experience of dragging files across your devices on MAC
Built with: React, NodeJS, Typescript, YJs, TipTap, ws