Why We Moved From Quill to Slate
When Graphite was first launched, the idea was to just have a clean, simple editor. After all, I was building it for myself. No one else was ever supposed to use it. After a few false starts, I eventually landed on Quill.js. Quill was built by some really smart people, had all the functionality I needed at the time, and was relatively simple to implement. The way I look at Javascript text editors is they fall into one of two categories: 1) Drop In 2) Custom. Some example of the two different categories:
Drop In
Quill
Custom
Quill, no doubt, was a drop in editor. You could get started without customizing any of it. Of course, as Graphite grew, I customized Quill a ton. But I began to want more. So did Graphite’s users. Things like table support and better performance were either going to require significant hacks, or it was going to require ditching Quill altogether. I seriously considered hacking Quill as much as I needed because Quill worked great with Yjs, a real-time communications framework that we used to enable collaborative documents. But the nail in the coffin for Quill came when a Graphite user who was writing really long short-stories (as an MFA in Creative Writing graduate, I was beyond excited about this!) was experiencing extreme bugginess with long documents. The cursor would jump to the beginning of the document. Or the end. Or the middle. Basically, editing any part of the document besides the very end was near impossible on a long document. After some research, it became clear that this was a known bug in Quill’s editor and there did not seem to be a fix on the horizon.
Time for something new. Enter, Slate.
One of Graphite’s main goals is to enable true productivity, and to do that, we’ve got some pretty lofty goals. We want you to be able to do whatever it is you need to do in a document to get your work done. Does that mean a checklist? Maybe an embedded tweet that you can not only view but comment on in-line? Maybe you need the @ mention your teammates and ask them to update the document with links to the cryptographic hashes of interviews they conducted (super nerdy but super cool use case!). No matter what the need is, Graphite wants to support it. But we want to do this without skimping on the normal word processing features you have come to expect from Microsoft Word or Google Docs.
When exploring what could support this, it quickly became clear that this would be especially hard with a drop-in editor. Not impossible, but hard. Mobiledoc Kit is what Ghost uses for their editor and it was a tough choice between that and Slate. However, in the end, Slate had a lot of the same feel and API documentation as a drop-in editor while giving the full customization capabilities of something like Mobiledoc Kit.
With Slate, you can already see some of the power it allows for. Start typing a new Graphite document. Now, switch to markdown if that’s your thing. It’ll work. Out of the box. Add a table, drop an image in that table, add a bulleted list in the table. All of it, supported.
These are basic examples of the power Slate helps Graphite wield. There’s so much more to come, but before we could even get this new editor out and into the wild, we had to clear some significant hurdles. If you’re technically interested, follow along into this next section. If not, the rest will probably bore you.
HTML Serialization
One of the cool things about Quill was that it took all of the content entered into its customized contenteditable div and turned it into HTML. So, when saving the content, it was already in HTML format. Then, loading the content back as HTML worked out of the box. This was really nice for read-only public documents. When someone shared a document and set it to read-only, it was already in HTML form, so building the page to support the display of this document was simple.
Slate, because it’s so customizable and powerful, does not serialize HTML for you. Everything is one giant JSON object. Think of it as a tree within an object that drills all the way down to the individual leaves. To save that as HTML required serializing ever node, mark, and inline style to HTML.
In the first pass of this new editor, that’s exactly what I did. I serialized everything I could, beat my head against the wall to figure out how to serialize data attributes, classes, ids, and more. But in the end, I realized that this method would significantly impact building new features. Not only would I (or someone on the team) have to build the new editor feature, but that feature would have to be properly serialized. It’s worth noting that if you’re saving HTML content, you need to deserialize it when you load it back into the editor using Slate. So, new feature, serialize, deserialize. I wondered if there was a way to just build the new features and not worrying about any of this serialization.
After some consideration, I realized there was only a couple reasons to use HTML content at all. It was necessary for rendering public (or private) read-only documents and for the word counter we had built. That’s really not much utility for all the work that would be necessary for serialization and deserialization.
So, the verdict was: kill the serialization.
All content would be saved as a JSON object, it would be loaded back into the editor as JSON. Any new features would just be built and work. But when HTML content was necessary, we’d use a simple vanilla Javascript function to grab the inner HTML of the editor element. Remember, the entire editor is a contenteditable div. So, we could just grab all the HTML within that DIV and save it or use it however we needed.
And that’s exactly what we did.
Public Docs/Private Docs
As mentioned about, public documents have a unique challenge compared to the private, encrypted documents (or even compared to collaborative documents). The public does not have an account. They can either view a shared document, or they can collaborate on it. In the first case, the HTML content that has to be served up and displayed in a nice way. Previously, I talked about the serialization issue, but what I didn’t mention is that grabbing the HTML content wasn’t as simple as choosing to save HTML instead of the JSON object. It actually requires saving both. Here’s why.
A public document can be read-only or collaborative. This is, of course, true of private, shared documents as well. But in a public document, the original user can switch back and forth between a collaborative public document and a read-only one. The public user will see that update in near real-time. So, if the user changes from read-only to collaborative, we need to be able to serve up the JSON content and populate the editor.
The solution, in the end, is pretty simple. When sharing a public document, both the HTML content is saved and shared as well as the JSON content. Then, depending on how the user is sharing the document, we can toggle back and forth between HTML and JSON.
Real-Time Collaboration
One of the most powerful parts of Graphite is that it was one of, if not the, first decentralized editors with real-time collaboration. This was made simple by the fact that Yjs had built-in Quill support. As soon as I ditched Quill, I knew I’d have to build something from the ground up. But hey, I was doing that with the editor anyway.
Turns out, I didn’t know a damn thing about websockets and real-time collaboration. Despite that being a core part of Graphite forever, it was so abstracted by Yjs, that I was in the dark. It took building a few real-time apps using Socket.io before I really became comfortable enough to implement this. But there was a catch. I knew I wanted to build it so I could add in authorship later. Authorship is the indication of who wrote what. So, when you’re in a Google Doc and you see the multiple cursors, or when you track changes and know who changed what, that’s all authorship.
With Blockstack, authorship should be tracked by the user’s Blockstack username. So, the trick was going to be making sure I could pass through that username as well as the content being edited on a websockets connection. Turns out, it wasn’t that hard. Authorship is not yet implemented, but it’s enabled with this new editor and this new real-time collaboration implementation.
One final issue I wanted to make sure we could address (again, this is not released yet, but it’s enabled) is encrypted collaboration over websockets. We’re already using wss encryption (which is the equivalent of TLS encryption for websites), but I really wanted to encrypt the content before it was sent from one client to another. Again, this wasn’t all that hard on a one-to-one document. However, when collaborating with many users, the content would have to be encrypted to many different public keys on almost every keystroke. The solution, it seems, will be a shared symmetric key. That’s not implemented, but at least I know it’s possible.
***
Graphite has some big goals for 2019, and this new editor enables a lot of them. We hope you’ll join us on the journey. And if you haven’t yet, sign up and start using Graphite now!
