Context
The big pushes this week were 1) fully wrapping up the error API and 2) pushing forward with data persistence in Dub. I’m really excited to be wrapped up with #1 and finally have a strong, clear path forward for #2.
More below :)
Overview
First: I made a lot of awesome progress on the error API last week -
Whenever Dub errors out, we need a way to anonymously log the error, the build, and diagnose issues. As of last Wednesday, I had a working local version of a service that tracks errors in Dub and saves them privately. Now, thanks to Robert’s help (shoutout @Robert), it’s hosted in the cloud & accessible anywhere with a whole bunch of cool security restrictions.
Second: I made a lot of great progress on properly storing, indexing, and reloading our browsers app data to persist across versions / opens and closes. I finally have a tested / working-ish structure using Realm (an open source alternative to Apple’s CoreData and JSON APIs) that I’m looking forward to getting over the line soon.
I also spent some time fixing a couple key bugs & exploring some upgrades to the browsing experience (supporting more advanced authentication standards like WebAuthn & security keys).
Process
In terms of the error API, there wasn’t a ton that needed to be done! Robert did some amazing (and really fast work) getting it all hosted on our cloud provider & I made a few tweaks to the code to speed up some of our lookups and additions. And, I added a few other key security restrictions to prevent misuse, spam, and bad data. All in all, it works great and is basically ready to go. As we add more error catching in Dub, we’ll be able to make good use of our new error service.
In terms of data persistence - this was a 3 steps back one step forward kind of week. But, I think the one step forward has put us in a really great place-
Previously, we were using our own form of data persistence with JSON. This is extremely flexible - it allows easy data caching on app startup and shutdown & decently easy saving / reading from disk. And, after some testing, it generally works great - except when it comes to LARGE amounts of data (like history). It gets slow to load and change, hurting the app experience overall. This pushed me to start looking for alternatives-
I’ve toyed around with other data persistence methods (like CoreData) but have found them generally clunky with high overheads. I had heard of Realm (which is an open source alternative owned by MongoDB) and decided to give it a shot. And… it’s quite cool!
It stores all the data in .realm files (similar to JSON) but has nice wrapper functions written that already optimize lookups and writes. The major drawbacks are that it requires you to use their (Realm’s) data structures instead of Swift’s native ones (which requires a lot of rewriting of our app). I’ve spent the past 2 or so days stripping down our app to a much simpler data structure (basically only two variables per class) to come up with an MVP implementation. I’m close - but primarily have been stuck on figuring out how to best cache the data on app startup.
Next Steps
The big next steps are FINISHING up data persistence - we’re extremely close to a launched MVP on Dub (a few small things left) and data persistence is at the top of the list.
The general plan is to spend the next sprint (til next Wed) pushing the remaining MVP features of Dub over the line & then rigorous testing & bug fixing.
Until then 👋