Jaisen Mathai

Letters To My Kids

2026-09-29T00:00:00-07:00

When my son turned 2, I decided to set up an email address for him. It wasn’t to secure an email address with his name. It was so I could send him notes and give them to him once he became an adult. If you’ve ever been a parent, you know too well those fleeting moments which pass you by and vanish. I wanted to capture them and I thought being able to do so in an email was a good idea.

That was in 2012. Fast forward over a decade and it turns out that my wife and I stuck with it. I checked last week and there are over 120 emails in there. Fleeting moments which we were able to capture. I’m really grateful for each one of them.

He’s 16 and I’m planning to create a hardback book of all the emails to give him when he turns 18. So I’ve been going through those emails reliving some of the richest memories of my life. Most of the emails are of fun moments but I’ve sent some rather raw ones too. A couple of emails talk about the challenges and frustrations of trying to parent well. But they all end on the same note of unconditional love. I’m most excited for him to read those. As he grows into adulthood, I think those letters will be especially meaningful to him.

Being on this side of it, 14 years later, it feels a bit surreal. 14 years of writing letters is a long time and it’s unlikely that I would have stuck with it. There were plenty of other ideas that I’m sure lasted a week or month before I got out of the habit. What made this stick was having the ability to send an email while lying in bed at night or attaching a note to a photo shortly after taking it.

I have 3 other kids and fortunately I set up email addresses for them as well. My daughter is 14 and I’m pretty excited to go through the emails I’ve sent her. I have a unique relationship with each of my kids and it comes through in these letter. My 5 year old twins will have to wait a while and we’ve sent them fewer emails. But the gratefulness I feel for having them for my older kids is plenty motivation to make up for lost letters for my younger ones.

If you have kids, I really recommend setting something like this up. It doesn’t feel like much at the start but as you approach the finish line it truly is a gift that’s hard to describe.

Letters To My Kids was originally published by Jaisen Mathai at Jaisen Mathai on September 29, 2026.

Sojourn for iOS Was 45 One-Shot Prompts

2026-09-10T00:00:00-07:00

Sojourn: Topical Bible Study is an app I published without typing a single line of its code.

That part isn’t interesting anymore. Plenty of people are writing code with AI. What I think is interesting is the process. Since ChatGPT 3.5, I’ve been a little bit obsessed about LLMs writing code. Not assisting me writing code, doing it all for me. While the process has evolved a lot, I’m going to talk about where it’s at now and give an explaination.

About the app

The app is an iOS app written in Swift. It has a streaming chat interface which parses the incoming text in real time. This is done to eliminate hallucinations of scripture by replacing LLM generated citations with text from an app-bundled database. All scripture references are guaranteed to be correct and users can tap on them to explore surrounding verses and chapters. The UX of the app holds a high bar in terms of performance and usability.

The app talks to a backend written in Python that connects to an LLM provider’s API. The backend exposes a tool for scripture lookups instead of relying on the model to do it itself. Nothing about the user’s chats or identity is stored on the backend. It all lives only within the app itself.

About the process

I should define what I mean by one-shot. It’s a single prompt that’s sufficient to implement an end-to-end feature regardless of complexity. In reality, there may be a couple follow up prompts to tweak something here and there. But mostly, it should be 0 follow up prompts.

Baseline context

I spent multiple days co-drafting the user experience and features I wanted in the app using Claude Design. The home page of the website embeds the artifact of that work as an iframe - you can see it on the app’s Sojourn: Topical Bible Study page.

This iframe ended up becoming a major input source for when I started creating the actual iOS app. Turns out Claude Code can understand a functional prototype quite deeply. So much so that the user experience of the app is nearly identical to what Claude Design generated. This was a pleasant surprise.

I also created a bunch of markdown content that’s stored in CLAUDE.md and adjacent artifacts which outline product definitions and constraints. For example, privacy is a cornerstone and it explicitly states that the app should never collect personally identifiable analytics.

This baseline context is created once, updated automatically, and guides the following steps.

`/spec`

Ever feature starts by me using a skill named /spec. This is where I take my time collaborating with AI to fully define everything about the feature including happy path, failure modes, look & feel, architecture and testing.

Every feature is stored as a GitHub issue. Once I feel the feature is adequately described, I give it the go ahead to create the GitHub issue based on a template in the skill. The issue title is a short summary of the feature and the description is a detailed explanation of it.

`/gh-issue`

Implementing a feature is as simple as me using the /gh-issue skill and passing the issue number to it. I generally get up and leave the room at this point. The skill does all sorts of cool things like looking up other issues or pull requests if they’re mentioned. Sometimes I’ll have two unimplemented features each with their own GitHub issue and I make sure they reference each other. This way, when implementing the first it knows to look up the second and proceed accordingly.

Once finished, it automatically creates a pull request.

I often kick this off before going to bed and wake up to a new feature in my app.

Publishing and deploying

I still do this by hand. But it’s still fully automated through scripts like run.sh which runs the app in the simulator, deploy-to-device.sh which installs it directly on my iPhone, and release.sh which publishes it App Store Connect to be published.

Nothing is one-off or done by hand. If I have to do it once, it gets automated into a skill or script. I recently switched laptops and getting up and running was quite smooth once I transferred App Store Connect keys.

For other projects, I have a /gh-pr skill which merges the pull request and deploys it. Works great for web apps but I don’t want every feature to be published to Test Flight or the App Store - so I batch them as I see fit.

Some examples

A feature I really enjoy is the home screen widget. It was 2,466 lines of code across 27 files. This included a new WidgetKit extension, a shared app group, a verse pool, deep links back into the app, and the XcodeGen config to build it. One issue. One prompt. One PR.

The home screen widget - one issue, one prompt, one PR.

Another feature I’m testing out is emailing yourself a conversation. It was 2,412 lines of code across 21 files, and it spans the client, the backend, and an email delivery provider. Also one issue.

Emailing yourself a conversation, tucked in above the message field.

Neither of those is a small feature, and neither is a boilerplate feature. What made them work is that the spec had already settled the hard questions.

How’s it going?

Excellent. The limiting factor is shepherding the user experience of the app. Just because you can add multiple features a day doesn’t mean you should. I take my time thinking through a feature until it feels right.

So far I’ve created 49 GitHub issues and closed 45 of them in a matter of weeks. That’s 45 features implemented. The 4 open issues are features or bug fixes I haven’t felt are important enough to do yet.

Now that you know how the sausage is made, download Sojourn: Topical Bible Study and let me know what you think by clicking my email address in the footer.

Caveats

I happen to have an immense breadth of experience. I’ve been building things professionally for decades including launching several startups. That’s given me pretty deep expertise across design, development, infrastructure, marketing…you name it. It’s a unique advantage when paired with AI and one I’m grateful for.

Sojourn for iOS Was 45 One-Shot Prompts was originally published by Jaisen Mathai at Jaisen Mathai on September 10, 2026.

Big Tech Wants To Be Your Bible Translator

2026-09-01T00:00:00-07:00

Throughout history, immense trust has been placed in those who could translate the Bible. Either into a new language or into layman’s terms. Mediation, what stands between you and the original text, is sacred.

Every new technology has pushed the envelope on mediation. The printing press, the Internet, mobile devices. But they all shared one thing in common: they increased accessibility to trusted versions of scripture.

AI is the first widely adopted technology which doesn’t have that in common.

The incentive is to keep you in chat, not direct you to scripture

The Googles and Facebooks of the world make more money the longer you stay on their platform. This is because they can show you more ads. If you spend 20 minutes on Facebook instead of 10, they make twice as much money because they can show you double the number of ads.

Does Big Tech really want to be a mediator?

Set aside the financial motivation. You only have to look at how Google Search has evolved.

Ask Google today and it mediates in place...an AI Overview that keeps you on the results page.

The same query once led here...a link out to Bible Gateway and the actual text.

This is more subtle than a wrong verse

I’ve written before about AI misquoting scripture, and that’s a real problem. But a wrong citation is at least catchable. You can check it.

A mediator who is motivated to be a persistent interpreter is harder to notice, because it doesn’t quite feel like you’re losing anything. They are helpful, warm, encouraging and talk about scripture. But they don’t want you to leave and become immersed in the word and that’s where convenience turns harmful.

Paul lays out an example of how we’re to engage in scripture in Acts.

Now these were more noble than those in Thessalonica, in that they received the word with all readiness of mind, examining the Scriptures daily to see whether these things were so. -

They didn’t rely on the mediator. They went to the source.

How does good faith-based AI look?

So, then, what does AI that’s helpful to deepen our faith look? I think it starts with incentive.

My exploration into this space, Sojourn: Topical Bible Study, measures success when you get out of chat and into scripture and is free to use. I’m learning first hand the incentives which point this new technology towards a deepened faith.

AI is a powerful tool for good … when in the right hands.

Big Tech Wants To Be Your Bible Translator was originally published by Jaisen Mathai at Jaisen Mathai on September 01, 2026.

Getting People Out Of Chat And Into Scripture

2026-08-25T00:00:00-07:00

I wanted to explore how AI could help deepen my faith. I enjoyed using Claude to research topics which were on my mind. It does a good job finding and quoting scripture but a poor job of letting me explore surrounding verses. I wanted an experience similar to what I was having but optimized to get me into scripture and not stay in chat.

Let’s set aside the fact that LLMs hallucinate scripture, Sojourn solves that but it’s for another post.

I used a few weeks of latent time to think about what was missing in my experience with Claude. I wanted a very lightweight way to peek into scripture with similar lightweight ways to fully immerse myself in it. I also wanted a way to work back to earlier context. I sketched out how that would look and it turned out that I loved the experience - (see Sojourn - Bible Study). Here’s how it looks.

1. Citations are exact, tappable

2. The full verse, one tap away

3. Read the full chapter

4. Access the surrounding chapters

I thought that others may appreciate exploring scripture this way. I published the app to TestFlight and shared it with people I knew. Turns out that they did like it!

Notes from early TestFlight users.

The feedback motivated me to publish it on the App Store to the world for free. Would strangers find it as useful as friends? Turns out, yes. They also liked it!

Devotional time is deeply personal and Sojourn: Topical Bible Study’s privacy policy reflects that. The app doesn’t ask for any information before you can use it and chats are never saved.

Getting people out of chat and into scripture

So how do I know strangers find it useful? Well, two anonymous metrics. Many use the app multiple times a week and many exit the chat and get deep into scripture. Here’s an example of a conversation that lasted 18 minutes and followed a common pattern of chat ⭢ scripture ⭢ chat ⭢ scripture. Notice they spent 13 minutes in scripture. That’s 13 minutes well spent!

Circles are chat turns. Diamonds are moments the person left chat for scripture.

This has me really optimistic about positive uses of AI for faith. I encourage other Christ followers to do the same and think about ways AI can be a means and not the end.

Getting People Out Of Chat And Into Scripture was originally published by Jaisen Mathai at Jaisen Mathai on August 25, 2026.

The Risk and Opportunity of AI for Faith

2026-08-12T00:00:00-07:00

Generative AI is simultaneously the most powerful and most dangerous technology to our faith.

Scripture has always taken the shape of the medium which carried it. In the Old Testament, Israel carried God’s word across generations through oral tradition.

We will not hide them from their children, telling to the generation to come the praises of Yahweh, his strength, and his wondrous works that he has done. -

As time went on, oral traditions evolved to writing, copying, and preserving. God instructed Isaiah to do exactly that and these manuscripts remain in existence today.

Now go, write it before them on a tablet, and inscribe it in a book, that it may be for the time to come forever and ever. -

Scripture then crossed language boundaries. It started becoming translated; from Hebrew to Greek to Latin to over a thousand languages today.

He said, “How can I, unless someone explains it to me?” He begged Philip to come up and sit with him. -

Then the printing press made scripture reproducible at scale. The internet made it accessible. The mobile phone put it in everyone’s pocket. Now AI makes it conversational.

Where it goes wrong

AI isn’t held accountable by the truth. Whatever it tells you, it says it with confidence. Even when it performs verifiable tasks incorrectly. Like misquoting scripture; which the YouVersion CEO claims it can do 15-60% of the time. We trust the response without understanding how it was formed. It’s simply too large of a responsibility to hand companies like Google, OpenAI or Anthropic.

And the risk isn’t new or hypothetical; there’s research backing it up. John Dyer surveyed Bible app users for his book, People of the Screen, and found that 30% of screen readers came away confused, compared to 16% of print readers, and fewer of them felt spiritually nourished. That’s Bible apps without AI. Extend that research to a chat interface and you can envision the direction it goes.

Where it goes right

In the same breath, AI can be immensely powerful. It makes scripture accessible across languages and reading levels in ways study Bibles and apps cannot. I’ve personally asked AI questions I would only ask 1 or 2 people I trust - but at 1am when no one was awake. I’ve also asked it some questions I’m not courageous enough to ask anyone.

I’m exploring this topic with optimism. Can AI deepen faith? Can it direct people away from itself? Can it be made free to use? I’m finding the answers to be yes and that has me excited. As Christ followers, we are called to spread the good news. We need to take the reins before the Googles of the world doing it inadvertently.

For those curious - my current exploration is through an app I created named Sojourn: Topical Bible Study. I think it’s doing a good job of getting people out of chat and into scripture. If you’re exploring, I’d love to chat.

Philip and the Ethiopian

Acts 8:26–31

26But an angel of the Lord spoke to Philip, saying, “Arise, and go toward the south to the way that goes down from Jerusalem to Gaza. This is a desert.” 27He arose and went. Behold, there was a man of Ethiopia, a eunuch of great authority under Candace, queen of the Ethiopians, who was over all her treasure, who had come to Jerusalem to worship. 28He was returning and sitting in his chariot, and was reading the prophet Isaiah. 29The Spirit said to Philip, “Go near, and join yourself to this chariot.” 30Philip ran to him, and heard him reading Isaiah the prophet, and said, “Do you understand what you are reading?” 31He said, “How can I, unless someone explains it to me?” He begged Philip to come up and sit with him.

World English Bible · public domain

The Risk and Opportunity of AI for Faith was originally published by Jaisen Mathai at Jaisen Mathai on August 12, 2026.

Giving In To Generosity

2026-03-04T00:00:00-08:00

The Bible has a lot to say about generosity. While it doesn’t only talk about money, that’s what I’m writing about today.

Secularly speaking, I’ve had a very healthy relationship with money. I live within my means and save diligently.

Biblically speaking, I’ve struggled. I tithe regularly and often give generously beyond that. But money has a way of pulling me toward security and status. Financial security, early retirement and proving to myself and others that I made it are all real ways that money takes my eyes off what really matters. Biblical generosity has nothing to do with the money and everything to do with honoring God. It’s a subtle but massively important distinction.

In my core, I know that I want unadulterated devotion in my faith. While I can be ruthless to myself, my faith lets me be equally gracious. This means I get to be fully honest and really explore my motives. Because, on the surface they are fine. Financial security can be masked as providing for my family.

Giving in to generosity means I need to systematize it. It needs to be on autopilot to avoid the risk of me mucking around with it. I’m still learning how to do this but here’s what I’ve come up with so far.

Donor Advised Funds

I initially looked into DAFs because of their tax efficiency. I do love that about them. But what I love more is that they let you take large steps of faith. We donated 5 years worth of charitable giving at the start of 2026 to our DAF. It was quite frightening because once I clicked that button, there was no way to claw the money back.

I contemplated doing it for weeks. But once I did it, I felt complete relief. All the fear I had vanished. I was surprised.

I was more surprised in the weeks after as my attitude towards the funds in our DAF shifted from we will spend this over 5 years to I think we can probably spend it all this year. Taking that first step and clicking the button flooded my heart with generosity. It’s a couple months later, and that hasn’t changed.

Charitable Business Ventures

I have a penchant for building things, trying to get others to use them, and trying to turn that into a business.

If it’s inevitable that I do that, why not try and weave generosity into it?

I’m interested in how artificial intelligence changes what a business model has to be. Work that used to take a team and a budget can now be done by one person for almost nothing. When building and running something costs close to zero, charging for it stops being a requirement.

Sojourn: Topical Bible Study is my example. It’s free to use with nothing to buy, so the only thing it has to do is get people further into scripture.

This idea particularly interests me because it lets me pursue something that feels natural to me while orienting it towards generosity from the start.

Sometimes I think that in the grand scheme of things that all of this is silly. And in some ways, from the outside, it is. But when we lift our eyes to God, He begins work on the inside.

Giving In To Generosity was originally published by Jaisen Mathai at Jaisen Mathai on March 04, 2026.

My Ridiculously Robust Photo Management System (Immich Edition)

2026-01-16T00:00:00-08:00

TL;DR: Changes to albums, descriptions, location, date/time and favorites through Immich are stored in the photo’s EXIF and automatically backed up to my Synology NAS and into Dropbox; no database needed. Below is a video for those who prefer to watch instead of read.

I created a simpler version of the plugin I use that is suitable for those with their own workflows which might differ from mine. It’s available at immich-exif.

I’ve been working on my photo management and archiving workflow for over 2 decades. It’s changed a lot over that time but usually in bursts. It works really well and I don’t have to touch it much.

By far, the most durable aspect of my photo management workflow is that it only relies on EXIF to store metadata about photos. No reliance on an external database to keep track of descriptions, favorites or albums. I’m not sure of any other workflow which does this but I think they all should. But we’re not here to talk about that. I want to tell you all how I’m using Immich.

I outlined my philosophy for photo management in detail in another post but I’ll summarize them here in order of importance.

Preserve. My photo library needs to be future-proof for decades into the future.
Unify. Photos from my wife and my phones need to be combined into a single library.
Experience. The photos and videos need to come to life and help us re-experience the moments when they were taken.

I was using Google Photos as a read-only viewer of my photos and fell in love with the discovery features. But a change back in 2019 to how Google Photos and Google Drive worked together broke my workflow.

The primary source of my photos is my Synology NAS at home. I organize my photos using a simple command-line tool, named Elodie, that I wrote over 10 years ago that’s grown to 10k lines of code, 1,300 stars and 150 forks. The best way to describe it is that it materializes your photo library onto the file system using only the EXIF. It’s my canonical organizer.

I clobbered together a plugin for Elodie that would replace the Google Drive + Google Photos capabilities that Google deprecated. This let me continue using Google Photos as a read-only viewer for my photos. But I was growing tired of relying on Google services and especially handing over all of the data embedded in my photos. So I stopped using Google Photos.

I was left with several years of not having a great way to view my photos. I desire a richer experience than the folder-based default experience I have with Synology and Dropbox. I was very optimistic about Synology Photos but it was unfortunately underwhelming. Maybe because my Synolgoy NAS is 10 years old but I also didn’t want to upgrade it.

I’ve been hearing a lot of positive feedback from others about Immich and had been keeping a close eye on it. At the end of 2025, I decided to take a closer look. The aha moment was when I learned about external libraries. If you’re not familiar with them, they’re existing folders that you can tell Immich about and it’ll add them to your photo library. Most importantly, you can mount them as read-only.

I immediately drew paralells to my prior use of Google Photos. I quickly determined that I could restore what I lost with Google Photos using Immich. But I wondered … what if I could do more? Been wondering for a few years, actually.

Wondering about an easier interface than the command line. October 31, 2022.

Turns out the answer is yes. I was able to turn Immich from a read-only viewer to a full fledged way to organize my photos while relying only on EXIF. Now, Immich doesn’t modify EXIF of photos. Instead, it stores information into its Postgres database. Immich can read from and write to XMP sidecar files. But sidecars are too clunky for me. I want everything embeded in the photo file itself. As far as I understand, Immich authors don’t want to edit the original photo file. I get their reasoning but I still want to be able to.

What I thought would require a 4 hour session with Claude Code turned into a two week rendevous learning a lot more than I anticipated about how Immich works. External libraries are great but there are some obvious gotchas with how Elodie works. If you add a photo to an album using Elodie then it will update the photo’s EXIF and also move the photo into a folder with the name of the album. To Immich, this is interpretted as deleting the original file and creating a new one. This creates all sorts of challenges but I was able to solve it using an eventually consistent approach. This means, all of the changes are stored and will eventually materialize.

There were other smaller challenges but I was able to overcome them all with some elbow grease. Immich’s API is pretty great and I’m probably just scratching the surface. But for now, I’ve been spending a lot of time enjoying my photos like I haven’t been able to for a while without compromising my core principles for making sure they’re around in a couple decades.

A detailed technical explanation is outside the scope of this post but if folks want to know I’ll write it up separately. For now, follow GitHub issue 496 to track progress and see the code.

My Ridiculously Robust Photo Management System (Immich Edition) was originally published by Jaisen Mathai at Jaisen Mathai on January 16, 2026.

Latent Product Development

2025-01-05T00:00:00-08:00

The best products and features emerge naturally from within your subconscious.

la·tent: lying dormant or hidden until circumstances are suitable for development or manifestation.

There’s something special about a good product. They’re malleable and composable within its intended use while holding onto some opinions. This doesn’t mean that they are full of features. In fact, it’s often the opposite. Many have few features which tend to be quite simple. A great product doesn’t have to do many things. It just needs to do all of the things it does exceptionally well.

What happens when you let ideas simmer in your subconscious? Their rough edges start getting slowly sanded away one by one. But something more important happens. You arrive at an inflection point where it feels right in ways the original version of your idea didn’t. Sometimes it’s a single breakthrough idea and other times it’s the culmination of many smaller ideas. This is the point where you really fall in love with it. Fielding questions you’ve never considered before will feel natural and your confidence talking about it will increase.

I get pitched a lot of ideas and I almost always encourage people to think deeper about them. Not in a forced way though - that does not yield novel results. Then how, I wondered. This is what’s worked well for me.

Take breaks often while working before your brain feels overloaded.
Perform activities which require low mental attention like walking or showering.
Start your morning with something relaxing like sitting on a couch drinking coffee (no devices).
End some evenings early and to go lay down in bed before you're tired.

The common thread is that latent thinking will happen naturally when your brain is in a restful state for 15 minutes to 30 minutes. Not much to ask in exchange for great ideas.

Latent product development is easier to do on smaller teams where everyone pays attention to craftsmanship. You can tell if the craftsmanship is ingrained into every aspect of the product including what the user does and does not see. I’d say it’s part of the team’s culture.

If your product has a well defined 24 month roadmap, forget it. There’s no room for latent product development. The entire product is extruded through the machine. The incentives are aligned elsewhere for reasons that are important for some.

Latent Product Development was originally published by Jaisen Mathai at Jaisen Mathai on January 05, 2025.

Life of a Blockchain Transaction - (Proof of Work)

2021-10-29T00:00:00-07:00

This article is also available as a thread on Twitter.

The thread this article is based on. October 29, 2021.

The life of a crypto/blockchain transaction. Proof of Work version.

You use your favorite app to purchase 1 ETH. The app creates a transaction and broadcasts it to the network. The transaction includes metadata including transaction and gas fees that you are offering to someone (aka a miner) for adding your transaction to the blockchain.
Your transaction enters a pool of unvalidated transactions waiting to be added to the blockchain by a miner. The likelihood of your transaction being selected by a miner is based on the amount of fees you are offering. Most apps will determine this fee on your behalf.
Many miners compete to add the next block to the blockchain. A block can contain any number of transactions greater than zero as long as it fits within a certain size. The winning miner is rewarded some of the cryptocurrency as well as all the transaction and gas fees.
Adding a block to the blockchain requires miners to find a hash. The hash is based on the data within the transactions AND the hash from the previous block. This is what makes it a chain because each block’s hash includes the hash from the prior block.
A hashing algorithm takes input data and gives back a string of characters. The SHA256 algorithm used by Ethereum generates a 64 character hash - making it practically impossible for any two input data to have the same hash.
The hash must meet certain criteria before it can be validated. It must be correct AND be lower than the “target hash” set by the network. For example, the hash “zzz” is HIGHER than “000”. Think of it as a sorted column in a spreadsheet.
It’s rare that the hash of the transaction data plus the prior block’s hash will be lower than the target hash. In order to generate a valid hash, miners will add a number (aka nonce) as part of the input data in hopes that it’s lower than the target hash.
The network dynamically sets the target hash to varying levels of difficulty based on the number of miners. The more difficult a target hash, the less profitable it is to be a miner & vice versa. This regulates the # of miners and adds a level of consistency to the network.
Miners end up trying LOTS of nonces before finding one that generates a valid hash. The energy required to do this is what is referred to as the “work”. The faster a miner can generate hashes (aka hash rate), the more likely they win the chance to add the next block.
When a miner generates a hash they believe to be valid, they send the hash, nonce and transactions to be validated by a node on the network. Validators can quickly determine if the proposed block meets all of the criteria before adding it to the blockchain.
This process starts all over each time a new block is added to the blockchain. Miners have to repick unvalidated transactions and start generating hashes. Your transaction is confirmed once it is included in a block that is added to the blockchain.

I’ve read a lot of documentation and articles which helped me understand how blockchain transactions work. Read the articles below or contact me if you have questions.

Life of a Blockchain Transaction - (Proof of Work) was originally published by Jaisen Mathai at Jaisen Mathai on October 29, 2021.

How To Protect Your Photos From Bit Rot

2016-12-19T00:00:00-08:00

Crocodile River, Johannesburg, South Africa

Download Elodie, the EXIF-based photo organizer app I made to manage my photos, and easily replicate the workflow in this post. You can also view the open source command-line version on GitHub.

Bit rot, or more accurately data rot, is when one of the bits in a file gets flipped. This happens for a multitude of reasons including inadequate insulation and humidity. A single corrupt bit can result in an viewable photo or video.

Of all the failure scenarios, bit rot is one of the more difficult to know about and prevent. The primary reason is because bit rot happens silently. You won’t know about it until after it’s already happened and by then it’s often too late.

Want to feel despair? Do a Google search for how to protect against bit rot.

It’s all just a matter of risk management. Except bit rot. That shit’s inevitable. — kngspook via HackerNews

What if bit rot wasn’t inevitable?

One way to protect your photos against bit rot is to know all the correct bits for every single photo. The best way to do this is to keep a checksum or fingerprint of every photo in a database. That way you can take a photo, calculate its checksum and compare it what’s stored in the database. If the fingerprints don’t match then you know that the photo has changed.

That’s as far as any of the articles you read will go.

I wanted to find a solution that was easy and required little to no effort on my part. If a photo became corrupt I’d like to be told. Perhaps an email or something.

How I built bit rot protection into my photo archive

It turned out that I already had a database with a checksum of every photo and video in my archive. I wrote some software that automatically organizes and archives my entire photo library (source code).

Sound interesting? Get a free chapter from my book, Photo Archiving for Nerds.

It was really easy to add a command to check every photo in my archive against the checksum database. Here’s what I got when I ran it the first time.

*sigh of relief*

Scheduling an automated daily bit rot check

Checking my photo archive for any bit rot is a simple command. Now I can schedule that command to run every day. Since my photo archive is replicated to 3 different locations I could easily retrieve a known good copy.

It made sense for me to schedule the bit rot check on my Synology NAS at home since 1 of my 3 archives is on it. How you choose to schedule the check is entirely up to you. You can use cron (Linux, MacOS), Automator (MacOS) or Task Scheduler (Windows).

Getting notified when bit rot is detected

I scheduled the nightly script to email me if it detects any bit rot to one or more of my photo or video files. If I get this email I can verify the file and replace it from a known good copy from my Laptop or from Google Drive since those are 2 of my 3 backup sources; the Synology being the 1st.

An email I got when bit rot was detected in one of my photos

For the time being, I’m perfectly okay getting this email and taking some manual action on the corrupted photo. It shouldn’t happen very often and when it does I am okay taking 5 minutes of my day to do a quick sanity check.

I could automate the script further to look through revisions of the file until it finds a checksum match. But I’ll save that for another blog post.

Make sure you read my other posts in this series.

How To Protect Your Photos From Bit Rot was originally published in Vantage on Medium, where people are continuing the conversation by highlighting and responding to this story.

How To Protect Your Photos From Bit Rot was originally published by Jaisen Mathai at Jaisen Mathai on December 19, 2016.

One Year of Using an Automated Photo Organization and Archiving Workflow

2016-12-13T00:00:00-08:00

Crater Lake. Crater Lake National Park, Oregon

Since I wrote this, Google Photos and Google Drive have stopped working together. I’m now using a Google Photos plugin for Elodie I wrote to simulate the same functionality which existed prior to this change.

It’s been a year since I put all of my photos on autopilot. All I need to do is AirDrop the photos and videos I want to keep and they’re organized and archived automagically. How, you ask?

Every media file gets de-duplicated against my entire photo collection.
The metadata is carefully inspected and geolocation coordinates are turned into city names.
The media file is deterministically placed in an appropriate folder based on date, location and other EXIF attributes.
A normalized filename including the date and EXIF title attributes are used when moving the media file to its final destination.
The source media file resides on my laptop in a folder based on date, location and other EXIF attributes. Here’s an example.

├── 2015-07-Jul│   ├── Mountain View│   │   ├── 2015-07-19_17-16-37-img_9426-walking-around-downtown.jpg

One copy of the media file is escorted to my Synology NAS to serve as a local-remote copy using the identical folder hierarchy and file name as my laptop.
A second copy of the media file is whisked away to Google’s servers to serve as a remote copy using, again, the identical folder hierarchy and file name as my laptop and Synology.
Google Photos indexes, analyzes, and applies machine learning to the photos and makes them accessible via web and mobile apps.

Current Statistics

The size of my photo library is smaller than a lot of people I speak with because I only keep about 25% of the photos and videos I take. The rest are shown the trash can. Here’s a breakdown.

13,105 photos
142 videos
30 audio recordings
828 folders
228 gigabytes

Grading My Workflow

I originally intended on this being an experiment to see if I could archive my photos without using any sort of database besides the filesystem itself.

Answer: yes. Grade: A+.

The full list of things I learned is too long for a blog post. That’s why I’m writing a book named Photo Archiving for Nerds.

EXIF Is More Powerful Than I Thought

Relying on EXIF was a critical hypothesis I made early on. I knew reading EXIF from JPEGs was doable.

It turns out that EXIF is supported by many RAW formats and while I don’t shoot in RAW it’s nice for when friends send me their RAW files from times we’re together.

Videos were the biggest question mark for me. I wasn’t sure how widely they supported EXIF. I was pleasantly surprised when I found out videos shot on iPhones have date and location EXIF embedded into them.

Turns out audio files, of the m4a persuasion, also support EXIF and iPhones embed them with date and location EXIF as well.

It felt like Christmas over and over again. But why stop while you’re ahead?

I moved on to investigating if I could reliably write EXIF. You know, for times like when you take photos with an SLR and location information is missing. Ultimately, I was able to edit EXIF on photo, video, and audio files.

Basically, every photo in my library has an embedded database with date, location, title, album, and more information.

A Few Words On Backups

My goal for backups was simple. It should be statistically impossible to lose a photo. Further, I wanted my backups to retain all of the edits and organization effort I put into the photos I browse while on the train or share with my family.

To make it impossible to lose a photo I needed to make sure there wasn’t any single point of failure. That ruled out relying on a cloud service like Dropbox or Google Drive as my “backup”.

I went with a workflow that kept a copy of each photo on my laptop, on my Synology and in Google Drive. I was safe unless Google Drive deleted my photos at the same time my house was on fire while I was there. Not to mention both my Synology and Google Drive keep multiple versions of each file. Statistically impossible enough for me.

Make sure you read my other posts in this series.

One Year of Using an Automated Photo Organization and Archiving Workflow was originally published in ART + marketing on Medium, where people are continuing the conversation by highlighting and responding to this story.

One Year of Using an Automated Photo Organization and Archiving Workflow was originally published by Jaisen Mathai at Jaisen Mathai on December 13, 2016.

My Automated Photo Workflow using Google Photos and Elodie

2015-12-07T00:00:00-08:00

Summit at Mission Peak. Fremont, California

This is a story of how two complete strangers, Google Photos and Elodie, with their own lives and ambitions met and fell in love. It’s the last of three posts on how I created an automated workflow for my photos and videos. You should understand why I wanted an automated workflow and how the software I wrote (Elodie) achieves that.

This post is long despite my efforts to keep it short. There are a lot of facets which coalesce into my workflow but I think you’ll find it worthwhile.

Let’s get started, shall we?

Google Photos is spectacular; something I can’t recall saying about another online photo service. The best features are search and Assistant.

Sharing photos and videos taken by me or created by Assistant is really easy. You can share the photos themselves or you can share links which work well for videos.

A video created by Assistant which I shared with Rachel. (URL modified)

Search works exceptionally well. Training Google Photos to recognize people in your photos requires virtually no effort and it’s really accurate. Kudos to the team for making search actually work on a personal photo library.

Assistant is a feature of Google Photos which continuously combs through your photos looking for fun things to make for you. I’ve had it make collages, animated gifs, panoramas and movies. What it creates is almost always awesome. Receiving a notification that Assistant has created something for me continues to be exciting. Paul Stamatiou summed it up well.

Paul Stamatiou on Twitter

Still LOVING Google Photos. (and still uploading). Assistant is always making me cool shit. It’s my lil’ butler. pic.twitter.com/WYqr89UuBw

Elodie helps me automatically archive my photo library

Elodie was created to automate the burden of organizing and archiving my photo library. It does this by relying entirely on EXIF instead of a proprietary database and keeping the folder structure in sync with what’s in EXIF.

Elodie does a lot of things well but 3 of them are the most important.

Organize photos and videos into folders while ensuring file names follow a consistent format.
Provide ways to wirelessly add photos to my library by using AirDrop, Google Photos’ iOS app or something else I might come up with.
Update the folder structure when changing or adding EXIF to photos and videos.

Using Google Photos with Elodie

Google Photos and Elodie work entirely independent of one another. But it turns out they also compliment one another nearly perfectly.

I’d love to use Google Photos by itself but it fails to future-proof my photo library. That’s where Elodie comes in.

Combining Google Photos and Elodie means I have amazing search, machine learning, artificial intelligence and sharing combined with a future-proof archive of my entire photo and video library.

The workflow starts with a single manual step and the rest is fully automated. The first manual manual step can be automated but I prefer to curate the photos which end up in my library. Here’s what the workflow looks like.

My automated workflow using an iPhone, Macbook, Google Photos, Elodie and a Synology.

This is how it works. I install the Google Drive desktop app and set Google Photos and Google Drive to work together. Then I ask Elodie to organize my photos into ~/Google Drive/Google Photos. At this point all of my photos, which Elodie has organized, are accessible through Google Photos.

As the diagram shows, I’m able to replicate a copy of my entire photo and video library to my Synology at home giving me 3 distributed copies of my photo library that’s automatically kept in sync.

That’s pretty awesome but there are some advanced uses which really puts my workflow over the top.

Saving Google Photos Assistant’s work to my library

I mentioned that Assistant creates some great pieces from my photos. The downside is that they are not available outside of the Google Photos app. It’s of little value to me unless I can archive them into my library; something Google Photos doesn’t easily support yet.

My favorite creations from Assistant are the collages and movies. There’s a lot of smarts going on behind the scenes. The photos selected are always good which results in creations I continually want to save back into my library.

A “stylized” photo created by Assistant being exported to Elodie using AirDrop.

Google Photos does support the ability to share Assistant’s creations. This means I’m able to AirDrop them to my MacBook which triggers Elodie organizing the photo or video into my library. All of it works almost too well.

I was very pleased to find out Google Photos retains the date and location of the photos it uses and puts them in the EXIF. That attention to detail lets Elodie organize photos precisely where they belong.

Albums in Elodie and Google Photos

Elodie lets you group photos within a folder into an album. Since Elodie only understands EXIF she does this by adding a custom EXIF tag. Now when Elodie processes the photo or video it will be placed in a folder named the same as the album.

Google Photos also has a notion of albums (sometimes called Collections in the UI). But these albums do not exist outside of Google Photos which you may have guessed by now have little value to me.

The great part is that Google Photos understands the folder structure enough to group photos in a folder together. You can search or view photos in an album with Google Photos’ great visual UI. The option to share is present but has never worked for me. I don’t know if this is a bug or not but it’s minor in the bigger picture of what I’m able to do.

An album in I created using Elodie of a hike to Mission Peak which I can view through Google Photos.

Using Google Photos to add photos to my library

My preferred method of getting photos off my phone and into my library is using AirDrop. Turns out my laptop isn’t always on and it’s not always near me. Thankfully Elodie can watch arbitrary folders to trigger her rules engine and Google Photos deterministically chooses where to upload photos.

This means I simply need to ask Elodie to watch ~/Google Drive/Google Photos/2016 and I’m able to use the Google Photos app the same way I use AirDrop. The workflow which gets enacted is a bit different but the end result is identical. See the workflow diagram earlier to understand the differences.

I can get photos and videos from my phone and onto my laptop without being in proximity to my laptop. Elodie then organizes them.

Replacing the Photos app on my phone

The cloud icon on the top left photo indicates it isn’t synced.

It’s been a long unrealized dream of mine to replace the Photos app on my phone with something that was as good or better while allowing me to not run out of space. Lots of apps promise this but I haven’t been able to do it until this combination of Google Photos and Elodie.

In the Google Photos iOS app’s main view it merges the photos on your Camera Roll with the ones in Google Photos. A small cloud icon on the thumbnail indicates that it has not yet been synced to Google Photos.

There’s some brilliance wrapped up in that cloud icon. It gets a little tricky since I don’t have any single way to get a photo or video into my library. I might AirDrop it from my Camera Roll or upload it through the Google Photos app. Either way it’s an offline workflow which invokes Elodie on my laptop to move it to its final destination.

Let’s take the example of the screenshot above. If I AirDrop that photo of myself with a latte to my laptop it will get organized by Elodie into my Google Photos folder. That photo gets synced via Google Drive and when I come back to my Google Photos app the cloud icon will be gone; even though I never uploaded the photo via the Google Photos app.

I couldn’t really ask Google Photos and Elodie to play together any better.

Final thoughts on my automated workflow

I’ve spent a lot of time over the past decade coming up with something that helps ease the burden of having tens of thousands of photos. This current setup is my holy grail. It turned out an order of magnitude better than I had expected.

Elodie has evolved quite a bit since I started using her two months ago and now I consider her indispensable. You can find Elodie on Github and help me make her even better.

Make sure you read my other posts in this series.

Commit graph from Github.

jmathai/elodie

My Automated Photo Workflow using Google Photos and Elodie was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

My Automated Photo Workflow using Google Photos and Elodie was originally published by Jaisen Mathai at Jaisen Mathai on December 07, 2015.

Understanding My Need for an Automated Photo Workflow

2015-12-01T00:00:00-08:00

Tuolumne Meadows. Yosemite, California.

Looking for Methuselah while motorbiking from Vegas to Yosemite through Death Valley.

I wanted a better way to archive, unify and experience my photos and videos. Of all the great software that’s available today none of them addressed my needs.

I came up with a solution and I wanted to write about it because I don’t think I’m alone in wanting a workflow that brings sanity to the tens of thousands of photos and videos I have.

My goals in order of importance

Preserve. My photo library needs to be future-proof for decades into the future.
Unify. Photos from my wife and my phones need to be combined into a single library.
Experience. The photos and videos need to come to life and help us re-experience the moments when they were taken.

Given the plethora of software I should briefly explain why I continue to have this problem.

iPhoto, Aperture, Lightroom and other desktop apps

Most desktop apps rely on an internal database which they use to keep track of photos in your library. They also store files on your hard drive in ways that make most sense to them; not you.

Proprietary databases and file structures fail to future-proof my photo library. It’s difficult to stop using these programs once you’ve invested a lot of time in them.

Preserve? D- Unify? C Experience? B+

Flickr, 500px, etc.? Sorry.

Online photo services also store information in their own database. It’s not so bad if they give you a programmatic API to manage your library but it still falls short. If you’ve ever tried to move all of your photos from one service to another then you’ve felt this pain. I do not envy the person tasked with that job.

The experience they provide is pretty great since you can share and converse around photos but the lack of preservation is a deal breaker.

Preserve? F+ Unify? B- Experience? A-

If you really like Google Photos you’ll want to read through this series because I come back to using it later.

Building a Better Mouse Trap

I’ve been doing this long enough to know whatever I end up choosing needs to be simple, feel natural and be rewarding.

So daunting.

Breaking apart various NAS devices to get our Trovebox software running on them. (circa 2014)

Let’s start with the basics

I started thinking about what my utopia would look like. I wanted something simple and elegant; not complex or cumbersome. In addition to meeting the goals listed above the workflow needs to do a few things well.

Work entirely off the file system and EXIF contained within each photo and video.
Provide an easy way to get photos and videos from my wife and my phone into our library.
Be fast enough that I didn’t have to wait minutes at a time to add videos which could be up to 1GB in size.

There’s more but it’s good to start with a small list and let your experience of using the software influence how it expands.

Why use the file system?

The file system is the lowest common denominator. Everything else we use is built on top of it. Once you start building too much magic on top of the file system is when you start getting into trouble.

Unfortunately, using the file system by itself is very limiting. You can’t access it remotely across your devices, for example. It’s something we’ll have to figure out a solution to but we don’t need to know how yet.

Why use EXIF?

EXIF makes every photo in your library portable. If you store information about the photo in EXIF then you keep it with you by simply copying the file somewhere else. This is probably the most important decision from which I expect all sorts of good things to flow. We don’t need to know and understand the benefits of leveraging EXIF. You’ll have to trust my gut on this.

Bonus, videos support EXIF.

Extra bonus, iPhones store time and location information in EXIF of videos. Android users aren’t entirely out of luck because we can always add EXIF where it doesn’t exist.

Getting photos and videos off our phones quickly and easily

We take all of our photos on our phones (iPhones, at the moment). Getting those photos off has proven to be pretty tedious. Transferring videos over the Internet is too slow and connecting cables to my laptop was a pain.

AirDrop seems to be a sane candidate but we’ll have to figure out how to make that work for our needs.

Start small, think big

If you make the right choices up front then the latter work tends to be pretty easy. There’s a lot we don’t know and some of those things are a pretty big deal.

Setting up load balancers for Photagious before racking them at Level3 a decade ago. (circa 2004)

How will my wife and I be able to see all of these photos on our phones and how will we be able to easily search for the photos we want to see?

How will we get all of our existing photos into this library and more importantly how will future photos be added?

We’ll figure all of this out but you’ll need to read my other posts to see how.

Make sure you read my other posts in this series.

Understanding My Need for an Automated Photo Workflow was originally published in Vantage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Understanding My Need for an Automated Photo Workflow was originally published by Jaisen Mathai at Jaisen Mathai on December 01, 2015.

Introducing Elodie; Your Personal EXIF-based Photo and Video Assistant

2015-12-01T00:00:00-08:00

Banfora Falls. Burkina Faso, Africa.

Collecting shells on the beach. Pescadero, California.

I previously wrote about wanting a workflow which brings sanity to my photo and video library. This post explains my first iteration at a solution which supports that workflow.

I was inspired by Dave Liggat’s blog post outlining how he approached a very similar problem as mine. I decided to incorporate his ideas and extend them into something which suited my needs.

I wrote the software and named it Elodie which you can find the source code for on Github. It’s a work in progress :).

Form follows function

Elodie sifts through an entire library of unorganized photos and neatly places them into an organized folder structure. For the sake of simplicity we can refer to this process as “Elodie applying rules” to a photo.

The basic workflow which Elodie enables. Only the first task is manual.

Plenty of software exists to organize photos into a folder structure by date but I knew I wanted something more sophisticated than that. I decided on encoding the date, location, title and album from EXIF into the folder structure.

Let’s say Elodie applied rules to an unorganized corpus of photos.

├── Unorganized Photos│   ├── IMG_9426.JPG│   └── IMG_9432.JPG

She would turn it into something more meaningful.

├── 2015-07-Jul│   ├── Mountain View│   │   ├── 2015-07-19_17-16-37-img_9426-walking-around-downtown.jpg│   │   └── 2015-07-24_19-06-33-img_9432-taking-the-caltrain.jpg

Besides being a more meaningful representation of the photos and videos themselves this structure will prove extremely valuable as we go along.

The file system as a real-time reflection of its contents

It wasn’t long before I began thinking of the file system as a real-time reflection of the files contained within it. If I add location information to the EXIF of photos taken on my dSLR then the file system should be updated to reflect that. Files should be renamed and moved according to their new meta-data.

Idempotence is a term in computer science which states that the same operation can be safely applied over and over again. Idempotent systems are resilient, predictable and elegant.

By making Elodie’s rules idempotent we could achieve synchronization between the folder structure and the files’ meta-data.

I ran 11,000 photos and videos through Elodie’s rules engine and I got back an organized library of photos. I could add meta-data anywhere it didn’t exist or needed to be changed and Elodie would add it to the EXIF and reorganize the photos accordingly.

Giving Elodie a facelift

I never intended on creating a UI for Elodie. My intentions were that she remain “headless”. But as the usefulness of the software increased it made sense to have a UI which matched the functionality.

In keeping with the intended simplicity of Elodie I wanted an unobtrusive UI. Something tucked away that I didn’t have to think about opening unless I needed to use it.

I wrapped the command line libraries in a taskbar app. An icon sits silently in the taskbar so you can drag photos over it which reveals a dialog. That dialog lets you quickly and easily batch edit the meta-data on photos. Like everything else that’s happened with Elodie the taskbar app has turned out much better than I originally envisioned.

Accounting for photos taken in the future

What about photos taken in the future? I needed an easy way to get them flowing into my organized photo and video library. I knew what I didn’t want. No cables. No slow uploads over the Internet. No “auto backup” of my entire camera roll.

El Capitan at Yosemite, CA

That rules out the usual suspects for getting photos off a phone. Fortunately with El Capitan I’m able to AirDrop photos from my iPhone to Macbook. My wife can also AirDrop photos to my Macbook.

AirDrop it is.

Any photo I want to add to my library I can simply AirDrop to my laptop. Anyone else can, in fact, AirDrop photos and videos to my laptop and they are organized into my photo library.

Easy. Fast. Automatic. Built-in.

Informing Elodie of new photos to process

AirDrop puts files into your ~/Downloads folder so that’s a good place to start. Let’s have Elodie do her magic any time new photos or videos show up there. But if we can do this for ~/Downloads we can do it for any other folder too. Pretty cool.

Elodie is sophisticated enough to not process random images or movies that show up in ~/Downloads. Your screenshots won’t end up in your library.

I’ve been doing this for a few weeks and it works really well. AirDrop works great and I use it 95% of the time but I found myself in situations where I wanted to offload photos while my laptop wasn’t next to me.

Is that even possible? Glad you asked.

Using Elodie with Dropbox or Google Drive

I have Elodie watching ~/Downloads for new photos and videos so I can use AirDrop. I could just as easily have her watch another folder inside ~/Dropbox or ~/Google Drive.

That lets me use their respective mobile app to upload photos from my phone to the cloud and my laptop will eventually sync the files down into the additional folder which Elodie is watching. It works really well because it is completely asynchronous.

Further you can select a folder in your Dropbox or Google Drive as the location which Elodie puts your organized photo library. This gives you a simple offsite backup. You can even use their mobile apps to access your photo library. I originally did this with Dropbox and it worked really well.

Using Synology CloudSync for a 3rd backup

You can never have too many backups and one of them should absolutely be in your home. Since I’m sync’ed with a cloud service I can use Synology’s CloudSync program to automatically keep an updated local copy of my entire photo library safe within my home.

Synology NAS (photo credits, CNET)

With no additional work I obtained three geographically distributed copies of my entire photo library which automatically stays up to date.

Bringing my photos to life

I’ve been really happy with Elodie so far. The only piece missing is the most rewarding.

How can I enjoy my photos now that I have them organized, backed up and preserved?

In my next post I’ll talk about how I use Google Photos along side of Elodie without compromising any of my originally stated goals. It’s pretty amazing so you’ll definitely want to read it.

Make sure you read my other posts in this series.

Introducing Elodie; Your Personal EXIF-based Photo and Video Assistant was originally published by Jaisen Mathai at Jaisen Mathai on December 01, 2015.

Startup Acquisition Video Log

2014-09-15T00:00:00-07:00

2014-01-21 (January 21st, 2014)

My initial video documenting the last 5 months before Trovebox was acquired. At this point my primary focus was on sales. I was cold calling customers and trying to close a sales deal with Columbia Law University.

2014-02-12 (February 12th, 2014)

We arranged to have booths at 3 conferences. Columbia Law University got back with us that our proposal was too high. We were also working with University of Southern California to partner on a solution to have digital yearbooks in schools.

2014-02-13 (February 13th, 2014)

We were hitting some pretty large headwinds with proposals not landing well with regard to pricing. Our other discussions were dragging on too. Overall the long sales cycle were not looking favorable given our runway.

2014-03-05 (March 5th, 2014)

My co-founder and I decided to shut Trovebox down. This decision kicked off several secondary initiatives. The first was to shut the service down while living up to the promise of data portability. The second was identifying a plan B to identify possible acquisitions.

2014-03-09 (March 9th, 2014)

We started talking to our mentors about what to expect when shutting down and what our options were. We wanted to make sure and deliver on our promise to users of data portability and felt confident we could do that.

2014-03-18 (March 18th, 2014)

We had our first of many meetings with companies interested in an acquisition. This meeting was with 10 people at Western Digital including Directors, VPs and SVPs.

2014-03-24 (March 24th, 2014)

This starts the cat and mouse game of waiting for a response from Western Digital. We presented to them 6 days prior. The attendance grew from 10 to 15 people to whom we demoed and answered questions.

2014-03-27 (March 27th, 2014)

We didn’t hear back from Western Digital yet so I sent a follow up email. They responded that they would like to set up a follow up meeting. In the meantime, I started coding a little bit to deal with the stress.

2014-03-29 (March 29th, 2014)

We met with a Director at Dreamhost to discuss interest in an acquisition.

2014-04-23 (April 23rd, 2014)

We had an idea that I decided to file a provisional patent for before presenting the idea to Western Digital. The idea was to have Trovebox running on NAS devices and with the ability to communicate with other NAS devices. This would create a network of tens or hundreds of thousands of NAS devices which had the ability to communicate with each other. I demoed a working prototype of this on their MyCloud NAS devices.

2014-04-29 (April 29th, 2014)

The idea of running Trovebox on NAS devices continued to take shape. We essentially took all the technology we created and quickly pivoted to an entirely different product. I installed Trovebox on several additional NAS devices and shipped them to family in Ohio and New York. We started showing the prototype to others with favorable feedback.

2014-05-06 (May 6th, 2014)

More cat and mouse with Western Digital. New opportunities started to arise with NetGear and IdeaLabs. We considered pivoting to this new idea but ultimately decided not to pursue the idea and continue focusing on acquisition opportunities.

2014-05-12 (May 12th, 2014)

Demoed the prototype running on a NetGear ReadyNAS to an SVP at NetGear. Discussions with Western Digital were progressing slowly but showed signs of hope.

2014-05-14 (May 14th, 2014)

Western Digital asked us to come back and demo to an EVP who would be the final decision maker. NetGear wanted a follow up meeting as well.

2014-05-16 (May 16th, 2014)

Demoed the prototype to an EVP at Western Digital.

2014-05-19 (May 19th, 2014)

Demo with Western Digital went well. I ran into some technical difficulties preparing the demo but was able to improvise. Discussions moved forward with Western Digital and we ultimately sold the company to them. I spent the next 2 years integrating our software onto their NAS. We successfully completed this and had our apps approved in the Apple App Store and uploaded to the Google Play Store. In the final stretch, new leadership that recently joined decided to kill the product before launching. I left the company several months later.

Our idea of an interconnected network of 10s of thousands of NAS devices that could safely store your photos while securely accessing your friends’ photos without the need of a centralized service never went beyond a prototype or outside the walls of Western Digital. Many of the ideas are resurfacing 5 years later as part of Web3.

Startup Acquisition Video Log was originally published by Jaisen Mathai at Jaisen Mathai on September 15, 2014.

My Shuttleworth Fellowship

2014-09-11T00:00:00-07:00

We are the Shuttleworth Foundation and we fund people who are unafraid to re-imagine the world and the way we live in it.

There exists in Cape Town, South Africa a small group of people taking bold risks by investing in people ambitious enough to change the world through openness. I only came to know about the Shuttleworth Foundation in 2012 while we were looking for funding sources. It’s founded and funded by entrepreneur Mark Shuttleworth .

On the surface the Foundation looks like part venture capitalist and part foundational grant institution. It’s an accurate description but to say that’s all the Foundation is would be a gross understatement.

Bi-annually the Foundation gives up to 3 years of fellowship to individuals working on something in an open manner . That includes openness in education, science, data, government, software and more. The Foundation measures success not in dollars but in positive social change.

My fellowship for Trovebox

My fellowship was for Trovebox (originally OpenPhoto). We set out to make possible the preservation of digital photo archives for posterity. Openness is innate to solving that problem. Ideas like data ownership and portability need to be cornerstones; not an afterthought. Our approach of open sourcing our software and experimenting with distributed storage for digital media are what made it a good fit to receive a fellowship.

We experimented with and prototyped having Trovebox installed on a NAS. Credit: Jaisen Mathai CC BY.

A mix of funding and community

The Foundation provides over $1,000,000 in funding over 3 years. It takes a lot of resources to do big things and the financial support provided by the Foundation is critical. But it’s the less interesting of two components.

I came for the money and stayed for the community. – Me

It’s impossible to know how the community plays into a fellowship until you arrive at your first Gathering. The Foundation has bi-annual Gatherings where every fellow, current and alumni, converge in a single location. It’s a week crammed together in a room talking about everyone’s projects or around a dinner table discussing everything from the lack of transparency at large corporations to putting educational materials into the hands of the poor in South Africa .

At a Gathering in Edinburgh, Scotland. Credit: Jason Hudson CC BY.

I’ve been surrounded by brilliance at many points in my life but I’ve never been inspired like I am when I’m with this group of people. The support system between fellows was unparalleled.

Experimenting is in the DNA

Each fellowship is unique despite sharing a common thread of openness. The Foundation understands that to get from point A to point B you might have to go through point C. And some of those times you end up at point Z which was more interesting than point B in the first place. I appreciated that they were always supportive of experimenting and changing course as a result.

I recall my fellowship and some of the changes we made which led me to feel we weren’t making any progress. The Foundation remained patient and supportive as I eventually found my way back to where I started. But this time I arrived equipped with a war chest of knowledge and a completely new and novel approach.

It took us 3 years but it had to take us three years. - Me

Most amazingly I remember talking to Helen and Karien in Budapest at the Gathering in May of 2014 about my contemplation of not applying for a third fellowship. I was excited about what we learned and the new approach we had at making permanence of media archives possible. At the same time I was exhausted and couldn’t see the path to sustainability. I’ll never forget Helen’s response urging me to apply again. That’s not the type of commitment or support you come across often.

Ultimately I decided not to apply; one of the most difficult decisions during this journey.

Becoming an alumni

By now I’ve been an alumni for just over a week. The Foundation published Arthur and my exit from the fellowship program on their site today. Reading that page and watching the video made the transition from my fellowship to the next chapter quite concrete.

It’s hard to put into words how much my fellowship meant to me and how much I learned as a result. As I continue towards our original goal of permanence for digital media I’ll do it with more openness, more ambition and with the support of a small army of people around the world.

Once a fellow, always a fellow.

My Shuttleworth Fellowship was originally published by Jaisen Mathai at Jaisen Mathai on September 11, 2014.

Timeline

2014-09-10T00:00:00-07:00

Over the 3 years we spent building Trovebox there were countless successes and failures. All of which helped shape the company, product and founders.

Below is a timeline view of events that makes it possible to quickly consume most of the ups and downs we had.

May 2011 - December 2011 (before funding)

The first months were some of the most exciting. We built a product from scratch and gained a following.

May - I left my job to start OpenPhoto. The first milestone was a successful Kickstarter campaign .
Aug - OpenPhoto was accepted into Mozilla WebFWD
Sep - OpenPhoto was a standby for TechCrunch Disrupt. We didn’t get to participate in the contest but were offered the chance to have a table in Startup Alley.
Sep - Pitched to Kapoor Foundation by way of intro from Marco DeMiroz. Did not get past first meeting.
Oct - Version 1.2 of OpenPhoto was released (7 contributors).
Oct - Initial beta version of iPhone app via TestFlight.
Oct - Pitched to Khosla Ventures. I met Vinod Khosla at TC Disrupt who asked that I contact him afterwards. Did not get past first meeting.
Nov - Version 1.3 of OpenPhoto was released (4 contributors).

January 2012 - July 2012 (before funding)

The first half of 2012 was largly focused on securing funding.

Feb - Was invited to interview with IO Ventures. This is a selective process and they receive 700 applications of which 40 get asked to interview. We did not get accepted.
Feb - Was invited to pitch to Band of Angels. This is a selective process and we were 3 of 50 companies selected to pitch. Did not get past the pitch.
Apr - OpenPhoto iPhone app released in the App Store.
May - Released “Import from Flickr”.
Jun - Released “Import from PicPlz”. This was our first initiative at providing an option for users of sites that were shutting down or making terms of use changes.
Jul - Version 3.0 of OpenPhoto was released (18 contributors).

August 2012 - December 2012 (after funding)

After receiving funding we got back to focusing on the product.

Aug - Received Shuttleworth Foundation fellowship.
Nov - Added support for 5 cloud storage providers.
Dec - OpenPhoto Android app released in the Google Play store.

2013 (after funding)

As we entered 2013 we focused almost entirely on sales and marketing to fuel growth. We decided in May to pivot to targeting businesses.

Jan - OpenPhoto relaunched as Trovebox (6 contributors).
Apr - Archive.org support added.
May - iPad support added to iOS app.
May - Realized that growth with consumers was too low and that we needed to find an alternative.
May - Partnered with Insight Sales Dynamics (ISD) and worked with Brian to land large enterprise deals.
May - (ISD) Meeting with CEO from Topia Software. Had 3 calls but did not close.
Jun - (ISD) Meeting with VP at Sony. Met in person 4 times over 8 months but did not close.
Jul - Group support added to Trovebox.
Jul - Received 2nd year Shuttleworth Foundation fellowship.
Sep - (ISD) Meeting with VP at Roku. Had 2 calls but did not close.
Oct - Switched Trovebox from consumers to businesses.
Dec - Released Trovebox Collect.

2014 (after funding)

This was an important year for us as we focused on growing our customer base by targeting businesses. We tried everything from cold calling companies to exhibiting at trade shows.

Jan - Began cold calling group travel vacation companies. This lasted a month and the results were terrible.
Feb - Launched beta of video support.
Mar - Exhibited at 3 expos. This was sort of a last ditch effort to try and figure out a strategy or market which would provide significant growth for us with businesses. This wound up not working out and we decided that we couldn’t get the growth we needed.
Mar - Considered Kickstarting a TroveboxHome project which would be a physical network attached storage device that ran Trovebox software or otherwise communicated intelligently with Trovebox in the cloud. After speaking with Dr. Song we decided not to pursue this idea.
Mar - Reached out to prior ISD contacts about opportunities and Western Digital was interested. This resulted in a meeting with about 15 key people from Western Digital’s Branded Products group.
Mar - Met with Western Digital and demoed our product. We had met with them a year earlier to private label our software but nothing materialized.
Mar - Met with Dreamhost to discuss interest in an acquisition.
Apr - Prototyped Trovebox running on a Western Digital MyCloud device. Demo’ed to Western Digital.
May - Prototyped Trovebox running on a Netgear ReadyNAS device. Demo’ed to Netgear.
May - Demo’ed to Drobo.
Jun - Signed Letter of Intent from Western Digital to be acquired.
Jul - Joined Western Digital.

Timeline was originally published by Jaisen Mathai at Jaisen Mathai on September 10, 2014.

The Struggle

2014-08-07T00:00:00-07:00

There were many days and even weeks where I was convinced all this was a huge mistake. Ben Horrowitz, a well known venture capitalist, blogged on this topic in 2012 in blog post also called The Struggle .

It’s complicated

Us humans are neither binary or entirely logical; thank goodness. We’re complex creatures and what constitutes happiness, contentment, joy or any of the opposite emotions depend on a large number of factors. This makes The Struggle even more acute.

It started for me near the end of 2011 when we were failing to raise funding. I knew we absolutely needed to secure funding to succeed. I won’t go into the details of our fundraising journey in this article but you can read about it here.

Being a team

The pros of having a co-founder undoubtably outweigh the cons. But I’d be lying if I didn’t admit that having a co-founder increased the pressure I put on myself. If we couldn’t figure out how to make Trovebox successful then I’d feel some guilt along with it.

Santa Cruz with Patrick during his first visit. Credit: Jaisen Mathai CC BY.

Understanding my motivations

There was a clear consequence of not getting investment; we’d have to abandon and shut down OpenPhoto. Founders start companies for various reasons but there’s always a common thread of believing strongly in something. It might be a belief that building something will improve people’s lives, make you a lot of money or change something that’s in need of change. Most often it’s a combination of these that drive founders.

I believed OpenPhoto needed to exist. I’m a bit of a nostalgic. I love to relive past moments. Furthermore it became really important to me that we preserve the memories of our family’s everyday life. One day I hope to sit with my kids and show them photos of their childhood; much like my own parents did with me.

I believed something needed to be changed. Looking around at how everyone dealt with, or didn’t deal with, archiving photos I knew something had to change.

I believed we could build a successful busines. I’ve always been driven to create businesses. In middle school I would buy Zotz candy for $.03 and sell it for $.05. A hefty 66% margin.

As with any strong belief it’s difficult to come to terms with the idea that things won’t pan out the way you believed.

But there’s a tougher reality that had dawned on me.

Does shutting down mean failure? How would I explain this to friends and family who thought I was making a mistake when I left Yahoo? Do I just go back and get a normal job?

Hello ego, pride and insecurity. Not traits I’m particularly proud of but they helped motivate me to continue building OpenPhoto. I imagine ego drives more entrepreneurs than you’d think.

The effect of having no money

Louis C.K., a comedian, has a great bit titled Being Broke . Besides being hilarious I felt like I could relate¹.

You ever have negative money? You look at your bank account and you have negative $10. […] I wish I didn’t have anything but I have less than that. If it’s free I can’t f*****g afford it.

It turns out that the lack of money can act as a crucial forcing function. When you’re facing the reality of shutting down a startup because you can’t pay your bills you’ll quickly focus on what matters most. Even in hindsight I know we didn’t waste much time focusing on the wrong things. I have our dwindling bank account to thank for this.

Growth is everything

There was one thing that rang true on each day of Trovebox: growth. Growth shares many characteristics with fundraising. Namely growth is largely out of your control and all you can do is try and make the stars align.

The feeling of not acheiving growth was as stressful as when we couldn’t finding funding. It was a sign that the product was headed in the wrong direction and it felt like we were trying to turn the Titanic around.

I spent the beginning of 2014 going through a spreadsheet of 600 potential customers we could sell to. I called them up on the phone and in the unlikely event there was interest it started a series of calls to close a sale. None of them panned out.

Perhaps our product wasn’t solving a need as critical as our earlier conversations with customers implied. Maybe it was that we grossly underestimated the length of the sales cycle.

I don’t believe we’ll ever know.

Dashed hopes

Then you have the golden opportunities. Ones which have all the signs and promise of being a home run. We had promising meetings with execs from Sony, Netgear, Roku and Western Digital. We even met with Sony 4 times in San Diego. Nothing.

Smoking cigars with our sales lead to celebrate one of our meetings. Credit: Jaisen Mathai CC BY.

These were particularly difficult because when we crashed we crashed hard. Forging partnerships with companies is difficult because you have little insight into their roadmap. A company maybe highly interested but then go silent for 6 months and come back to restart the conversation. You can only do your best and stay on their radar.

Everyday will bring a struggle

Those are some of the larger struggles I remember. But every day brings struggles. A lot of it is psychological and emotional. Keeping on top of that without losing your sanity isn’t easy.

That’s why it’s so critical to have a good support system. Else you’ll got nuts and take your company down with you.

¹ Just a disclaimer that I’m not implying I was poor. When I think of people being poor it’s not just a lack of money but a lack of opportunity. I empathize greatly with people who find themselves in that situation. I never considered myself poor despite not having money.

The Struggle was originally published by Jaisen Mathai at Jaisen Mathai on August 07, 2014.

My Support System

2014-06-05T00:00:00-07:00

It was over 3 years ago when this journey started. I thought if things went terribly that it would last as little as 6 months and if things went really well it could have gone on for a decade.

Curveballs.

It’s the part that makes startups hard: managing your own psychology when it looks like everything is falling apart.

So many times it felt like everything was falling in around us. I can recall the 3 times Patrick and I talked about how there’s no light at the end of the tunnel. Twice on Castro St. in Mountain View where we worked when Patrick was in town we planned how we’d shut Trovebox down.

When they say your chances of success increase if you have a great co-founder this is what they must have been talking about. On my worst days it felt like Patrick was the only one who could pull me out of the depths. Those days were plentiful.

Family

Despite the consuming nature of doing a startup my first commitment was always to Rachel and our 2 kids. Rachel didn’t know what she was signing up for on that day when I asked her what she thought about me leaving my job and decent health insurance along with it. Life threw us many curveballs but I had her unflinching support at every turn.

Mi familia. Credit: Jaisen Mathai CC BY.

Then there’s my parents who think I’m crazy. That gets reinforced each time I chase a startup idea. This is the second time I left a job to start something. They want nothing more than for me to get a job at a big stable company and be a loyal employee until I retire. Nonetheless they supported me as they have in the past despite thinking I was being foolish.

The Shuttleworth Foundation

I wrote in more detail about my experience as a Shuttleworth fellow if you wanted to read it. But to make a long post short they’ve been the best partner I could ask for. Not only did they provide nearly $1M in funding but they provided a network of the brightest and most ambitious people I’ve met. All working on something that makes the world a better place. There’s nothing else quite like it.

Shuttleworth Gathering at the MIT Media Lab. Credit: Jaisen Mathai CC BY.

Friends

There were countless friends who believed in me as well. They helped me keep my sanity when I questioned if I had made a terrible mistake.

It doesn’t take a village to build a startup but it helps to have people in your corner. You don’t need them every day but you will need them.

My Support System was originally published by Jaisen Mathai at Jaisen Mathai on June 05, 2014.

UX Nirvana ~ Elegant Pagination with Infinite Scrolling

2014-04-04T00:00:00-07:00

We just pushed the impossible to production; infinite scrolling with pagination that actually works.

Trovebox is a media management and sharing service for teams and organizations. Start your free trial today.

Two years ago our photo gallery looked much different than it does today. It was a grid of cropped photos with pagination links. The user interface was terrible and the user experience worse.

version 1.0 ~ our arcane grid layout with pagination

Photo collections often contain tens of thousands of photographs and serve a single purpose; to be visually consumed. Moving your mouse cursor over a link and clicking to see the next twenty-five photos felt arcane. Infinite scrolling began gaining popularity many years ago but brought along new sets of problems. While I believe most users enjoy infinite scrolling there’s a vocal minority who abhor it. Rightfully so, perhaps.

Infinite scrolling tends to break a number of behaviors we’ve become accustomed to on the web; mundane tasks like deep linking to content and using the browser’s back button. Tasks we take for granted until we can no longer do them.

We didn’t have all the answers but knew that a user interface with infinite scrolling would be better than our older paginated one. Our primary goal was making it enjoyable to view as many photos as possible as quickly as possible. Our new gallery adopted a fixed height masonry layout that minimized cropping and automatically loaded more photos as you reached the bottom of the page.

version 2.0 ~ our improved masonry layout with infinite scroll

The masonry layout required less mental energy to browse photos; leading to a more enjoyable viewing experience.

We realized as you view photos you aren’t thinking about pages or having to click to see more. Our minds have been trained to do that but it‘s a disruptive experience. This is where infinite scroll shines.

When we launched infinite scroll we knew we gave up some nice-to-haves. They weren’t critical for us but were important enough that we created a couple issues to track the progress of their solutions.

Our infinite scrolling interface works so well you’ll find yourself on the twentieth page of photos without realizing it. That’s great news but users have become accustomed to copying a link with the expectation that it will point to what they just saw. Infinite scroll isn’t a reason we should ask our users to change their habits. The problem has many nuances though and the solution isn’t as straightforward once you put yourself in your users’ shoes.

It seemed obvious that we could use pushstate to update the browser’s URL bar as the user scrolled through pages of photos. Links copied from the URL bar would load photos starting from the page they were looking at the moment they copied it.

With a mix of Backbone.js and Waypoints we were able to quickly get the URL bar updating as the user scrolled through photos. That’s when we realized a limitation of infinite scrolling we hadn’t previously thought of; infinite scrolling tends to work as you scroll down. What if you needed to scroll up to view photos from page 9 when the link started at page 10?

I haven’t seen an implementation of infinite scrolling which handles this and we couldn’t come up with an intuitive way to make it. That led us back to the drawing board to come up with some ideas. At minimum we could let the user know there are prior pages of photos. We quickly realized that’s just as terrible as not indicating it at all.

Shortly after realizing how terrible our idea was it dawned on us that we could include pagination similar to what we had in our older user interface. But only when the gallery was deep linked; starting on pages 2 and higher.

Here’s an example and a screenshot below where you can see the pagination near the top right.

Our infinite scroll interface started to show signs of having the best of both worlds. The most common navigation was easier by an order of magnitude and less common navigation was as easy as it’s always been.

While we’re very happy with the results there’s more work to be done. How should the back button work? What about search engine optimization?

We hope to tackle these issues next. But for the time being we believe we’re 90% of the way there. That last 10% will keep us busy.

Footnotes: At Trovebox we not only use a tremendous amount of open source software but we release the majority of our work as open source. You can follow the project on Github and follow me on Twitter.

Photos in the screenshots were taken by Duncan Rawlinson.

UX Nirvana ~ Elegant Pagination with Infinite Scrolling was originally published by Jaisen Mathai at Jaisen Mathai on April 04, 2014.

Jaisen Mathai

Letters To My Kids

Sojourn for iOS Was 45 One-Shot Prompts

About the app

About the process

Baseline context

/spec

/gh-issue

Publishing and deploying

Some examples

How’s it going?

Caveats

Big Tech Wants To Be Your Bible Translator

Getting People Out Of Chat And Into Scripture

The Risk and Opportunity of AI for Faith

Giving In To Generosity

My Ridiculously Robust Photo Management System (Immich Edition)

Latent Product Development

Life of a Blockchain Transaction - (Proof of Work)

How To Protect Your Photos From Bit Rot

What if bit rot wasn’t inevitable?

How I built bit rot protection into my photo archive

Scheduling an automated daily bit rot check

Getting notified when bit rot is detected

One Year of Using an Automated Photo Organization and Archiving Workflow

Current Statistics

Grading My Workflow

EXIF Is More Powerful Than I Thought

A Few Words On Backups

My Automated Photo Workflow using Google Photos and Elodie

Google Photos helps me find, share and experience photos

Paul Stamatiou on Twitter

Elodie helps me automatically archive my photo library

Using Google Photos with Elodie

Saving Google Photos Assistant’s work to my library

Albums in Elodie and Google Photos

Using Google Photos to add photos to my library

Replacing the Photos app on my phone

Final thoughts on my automated workflow

Understanding My Need for an Automated Photo Workflow

My goals in order of importance

iPhoto, Aperture, Lightroom and other desktop apps

Flickr, 500px, etc.? Sorry.

Building a Better Mouse Trap

Let’s start with the basics

Why use the file system?

Why use EXIF?

Getting photos and videos off our phones quickly and easily

Start small, think big

Introducing Elodie; Your Personal EXIF-based Photo and Video Assistant

Form follows function

The file system as a real-time reflection of its contents

Giving Elodie a facelift

Accounting for photos taken in the future

Informing Elodie of new photos to process

Using Elodie with Dropbox or Google Drive

Using Synology CloudSync for a 3rd backup

Bringing my photos to life

Startup Acquisition Video Log

2014-01-21 (January 21st, 2014)

2014-02-12 (February 12th, 2014)

2014-02-13 (February 13th, 2014)

2014-03-05 (March 5th, 2014)

2014-03-09 (March 9th, 2014)

2014-03-18 (March 18th, 2014)

2014-03-24 (March 24th, 2014)

2014-03-27 (March 27th, 2014)

2014-03-29 (March 29th, 2014)

2014-04-23 (April 23rd, 2014)

2014-04-29 (April 29th, 2014)

2014-05-06 (May 6th, 2014)

2014-05-12 (May 12th, 2014)

2014-05-14 (May 14th, 2014)

2014-05-16 (May 16th, 2014)

2014-05-19 (May 19th, 2014)

My Shuttleworth Fellowship

My fellowship for Trovebox

A mix of funding and community

Experimenting is in the DNA

Becoming an alumni

`/spec`

`/gh-issue`