in principles ~ read.
If you're counting bytes, you're doing it wrong

If you're counting bytes, you're doing it wrong

unsplash-logoLuana Azevedo

I had a recent experience at work that was a bit of an eye-opening moment. Our team was tasked with a seemingly simple job: take a stereo file and play it as mono in the browser.

Counting bytes

As usual, the team got together for a story kickoff and quickly came up with a solution. We would "transcode" (for lack of a better word) the file from stereo to mono on the fly. A specific byte in the file header tells us the file format. If the file is in stereo, we can transcode it by performing an arithmetic operation on the byte content of both channels. We started implementation, and the story was finished and delivered to production within a week.

HTTP headache

Unfortunately, after a few days our customers started reporting issues. The file was playing in mono, but there were errors in the streaming. Playback was stuttery, and the player going backwards and forward in a loop.

We looked into this found out that there was an issue with the HTTP content-range headers that were presented to the browser. These headers are used by the browser to request a small fraction of the file at a time, and is used heavily when streaming.

This could potentially indicate an issue in the Web framework we were using. My colleague, a very capable developer, decided to solve the issue by bypassing the framework's headers and implementing our own. After all, he said, it was just a simple arithmetic operation to calculate what bytes need to be returned to the browser.

Trust your gut feeling

By this time I was feeling a bit uncomfortable with the approach we were taking. We were no longer dealing with one problem (formatting) but two (formatting and streaming). My gut feeling was that there should be an easy way to implement the story without having to perform byte arithmetics or write our own HTTP headers.

The approach that my colleague suggested didn't quite work: one week later we were still struggling to smooth out the playback issue. At that point, and facing customer pressure, we were forced to roll back all our changes and revert to the latest stable release, with the file playing in stereo.

My gut feeling was proved to be correct.

Abstraction levels in the application stack

I'm an application developer. This means that I sit at the top of the what I call the app development pyramid. we could draw it as follows:

I want to clarify that by sitting "at the top" I don't mean that my work is more important that the efforts of those who work at other levels. Building libraries and framework is challenging and fulfilling work. What I mean by that is that my work leverages all the effort put by other people in building those wonderful tools for us.

The higher you work in this pyramid, the higher the productivity (in the sense that you can build an an app with just a few lines of code), but also the higher the level of abstraction. Frameworks and libraries make it easier to reason with concepts like asynchronicity or encoding, and remove the need to deal with low-level constructs like semaphores or byte counting.

In this particular case, I would expect to find a library to deal with the formatting, and to rely on the Web framework for the HTTP content-range headers.

Is byte counting wrong?

If you're an application developer and you're having to count bytes, you're probably taking the wrong approach. Chances are, there is a different way to get the job done without having to dig that deep into the implementation details.

Of course, if you're building operating systems or embedded firmware, counting bytes is probably part of your day-to-day job.

The fix

In the end we decided to change the file formatting at the point when it was being written to a disk cache. This required reformatting the file at one single point by using a well-known library instead of trying to perform custom arithmetics on one fragment at at time. The code to stream the file remained unchanged. Within a day, the feature was fixed.

It's always good to remember at what level of the application stack you're working. The approach that works well at one level may not be right for another. If you're an application developer and you're having to count bytes, take some time to take a step back from the problem. Perhaps you can leverage an existing framework or library to do the job for you.