6 minute read
This September, we quietly launched a new version of the Shopify admin. Unlike the launch of the previous major iteration of our admin, this version did not include a major overhaul of the visual design, and for the most part, would have gone largely unnoticed by the user.
Why would we rebuild our admin without providing any noticeable differences to our users? At Shopify, we strongly believe that any decision should be able to be questioned at any time. In late 2013, we started to question whether our framework was still working for us. This post will discuss the problems in the previous version of our admin, and how we decided that it was time to switch frameworks.
Batman
Since early 2013, the Shopify admin has been running on an in-house JavaScript MVC framework called batman.js. This version of our admin was built to attempt to solve problems with UI response time in the original, standard Rails version of our admin, and to make prototyping of new features faster and easier.
This route ultimately required completion of the project to determine whether our hypothesis held. Along the way, we encountered many challenges. Early in the project, the admin suffered from crippling memory leaks, occasionally severe enough to crash the browser. We had hypothesized that asynchronous rendering would improve application performance by affording CPU cycles for other tasks, but performance analysis proved this incorrect. We also had recurring problems with detached DOM nodes causing memory leaks. We iterated through solutions which rendered synchronously, and then later, used a combination of synchronous and asynchronous rendering. None proved truly performant - we solved previous problems, but introduced serious response time regressions on mobile devices and older computers.
Possibly the greatest difficulty of all was the burden of successfully maintaining two separate tech stacks, with a great amount of duplication between the two. Models needed to be represented both in Rails and in Batman, and both had to be tested. Business logic was leaking into the client-side rather than being isolated to the server side, or would be duplicated in both. When we added additional mobile and point-of-sale clients to our offerings, any business logic isolated to the client-side had to be duplicated in these new clients as well. In cases where business logic was duplicated across different sides of the software, it would become ambiguous as to what was the correct or intended implementation.
Developer onboarding was a significant ongoing challenge. Batman was a highly complex framework with poor documentation and a steep learning curve. Clarity in the source code was lost in the duplication of code and ambiguity of what a good implementation looked like. This meant that our goal of fast and easy prototyping, which was one of our original goals for the project, was never realized.
Rebuilding the Shopify Admin
In early 2014, we made the decision to remove Batman. The new admin would make a return to more classic architecture with some modernizations. Our approach would be ERB views and server-side rendering, with the use of Turbolinks, and a lightweight custom JavaScript binding system. This allowed us to tackle problems of code duplication and developer productivity in a single blow.
Before finalising on this approach, we analysed a number of other options including Angular. We determined that writing our own binding system would allow us to keep the framework small, and the process would help us to determine exactly what our requirements were from it. We suspected that moving to another JavaScript MVC framework would bias us towards that framework’s design. We were also skeptical as to whether any JavaScript MVC framework would be immune to the same problems we experienced with Batman, and that we would ever be able to create a native feeling application at our scale. These frameworks work well for highly interactive apps with complex user interface requirements. We only needed a small piece of what a full framework would offer. In building our own binding system, we were able to include only the functionality that we require. If this system continues to grow to a point where it resembles a full-fledged JavaScript framework again, we can make that change when the time comes.
How we got there
It is a deeply held belief at Shopify that all decisions must be able to be challenged at any time. We also highly value simple solutions to all problems, as well as getting things done. Despite the time put into developing Batman, we had cause to seriously question its continued viability. In addition to the problems felt by our merchants, our codebase suffered from poor developer approachability. If a developer cannot look at a section of the codebase and be able to reasonably infer the correct approach to take in making changes, the codebase will suffer. Duplication and poor documentation made it difficult for developers to make changes to the admin. Once we started questioning Batman, we saw that the evidence strongly indicated it was time to move on to the next chapter.
From the inception of this new project to final removal of all Batman code, it took only nine months to roll out the new admin. Because we chose a solution wherein simplicity and developer approachability was paramount, we were able to split work across many different teams - a luxury that was not previously possible. This resulted in rapid completion of the project, especially in comparison to the previous version, which had endured constantly moving goal posts. Our developers are able to work far more efficiently, and we are able to provide a faster and more stable experience for our merchants.
We still encountered challenges when developing our new admin, but many of these challenges were a result of Batman throwing a few final punches on its way out. Because the Batman admin had been so siloed within the admin team, there were some sections which should have been owned by other teams but were realistically owned by no-one. Their nuances and complexity were poorly understood by everyone, so converting these sections ended up taking much longer than it should have. However, now that the codebase can be easily understood by all members of the engineering team, sections can be owned by teams outside of the core admin team.
What success looks like
In the course of the project, we pared down our codebase a fair amount:
- Shaved 2.5MB from our minified JS bundle (including vendors), from 4.1MB to 1.6MB
- Replaced over 28,000 lines of CoffeeScript in the assets folder with under 15,000 reducing client-side code by about half, without counting vendors
- Reduced our JavaScript test suite from just shy of 30,000 lines of CoffeeScript to under 7,500
- Eliminated over 4,000 lines of Ruby
The ultimate measure of success for this project was developer productivity. We are seeing much quicker turn-around time for new features, and for onboarding of new developers. We have also been able to spread ownership of Admin areas across the engineering team.
What ultimately enabled us to make the decision to switch was strong company values. Open communication, a questioning nature as a core component of our culture. Something didn't feel right, so instead of accepting it as an unideal status quo or only making criticisms behind closed doors, we articulated the problems we were having and came up with a solution. We were able to move forward quickly towards this solution by openly accepting failure as a learning opportunity.
The Shopify admin has now been converted entirely away from Batman. In a follow-up post, we will cover some of the specific technical challenges we faced, and how we solved these problems.
Additionally: Shopify is growing, and we're looking for talented developers who can help us make commerce better. Find out more on our careers page.
View original comments on this article