Shopify heads to Dublin for SREcon Europe

Production engineers from Shopify will be crossing the pond to speak at SREcon Europe from July 11 to 13, 2016 in Dublin, Ireland. From flash sale engineering to fuzz testing to multi-tenant architecture across multiple data centers, we got you covered!

Image credit: Giuseppe Milo

Jaime Woo

Continue reading →

How to set up your own mobile CI system

Over the past few years the mobile development community has seen a dramatic shift towards the use of continuous integration (CI) systems similar to changes present in other communities — particularly web developers. This shift has been a particularly powerful moment for mobile developers, as they’re able to focus on their apps and code rather than spending their time on provisioning, code signing, deployment, and running tests.

I’m a software developer at Shopify currently working on our Developer Acceleration’s Mobile team. My job is to design, create, and manage an automated system to provide an accelerated development experience for our developers.

Based on our experiences at Shopify, we will be talking about “hosted” vs “BYOH” systems, how to provision Mac OS X and Ubuntu machines for iOS and Android, and the caveats we ran into throughout this series. By the end, you should be ready to go build your very own CI setup.

 

    Julian Nadeau

    Continue reading →

    Five Shopify talks at RailsConf 2016

    RailsConf banner

    Updated June 9, 2016

    RailsConf is tomorrow! For the first time, the conference will be in Kansas City, known for jazz and barbeque and home to the Royals. If you're heading down, here are the details for the five presentations we'll be giving:

    • How We Deploy Shopify - Kat Drobnjakovic

    Shopify is one of the largest Rails apps in the world and yet remains to be massively scalable and reliable. The platform is able to manage large spikes in traffic that accompany events such as new product releases, holiday shopping seasons and flash sales, and has been benchmarked to process over 25,000 requests per second, all while powering more than 243,000 businesses. Even at such a large scale, all our developers still get to push to master and deploy Shopify in 3 minutes. Let's break down everything that can happen when deploying Shopify or any really big Rails app.

    Wednesday, May 4, 11:40 am to 12:20 pm, Room 3501 G

    • Foreign API Simulation with Sinatra - Konstantin Tennhard

    Nowadays, we often rely on third party services that we integrate into our product, instead of building every aspect of an application. In many cases, well written API clients exist, but on occasion you run into the issue that there isn't a ready to use client or it simply doesn't fit your needs. How do you write a good API client and more importantly how do you test it without hitting the remote API. So far, the standard approach has been replaying requests with VCR or stubbing them with Webmock. There is a third option: simulating foreign APIs with Sinatra from within your test suite!
    Wednesday, May 4, 1:40 pm to 2:20 pm, Room 3501 EF
    • How Sprockets Works - Rafael Mendonça França

    Almost all applications have assets like CSS, JavaScript and others. That means the asset pipeline is an integral part of the Ruby on Rails framework. In this talk we'll show you how the asset pipeline works, and how you can take full advantage of the asset pipeline's features. Ever wondered how to convert an SVG to PNG automatically? Wanted to know what exactly happens to your CoffeeScript files? We'll explore that, and more.

    Wednesday, May 4, 3:40 pm to 4:20 pm, Room 3501 H

    • Testing Rails at Scale - Emil Stolarsky

    It's impossible to iterate quickly on a product without a reliable, responsive CI system. At a certain point, traditional CI providers don't cut it. Last summer, Shopify outgrew its CI solution and was plagued by 20 minute build times, flakiness, and waning trust from developers in CI statuses. Now our new CI builds Shopify in under 5 minutes, 700 times a day, spinning up 30,000 docker containers in the process. This talk will cover the architectural decisions we made and the hard lessons we learned so you can design a similar build system to solve your own needs.
    Wednesday, May 4, 4:30 pm to 5:10 pm, Room 3501 AB
    • Rails 5 Features You Haven't Heard About - Sean Griffin

    We've all heard about Action Cable, Turbolinks 5, and Rails::API. But Rails 5 was almost a thousand commits! They included dozens of minor features, many of which will be huge quality of life improvements even if you aren't using WebSockets or Turbolinks.

    This will be a deep look at several of the "minor" features of Rails 5. You won't just learn about the features, but you'll learn about why they were added, the reasoning behind them, and the difficulties of adding them from someone directly involved in many of them.

    Thursday, May 5, 11:40 am to 12:20 pm, Room 3501 DC

     

    Come say hey, and we're looking forward to chatting all things Rails! And if you're a dev interested in joining Shopify. Kayla Boyer will be at RailsConf: reach out to her through Twitter. To check out open roles or to learn more, head over to our Careers page.

    Jaime Woo

    Continue reading →

    Adventures in Production Rails Debugging

    At Shopify we frequently need to debug production Rails problems. Adding extra debugging code takes time to write and deploy, so we’ve learned how to use tools like gdb and rbtrace to quickly track down these issues. In this post, we’ll explain how to use gdb to retrieve a Ruby call stack, inspect environment variables, and debug a really odd warning message in production.

    We recently ran into an issue where we were seeing a large number of similar warning messages spamming our log files:

    
    /artifacts/ruby/2.1.0/gems/rack-1.6.4/lib/rack/utils.rb:92: warning: regexp match /.../n against to UTF-8 string
    

    This means we are trying to match an ASCII regular expression on a UTF-8 source string.

    Scott Francis

    Continue reading →

    Developer Onboarding at Shopify

    Hi there! We’re Kat and Omosola and we’re software developers at Shopify. We both started working at Shopify back in May, and we felt both excited and a little nervous before we got here. You never know exactly what to expect when you start at a new company and no matter what your previous experience is, there are always a lot of new skills you need to learn. Thankfully, Shopify has an awesome onboarding experience for its new developers, which is what we want to talk about today. The developer onboarding process at Shopify is constantly adapting to the needs...

    Kat Drobnjakovic

    Continue reading →

    Introducing Shipit

    After a year of internal use, we’re excited to open-source our deployment tool, Shipit. With dozens of teams pushing code multiple times a day to a variety of different targets, fast and easy deploys are key to developer productivity (and happiness) at Shopify. Along with key improvements to our infrastructure, Shipit plays a central role in making this happen. Motivation Coordinating frequent deployments in a large development team poses a number of challenges. You need to ensure that no one else is currently deploying, that the revision you are about to deploy has been successfully tested on CI, and even...

    Jean Boussier

    Continue reading →

    Secrets at Shopify - Introducing EJSON

    This is a continuation of our series describing our evolution of Shopify toward a Docker-powered, containerized data centre. Read the last post in the series here.

    One of the challenges along the road to containerization has been establishing a way to move application secrets like API keys, database passwords, and so on into the application in a secure way. This post explains our solution, and how you can use it with your own projects.

    Burke Libbey

    Continue reading →

    Announcing go-lua

    Today, we’re excited to release go-lua as an Open Source project. Go-lua is an implementation of the Lua programming language written purely in Go. We use go-lua as the core execution engine of our load generation tool. This post outlines its creation, provides examples, and describes some challenges encountered along the way.

    Francis Bogsanyi

    Continue reading →

    There's More to Ruby Debugging Than puts()

    "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan

    Debugging is always challenging, and as programmers we can easily spend a good chunk of every day just trying to figure out what is going on with our code. Where exactly has a method been overwritten or defined in the first place? What does the inheritance chain look like for this object? Which methods are available to call from this context?

    This article will take you through some under-utilized convenience methods in Ruby which will make answering these questions a little easier.

      Blake Mesdag

      Continue reading →

      Building Year in Review 2014 with SVG and Rails

      feature

      As we have for the past 3 years, Shopify released a Year in Review to highlight some of the exciting growth and change we’ve observed over the past year. Designers James and Veronica had ambitious ideas for this year’s review, including strong, bold typographic treatments and interactive data visualizations. We’ve gotten some great feedback on the final product, as well as some curious developers wondering how we pulled it off, so we’re going to review the development process for Year in Review and talk about some of the technologies we leveraged to make it all happen.

      Tessa Thornton

      Continue reading →