Engineering Blog

Update on the API V3 Release and V1 & V2 Retirement

We strive to provide athletes with the most motivating and fun environment for training and sharing activities. To keep this focus, we’ve had to make a hard choice this year between investing in our features and making the API broadly available. (Please see our Strava API Update blog post on this subject.)

This means that as we roll out our latest version of the API, version 3, we will have to limit the amount of developers who have access to it, as well as discontinue V1 and V2 APIs on June 30, 2013. Libraries, sites, and applications that use V1 and V2 endpoints will cease to work as the underlying API endpoints will no longer be available. By delaying the retirement of V1 and V2 of the API until June 30, we hope to give you sufficient time for you to make alternative arrangements should you be impacted.

Limited Access to V3 We will be accepting a small handful of developers into the V3 Early Access Program who can help us blaze new trails for the Strava community. If you meet the criteria below, complete this survey and tell us what you’d like to do with the API.

What we’re looking for:

• Novel uses of Strava data that benefit athletes
• New ways to connect and grow the Strava community

What we can’t allow:

• Applications that encourage competition
• Replicating Strava functionality

We understand this change impacts many of you. If you have further questions, or would like to get in touch with us, please refer to the Strava API Update blog post.

Strava API Update

Strava V3 API Rollout Process

Please see the update to this post as of 28 May, 2013.

Strava is committed to being the de-facto platform for endurance athletic applications. To provide the best service, Strava is making investments in engineering, support, community relations, documentation, and product management to support our API. Therefore, we will gradually release the Strava Version 3 API with a very few select developers, then make it available more broadly later this year. Unfortunately, this gradual approach means that many of you who have been waiting for access to V3, as well as those using V1 and V2, will have to wait until later this year to gain access.

Retiring V1 and V2 APIs by June 1, 2013

As part of the V3 API roll-out process, we will be discontinuing V1 and V2 APIs on May 31. Libraries, sites, and applications that use V1 and V2 endpoints will cease to work as the underlying API endpoints will no longer be available.

What’s Coming in Version 3

Version 3 of the Strava API features a simplified and consolidated representation of Strava resources. The top level resources are Athletes, Activities, Segments, Uploads, and Clubs.

The new API implements OAuth2 allowing both developers and athletes to flexibly and securely expose data in a variety of applications and services. The OAuth2 flow allows applications and services to have new athletes sign up for Strava or log into Strava without having to get access to or store athlete credentials. In addition, JavaScript developers will appreciate the addition of support for JSONP. The API is rate metered to assure that all applications receive fair and appropriate access to data.

Apply to Join the V3 API Early Access Program

To ensure we’re providing the very best development platform, we will be working with a select group of developers before the general release of the Strava V3 API. Participating developers will contribute to the program and benefit at the same time. We are looking for a small group of developers to join this Early Access Program who meet the following criteria:

• Encourage safe and fun competition. (Learn more about how you can Stand with Us.)
• Provide a unique and free service or application that benefits Strava athletes
• Self-sufficient and resourceful developer
• Willing to share examples and constructive feedback on the API to benefit other, future developers
• Respect the privacy of your fellow athletes

If you feel you are an ideal candidate to participate in the V3 Early Access Program, please complete this survey.

We’re Hiring

We can’t guarantee V3 access to everyone, but we are confident that we’ll be able to provide developers with an organized and consolidated API experience later this year as a result of this gradual roll-out process. In the meantime, there are more ways you can contribute to Strava, including joining our team! Check out our Careers page to learn more.

Thanks,

Craig

Evolution of an Easter Egg

Editor’s Note: Mateo leads our Android development team. Mateo is also our resident comedian. Naturally then, Mateo enjoys himself a good Easter Egg. Below are details of a few he’s laid.

Version 2.0

If you have a GPS lock and you long-click on the Record button, two things happen: the phone vibrates S-T-R-A-V-A in morse code, and a toast with some random encouragement displays.

Verdict: Way too abstract, how would anybody stumble across this? I suspect only those who read the source code. Also, I’m not sure anyone can detect vibrating morse code. I did code this on my first day at Strava…

Version 2.1

I made some minor tweaks: toast layout now semi-transparent, and I added some activity specific text (since we released a separate Run app in 2.1).

Verdict: No-op

Version 2.2

There was an limited version of social hidden in this release. If you name an activity “Talking Rain” you’re presented with the option of enabling Social Preview (Talking Rain is an inside Strava joke – we used to drink this stuff like water, which it is: more expensive Costco water).

Verdict: Too close to our actual product road map – not sure if PM team would be pleased. I don’t think anyone discovered this – who names a ride Talking Rain?

Version 3.0

I removed both of the above Easter Eggs to make room for something new. Unfortunately with 3.0 release and the holidays, all of my idle time was consumed.

Verdict: All work, no fun.

Version 3.1

If you’re premium and you name an activity “Talking Rain” you may quick Kudo any activity by long-clicking on it from the Feed. This will likely become a real feature once we make some tweaks on the back-end.

Verdict: A small, non-offensive egg.

iOS App Features and Shortcuts

In addition to major features like advanced segment effort analysis, club feeds, and notifications, the Strava iOS team recently shipped a handful of smaller features in version 3.1.1. Download it for free here. These shortcuts are geared toward making Strava for iPhone easier and faster to use.

• Reveal full-size profile image
• Long-tap on an activity in the Feed for Kudo/Comment shortcut
• Kudo bomb
• Additional sharing options from the Activity Detail screen
• Landscape Segment Effort Analysis

Full-size Profile Images

While viewing an athlete’s profile, tap the athlete’s image to reveal a full screen view of their profile picture.

Kudo and Comment Shortcut

Tap an activity in the Feed to reveal an action sheet with shortcuts to kudo or comment on that activity.

Kudo Bomb

Shake your iOS device while viewing the “Also on this Activity” screen to reveal an action sheet with options to give kudos to everyone who was on that ride or run.

Additional Sharing Options

Twitter and Facebook aren’t the only ways you can share an activity. Tap the action button on the Activity Detail screen to reveal an action sheet with options to share via messages, email, or to copy the URL to the activity on Strava.

Landscape Segment Effort Analysis

Rotate your phone to landscape while looking at the new segment effort analysis screen to get a detailed view of the active graph.

Technical Discussion

This wouldn’t be a engineering blog post without at least some technical explanation. None of the features listed above require unusual or groundbreaking techniques, however they all reveal different ways that you can interact with the phone and provide your app’s users with additional affordance.

The Kudo Bomb feature is a cute illustration of the concept that different types of gestures can activate different features:

Implementing shake recognition on an iOS device is fairly straightforward. It so happens that UIViewController is a subclass of UIResponder. To handle custom touch events you’d override the -touches... methods and provide your own implementation for when the user touches the screen. To respond to motion events you override one or more the -motion... events, in our case -(void)motionEnded:withEvent:. We’re just interested in the shake motion which is UIEventSubtypeMotionShake (as of iOS 6, it’s the only motion subtype). So to do something when the user shakes the device while looking at your view controller you’d do something like this in your view controller:

Note that your view controller must be able to become the first responder, hence the overridden canBecomeFirstResponder method.

To test shake motions in the iOS simulator you can choose the Hardware > Shake Gesture menu item, or type ^⌘Z.

A fully functional example project is available on GitHub.

Intent Filters in the 3.0 Android App

URI Intent Filters – they sound complicated (and nerdy), but the concept is actually simple: when certain types of links are encountered on your device (e.g. on a web page, in your email, in another app, etc.), you have the option of allowing an app to handle them instead of the browser. In the case of Strava’s Android App (both Cycling & Run), starting with version 3.0 a link to Strava (such as a ride or run URL) can be opened directly by the app. Below are some examples:

Links to Rides & Runs on Facebook, Twitter, Path, etc..

1. Click a link to Strava	2. Select the Strava app	3. The Strava app handles it!

Such links can appear anywhere, and thanks to the power of Android, the Strava app can handle them all. More examples:

A web page	SMS message	TweetDeck

Email Notifications

This is my favorite use case: with Intent Filters you essentially get Strava push notifications, as any link within a Strava email can be intercepted by the Strava app. The Strava app currently handles Activity, Segment, Profile and  Follower related links. Any other links will just send you to the browser (as normal).

1. Click a link to Strava	2. Select the Strava app	3. The Strava app handles it!

That’s it! If you can’t wait for the official 3.0 release get the Early Access release and play with the link clicking today!

NOTE: if you’re new to email notifications in Strava, they can be managed from within the Strava app (Settings->Notifications and Followers), or on the website.

Attention Android Users: 3.0 Early Access is Available

1. Want to get your hands on the next version of the App now?

2. Can’t wait any longer to kudo & comment on the go?

3. Will you possibly be stuck in line somewhere and really want to pass time by seeing where your friends have ridden and run lately?

4. Most importantly, are you willing to deal with a few rough edges?

If you answered “YES!” to all the above, click this link for all the details.

KOMs, Powered By Redis

One of the first scaling challenges we encountered at Strava was maintaining real-time segment leaderboards. There are a number of questions that we ask of a segment leaderboard, such as:

• Who’s in the top 10?
• Who’s ranked 150 through 175?
• What is my rank?
• Who is ranked near me?

Ideally all of these questions can be answered efficiently, e.g. better than O(N), where N is the size of the segment leaderboard (which is also the number of unique athletes that have efforts on the segment). As the total number of efforts across all segments tallied over 100 million, we began to see performance issues:

• Some leaderboards were becoming difficult to cache. On a busy day the contents of popular leaderboards were shifting frequently with new riders and new PRs, forcing us to rebuild them often. Well-travelled segments were also often the most viewed. (NOTE:  PR = personal record, or an athlete’s best time on a segment. Only PRs appear on a leaderboard.)
• Our queries were slowing down as leaderboards grew. Ranked queries in SQL are notoriously convoluted. These queries began bubbling up in our slow query logs for popular segments, which in some cases now had over 50k efforts and over 3k unique athletes.
• Our database cache was thrashing. We couldn’t keep in memory all the indices used by the leaderboard queries. As our indices hit I/O things really started to get bad.

To give you a sense of things, a table which would hold all efforts on all segments across all athletes and activities, including leaderboard worthy efforts (PRs), would include these columns:

segment_efforts(id, segment_id, activity_id, athlete_id, elapsed_time)

…and have a compound index on [segment_id, athlete_id, elapsed_time]. There are a number of different ways to write a query to generate a leaderboard for a particular segment. None of them are awesome – the way below is as good as any:

SELECT MIN(e1.id) id, e1.athlete_id, e1.elapsed_time
FROM segment_efforts e1
INNER JOIN (
  SELECT athlete_id, MIN(elapsed_time) fastest_time
  FROM segment_efforts
  WHERE segment_id = ?
  GROUP BY athlete_id
) e2 ON e1.athlete_id = e2.athlete_id AND
  e1.elapsed_time = e2.fastest_time
WHERE e1.segment_id = ?
GROUP BY e1.athlete_id, e1.elapsed_time
ORDER BY e1.elapsed_time, e1.id

Lovely, right? As a side note: you’ve officially been using Rails too long if seeing pluralized table names (e.g. segment_efforts vs. segment_effort) no longer registers some disgust.

To support our growth and improve performance, we needed to denormalize the leaderboard data (or find a bigger box or shard, but I’m going to skip those discussions). The first obvious place to turn was back to our database:

• We could add a sparsely populated pr column to our table, along with an accompanying index ([segment_id, pr, elapsed_time]). We’d then update this column as necessary when new efforts are inserted. This would involve updating the table (and index) much more frequently than before. The table was already our hottest table for writes, so this wasn’t attractive.
• Alternatively, we could manage a separate leaderboard table, which would only store PRs, with a unique index on [segment_id, athlete_id]. Aside from creating another write heavy table, there were some queries like ”What’s my rank?” which would remain inefficient. Keeping a Rank column up-to-date was a non-starter, as there would be way too many updates.

These solutions came up short. Here’s where Redis fit in. We were already using Redis for our background jobs, and its Sorted Set data structure mapped very nicely to our problem. Each segment would have its own sorted set, with the AthleteID as the member and the athlete’s fastest elapsed time as the score (see zadd). All of our queries (“Who’s in the top 10?”, etc.) should have better than linear run-times.

A sorted set can only hold two values per entry (athlete_id, elapsed_time), so we still need a place to store the ID of the actual effort to which these values map. We could go back to the database to look these up (recall our index on [segment_id, athlete_id, elapsed_time]), but instead we decided to store this information in a complimentary data structure in Redis, using a hash for each segment, keyed by AthleteID. For example, retrieving the top 10 leaderboard for a segment would look something like:

athlete_ids = zrange(<sorted_set_key>, 0, 9)
effort_ids = hmget(<efforts_hash_key>, *athlete_ids)

Using Redis did introduce some new challenges. Keeping the efforts hash up-to-date and in sync with the leaderboard sorted set required some extra book keeping. In theory you’d want these updated in the same transaction, but Redis doesn’t quite have the equivalent database concept. Instead, we could do these updates holding some sort of exclusive lock (per segment), or we could use Redis watches and retry on failure. We chose the later.

Finally, we have the issue of leaderboard data now living in two separate stores: our database and Redis. In theory we’d like to wrap related updates to both of these in a coordinated transaction, but we’re not launching missiles into space at Strava… So, we must assume there will be drift. Our database is still the “system of record”, so as we discover dirty data in Redis, we lazily correct it. We also make the Redis updates dependent on the database transaction committing, so at least we can reduce phantom data in Redis (entries for efforts that were never actually committed to the database).

When storing data in Redis we found it was important to adopt sane key namespaces. If you just throw non-expiring keys into it willy-nilly you’ll have a hard to cleaning them up. Everything must fit into memory with Redis, so you’ll want to be judicious about how and what you store in it. There are some wildcard queries you can use to find keys, but you wouldn’t want to troll your entire key space with them. In general, a nice model for us has been to think of our data-sets in Redis as indices.

Did I mention yet that we love Redis? From a systems POV it has been awesome: replication is easy, memory footprint is small (we do optimize for this), and the CPU is largely bored. Today all of our leaderboards fit on a single replicated Redis box using under 16GB of memory. As our data-set grows we feel comfortable sharding it as our access patterns are already so straightforward (thankfully Redis forces you to keep things simple).

Since our initial transition to Redis we’ve added filtered leaderboards (by weight class and age group) and date-range windows (this year, this month, today). These features introduced some additional layers of complexity, but in general we see this design scaling well in the future.

Your Suffering + Our Code = Strava

Whether it’s building an enterprise system to streamline an office job, or developing a dongle to improve the payment process of buying a burrito, at some level all software is rooted in the offline world. Nowhere is that connection more real than with products we build at Strava. It’s you, the athlete, sweating and suffering on your local roads and trails, who provides us with our raw material: your performance data. Engineers at Strava are honored to have the job of aggregating, analyzing, and ultimately presenting your data back to you in a way that’s intuitive and motivating.

It’s these stories from the nexus of engineering and endurance sports – our little corner of the software development world – that we want to share here. We’ll cover a range of topics: technical challenges we’ve encountered, aspects of our technology stack, developer tools we employ, updates on our public API, and more. Stay tuned.