Node.js at Scale
Tom Hughes-Croucher (Joyent)
Thursday, August 4, 2011

ABSTRACT

When we talk about performance what do we mean? There are many metrics that matter in different scenarios but it’s difficult to measure them all. Tom Hughes-Croucher looks at what performance is achievable with Node today, which metrics matter and how to pick the ones that most matter to you. Most importantly he looks at why metrics don’t matter as much as you think and the critical decision making involved in picking a programming language, a framework, or even just the way you write code.

BIOGRAPHY

Tom Hughes-Croucher is the Chief Evangelist at Joyent, sponsors of the Node.js project. Tom mostly spends his days helping companies build really exciting projects with Node and seeing just how far it will scale. Tom is also the author of the O’Reilly book “Up and running with Node.js”. Tom has worked for many well known organizations including Yahoo, NASA and Tesco.

Kafka
Neha Narkhede (LinkedIn)
Wednesday, July 27, 2011

ABSTRACT

Kafka is a distributed publish-subscribe messaging system aimed at providing a scalable, high-throughput, low latency solution for log aggregation and activity stream processing for LinkedIn. Built on Apache Zookeeper in Scala, Kafka aims at providing a unified stream for both real-time and offline consumption. We provide a mechanism for parallel data load into Hadoop as well as the ability to partition real-time consumption over a cluster of machines. Kafka combines the benefits of traditional log aggregators and messaging systems and has been used successfully in production for 8 months. It provides API similar to that of a messaging system and allows applications to consume log events in real-time.

Written by the SNA team at LinkedIn, Kafka is open sourced under the Apache 2.0 License and is an Apache incubator project. In this presentation, we will highlight the core design principles for this system, and how this system fits into LinkedIn’s data ecosystem as well as some of the products and monitoring applications it supports in our usage.

BIOGRAPHY

Neha Narkhede is a Senior Software Engineer in the Search, Network and Analytics Team at LinkedIn, focusing on Distributed Systems. She is one of the initial contributors to Project Kafka. In the past she’s worked on search systems in large scale databases and has been an active contributor to several projects LinkedIn has open sourced, including Voldemort, Bobo and Zoie.

Game Dynamics
Rajat Paharia (Bunchball)
Monday, July 25th, 2011

ABSTRACT

Status, achievement, reward, competition, self-expression: by addressing these fundamental human needs and desires, designers can make experiences both compelling and satisfying. Game designers, in particular, have known for years how to incent and motivate players by addressing these needs through the use of mechanics like points, levels, leaderboards, virtual goods, challenges, and real-time feedback. We’ll cover the migration of these mechanics out of the gaming world and into the world at large, including destination sites, devices, productivity applications, corporate intranets, and LinkedIn.

BIOGRAPHY

Rajat Paharia is the founder and Chief Product Officer of Bunchball. He founded the company in 2005, and released the industry’s first gamification platform, Nitro, in 2007. Rajat works closely with Bunchball’s Fortune 500 clients to craft gamification strategies that engage customers, partners and employees, including companies like NBC, Warner Brothers, Comcast and Hewlett-Packard. Rajat’s skill set combines a unique expertise in technology, design and human behavior developed during a four-year career at design firm IDEO and through degrees in computer science and human-computer interaction.

State of HBase
Michael Stack (StumbleUpon)
Monday, July 18th, 2011

ABSTRACT

Attendees will learn about the current state of the HBase project. We’ll review what the community is contributing, some of the more interesting production installs, killer apps on HBase, the on-again, off-again HBase+HDFS love affair, and what the near-future promises. A familiarity with BigTable concepts and Hadoop is presumed.

BIOGRAPHY

Michael Stack works at StumbleUpon and is Chief Janitor on the HBase Project and a member of the Hadoop PMC.

I’m thrilled to announce that we have finally cut off a branch and are ready to do the 0.90 open source release for Project Voldemort. For folks still enchanted by the Potter mania, no, I am not talking about Harry’s nemesis. Project Voldemort is an open-source distributed key-value store being used here at LinkedIn and at various other companies. This is one of our biggest open-source releases and contains features that we have worked on and deployed at LinkedIn in the last one year.

Over the past year our user base internally has grown tremendously. About 3 years ago Voldemort was deployed as a small cluster to serve our first product Who viewed my profile. Over time we have grown to multiple clusters, some of which now span over LinkedIn’s two data-centers. At its crux a single cluster serves either one of the two types of applications – those dealing with read-only static data (computed offline in Hadoop after some number crunching) or read-write data. Now with around 200 stores overall in production we have been serving various critical user facing features. Some examples of applications that use us include People you may know, Jobs you may be interested in, Skills, LinkedIn Share Button, Referral Engine, Company Follow, LinkedIn Today and more. With new users ramping in nearly every other week our clusters have collectively started doing over a billion queries per day.

One of the most important upgrades we have done in production recently has been switching all our clients and servers from the legacy thread-per-socket blocking I/O approach to the new non-blocking implementation which multiplexes using just a fixed number of threads (usually set in proportion to the number of CPU cores on the machine). This is good from an operations perspective on the server because we no longer have to manually keep bumping up the maximum number of threads when new clients are added. From the client’s perspective we now won’t need to worry about thread pool exhaustion due to slow responses from slow servers.

While upgrading the client side logic, we also redesigned the routing layer to model it as a pipelined finite state machine (we call it the pipeline routed store). For example, a put() request on the client is now modeled as a series of following states:

1. Generate list of nodes to put to
2. Put sequentially till first success
3. Put in parallel to the rest of the nodes
4. Increment vector clock.

Designing every client request as states with transitions makes it easy to extend the pipeline and add new features (states) with minimum hassle. This enabled us to add support for a long-standing feature request – hinted-handoff, an additional consistency mechanism that handles cases of transient failures by using other live nodes as a backup system. Also in preparation for our new datacenter we were able to quickly plug in a new topology aware routing strategy that we call zone aware routing. For this strategy we cluster nodes into logical groups called zones (in our case a zone = data-center). Our routing strategy is then just a simple extension of Amazon Dynamo’s partitioning algorithm with special constraints on how we jump the ring.

The other important feature that we use at LinkedIn is the read-only stores pipeline. Our largest read-only stores cluster powers most of the recommendation features and fetches around 3 TB of data every day from Hadoop. Some of the initial work we did in this area was to make the fetch + swap pipeline more efficient by migrating off the old servlet based tool to a new administrative service based tool. This gave us better visibility into progress of data transfer (fetch phase) along with more control for swaps. Besides optimizing the data-transfer pipeline, we also spent some time iterating on the underlying storage format and data layout. The final storage format that we have come up with has a better memory footprint, supports iterators and has been tweaked for making rebalancing of read-only stores as simple as possible.

Besides working a lot on our Java client, we also updated some clients relying on Voldemort’s server-side routing. In particular the Python and Ruby clients have gone through some iterations. Many of our internal customers use these for quick prototyping of their ideas during our monthly Hackdays. In fact one of the by-products of this inDay has resulted in Voldemort having a nice GUI, more about which you can read here.

With some of the core pieces of the system in place now, our road-map is to pay more emphasis on the operation / management aspects. For example, we want to provide better tools to make ‘migration’ or ‘rebalancing’ of clusters as easy as pressing a button. Doing so would allow us to have larger clusters thereby decreasing operational overhead of maintaining many small clusters. We’re also slowly changing our old philosophy of adding new clusters for every new application and instead plan to add better support for multi-tenancy.

Fork / Download Voldemort, read all about updating to the new version, find an interesting new sub-project, submit a patch and come join the Death Eaters.

Stratified B-Tree and Versioned Dictionaries
Andy Twigg (Acunu)
Monday, June 20, 2011

ABSTRACT

A classic versioned data structure in storage and computer science is the copy-on-write (CoW) B-tree — it underlies many of today’s file systems and databases, including WAFL, ZFS, Btrfs and more. Unfortunately, it doesn’t inherit the B-tree’s optimality properties; it has poor space utilization, cannot offer fast updates, and relies on random IO to scale. Yet, nothing better has been developed since. We describe the `stratified B-tree’, which beats all known semi-external memory versioned B-trees, including the CoW B-tree. In particular, it is the first versioned dictionary to achieve optimal tradeoffs between space, query and update performance.

BIOGRAPHY

Andy Twigg is Chief Scientist at Acunu, a UK startup building a ground-up platform for Big Data, initially by rewriting the storage stack. His PhD from Cambridge on compact routing was nominated for the British Computer Society Best Dissertation Award, and he is a Junior Research Fellow at St John’s College, Oxford and member of the CS Department.