Apache Mesos

+ Apache YARN

= Myriad

Adam Bordelon

Myriad Project Lead


Apache Mesos Committer

Mesos from 10,000 feet

Open Source Apache project

Cluster Resource Manager

Scalable to 10,000s of nodes

Fault-tolerant, no SPOF

Multi-tenancy, Resource Isolation

Improved resource utilization

Mesos is more than

Yet Another Resource Negotiator

Long-running services; real-time jobs

Native Docker; cgroups for years; Isolate cpu/mem/disk/net/other

Distributed systems SDK;
~200 loc for a new app

Core written in C++ for performance, Apps in any language

Mesosphere DCOS

History of Myriad

Mohit Soni from eBay builds prototype, presents at MesosCon

Santosh Marella from MapR
attends Mesosphere HackWeek,
builds independent prototype

Adam introduces Mohit and Santosh, both prototypes merged together

Myriad presented to Apache Incubator, accepting new contributors

Myriad improves Mesos

Tighter integration with Hadoop frameworks like HBase, Hive, Pig

Borrow resources from Hadoop when traffic spikes for tier-1 services

Backfill unused resource capacity with best-effort Hadoop jobs

No Mesos code changes necessary

Myriad improves Hadoop

Elastic scaling

Fault-tolerant: Maintain NM capacity

Share resources with other workloads, improve resource utilization

Multiple isolated Hadoop clusters sharing node resources and DFS

No YARN/Hadoop code changes

Learn More!