Spark Cluster Manager
A cluster manager manages a cluster of computers. To be more specific, it manages resources such as CPU, memory, storage, ports, and other resources available on a cluster of nodes. It pools together the resources available on each cluster node and enables different applications to share these resources. Thus, it turns a cluster of commodity computers into a virtual super-computer that can be shared by multiple applications.
Distributed computing frameworks use either an embedded cluster manager or an external cluster manager. For example, prior to version 2.0, Hadoop MapReduce used an embedded cluster manager. In Hadoop 2.0, the cluster manager was separated from the compute engine.
Spark comes prepackaged with a cluster manager, but it can also be used with a few other open source cluster managers. Spark supports three cluster managers: Apache Mesos, Hadoop YARN, Kubernetes, and the Standalone cluster manager.
Statlearner
Statlearner