Newsletter

Holen Sie sich die neuesten Updates von Hortonworks per E-Mail

Einmal monatlich erhalten Sie die neuesten Erkenntnisse, Trends und Analysen sowie Fachwissen zu Big Data.

AVAILABLE NEWSLETTERS:

Sign up for the Developers Newsletter

Einmal monatlich erhalten Sie die neuesten Erkenntnisse, Trends und Analysen sowie Fachwissen zu Big Data.

cta

Erste Schritte

Cloud

Sind Sie bereit?

Sandbox herunterladen

Wie können wir Ihnen helfen?

* Ich habe verstanden, dass ich mich jederzeit abmelden kann. Außerdem akzeptiere ich die weiteren Angaben in der Datenschutzrichtlinie von Hortonworks.
SchließenSchaltfläche „Schließen“
October 10, 2018
Vorige FolieNächste Folie

Apache Hadoop Ozone – Object Store Overview

1. Introduction

This article is second in the series about Ozone – a distributed key-value store that can efficiently manage small and large files alike.

An earlier article introduced the Ozone design philosophy. The current article is for users who want to try Ozone. We discuss an overview of Ozone concepts, describe how to bring up an Ozone cluster and use the Ozone command shell.

2. Ozone Concepts

Ozone is made up of volumes, buckets and Keys.

  1. Volumes – Volumes are similar to accounts. Volumes can be created or deleted only by administrators. An administrator will typically create a volume for an organization or team.
  2. Buckets – A volume can contain zero or more buckets. Ozone buckets are similar to Amazon S3 buckets.
  3. Keys – Keys are unique within a given bucket and are similar to S3 Objects. Key names can be any string. Values represent the data that you store inside these keys, currently Ozone enforces no upper limit on the key size.

3. Installing Ozone

3.1. Get the Binaries

  1. Download Binary release – get the latest Ozone release from the Ozone website at https://hadoop.apache.org/ozone/.
  2. Build from source code – The instructions are at https://cwiki.apache.org/confluence/display/HADOOP/Building+Ozone

3.2. Deploy a cluster

There are multiple ways to setup an Ozone cluster.

  1. Development cluster with Docker – Instructions at https://cwiki.apache.org/confluence/display/HADOOP/Development+cluster+with+docker
  2. Single-node development clusterhttps://cwiki.apache.org/confluence/display/HADOOP/Single-node+dev+cluster
  3. Multi-node Ozone clusterhttps://cwiki.apache.org/confluence/display/HADOOP/Multi-node+Ozone+Cluster

4. Ozone command-line shell

4.1. Working with Volumes, Buckets, Keys

The Alpha release includes the following interfaces:

  1. REST & RPC API
  2. Command-line shell to perform object store operations.
  3. OzoneFS – A Hadoop Compatible Filesystem interface.

Let’s look at a few examples of using the command-line shell to perform object store operations.

4.1.1. Create a Volume

The following command creates a new volume named videos, owned by the user alice. Creating new volumes is an administrative operation.

4.1.2. Create a Bucket

The following command creates a new bucket named vacation in videos.

4.1.3. Upload a Key

This is the equivalent of ‘putting’ a file into the object store. The following command will upload the  local file video-001.mp4 to the ozone bucket with the name machu-picchu.mp4.

4.1.4. List Keys in a Bucket

4.1.5. List Buckets in a Volume

4.1.6. List Volumes

5. Further Reading

  1. Ozone web site – https://hadoop.apache.org/ozone/
  2. Ozone Command Shell Reference – https://github.com/apache/hadoop/blob/trunk/hadoop-ozone/docs/content/CommandShell.md
  3. HDDS wiki page – https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Distributed+Data+Store+and+Applications

Antwort verfassen

Ihre E-Mail-Adresse wird nicht veröffentlicht. Pflichtfelder sind mit * gekennzeichnet