v1.

2 | August 2024 | TN-2106

SOLUTIONS DOCUMENT

Nutanix Objects
Legal
© 2024 Nutanix, Inc. All rights reserved. Nutanix, the Enterprise Cloud Platform, the
Nutanix logo and the other Nutanix products, features, and/or programs mentioned
herein are registered trademarks or trademarks of Nutanix, Inc. in the United States
and other countries. All other brand and product names mentioned herein are for
identification purposes only and are the property of their respective holder(s), and
Nutanix may not be associated with, or sponsored or endorsed by such holder(s). This
document is provided for informational purposes only and is presented "as is" with no
warranties of any kind, whether implied, statutory or otherwise.
Nutanix, Inc.
1740 Technology Drive, Suite 150
San Jose, CA 95110
 Nutanix Objects

Contents

1. Executive Summary................................................................................. 5
Object Storage Overview.................................................................................................................... 6
Object Storage Compared to Traditional Storage Systems................................................................7

2. Nutanix Objects Use Cases.................................................................... 9

Big Data...............................................................................................................................................9
Cloud Native......................................................................................................................................10
Backup and Archive.......................................................................................................................... 10

3. Nutanix Objects Architecture............................................................... 11

Nutanix Objects Load Balancer VMs................................................................................................12
Nutanix Objects Worker VMs............................................................................................................13
Nutanix Objects Multitenancy........................................................................................................... 18
Nutanix Objects Networking..............................................................................................................18
Nutanix Objects Client Access..........................................................................................................20
Nutanix Objects Buckets...................................................................................................................21
Nutanix Objects High Availability...................................................................................................... 23
Nutanix Objects Management and Monitoring................................................................................. 25
Objects Browser................................................................................................................................ 26
Object Store Features.......................................................................................................................27
Nutanix Objects Upgrades................................................................................................................ 30
Nutanix Objects Security...................................................................................................................31

4. Nutanix Objects Data Protection and Recovery................................. 34

Streaming Replication....................................................................................................................... 34

5. Nutanix Objects Global Namespace.................................................... 37

Federation Architecture..................................................................................................................... 37
Viewing Utilization and Scaling.........................................................................................................38
Routing Client Requests................................................................................................................... 39
Managing Namespaces.....................................................................................................................40
Fault Tolerance and Migration.......................................................................................................... 43
Client Access Locality....................................................................................................................... 44
Multicluster vs. Federation................................................................................................................ 45
6. Conclusion.............................................................................................. 46

About Nutanix.............................................................................................47
List of Figures.............................................................................................................................................48
 Nutanix Objects

1. Executive Summary
Nutanix Objects, which is part of the Nutanix Unified Storage (NUS) suite, is a software-
defined, scale-out object storage solution that provides a massively scalable repository
for unstructured data such as backup, archive, and application data. Strongly S3
compliant and capable of excellent performance, Nutanix Objects also serves as a data
lake store for big data use cases. It's a highly versatile object storage solution that is fully
integrated into the Nutanix Cloud Platform.
You can deploy Nutanix Objects on an existing or standalone cluster. Unlike standalone
object storage appliances, Nutanix Objects consolidates VM and object storage
(and other storage services such as Nutanix Files), eliminating infrastructure silos.
Administrators can manage Nutanix Objects with Prism, just like VMs and other Nutanix
storage services, which unifies and simplifies management. Nutanix Objects is flexible
and feature-rich, with support for popular features such as versioning, write once,
read many (WORM), life cycle management, and cloud tiering. Nutanix Objects also
supports native data replication for offsite protection of object data and provides a unified
global namespace across geo-distributed object stores for simplified data access in
multilocational data environments. You can obtain analytics-driven insights into Nutanix
Objects environments through integration with Nutanix Data Lens.
Nutanix Objects can run on a dedicated cluster or a cluster running user VMs, and
you can use it with Nutanix AHV and ESXi. Nutanix Objects includes native high
availability and uses Nutanix storage for intracluster data resilience. Nutanix storage also
provides data reduction through techniques such as inline erasure coding (EC-X) and
compression.
In this document, we cover the following topics:
• Overview of the Nutanix architecture with Objects
• Bucket policies
• High availability
• Self-service with Objects Browser

© 2024 Nutanix, Inc. All rights reserved | 5

 Nutanix Objects

• Security
• Data protection
• Global namespaces
Table: Document Version History
Version Number Published Notes
1.0 November 2023 Original publication.
1.1 June 2024 Updated for Nutanix Objects
version 5.0.
1.2 August 2024 Updated the Nutanix Objects
Architecture and Nutanix
Objects Data Protection and
Recovery sections.

Object Storage Overview

As technology evolves, the data collected by smart devices, sensors, healthcare
applications, cameras, and other devices increases exponentially. The methods we use
to manage data must also evolve to accommodate the drastic increase in scale and
the rates at which data needs to be ingested, retrieved, and processed. With continued
advancements to the Internet of Things (IoT), object storage has gained wider adoption
as one of the best ways to rapidly store and retrieve continuously growing data sets that
are often machine-generated and petabytes in scale.
The object storage architectural model handles data as objects. This data can be any
type—sensor data, photos, logs, videos, social media entries, and more—and it's
organized in a flat namespace with no directory structures or traditional blocks. This
storage architecture provides greater accessibility and velocity when handling data
because it removes the complexities and overhead introduced by multiple formatting
layers. Each object comprises the data (payload), its metadata, and a unique key used to
identify the object. The metadata used in object storage is rich and extensive, containing
granular information about the object it describes. The flexibility of this metadata provides
greater control when you need to manage or manipulate an object.

© 2024 Nutanix, Inc. All rights reserved | 6

 Nutanix Objects

In Nutanix Objects, everything is object-centric. Objects are stored inside buckets, which
are logical storage containers that can hold billions of objects. You can access these
objects using simple HTTP or HTTPS REST API calls, such as PUT, GET, delete, and
so on. Nutanix Objects is compatible with Amazon's Simple Storage Service (S3) API to
give application developers a familiar interface that requires little to no change to their
existing code to move to Nutanix Objects. In addition, Objects enables Nutanix platform
users to store and manage unstructured data on the proven, highly scalable Nutanix
architecture. When compared to cloud-hosted solutions, this on-premises model offers
more consistent control over the costs associated with storing objects, as well as greater
transparency around the location of those objects.

Object Storage Compared to Traditional Storage Systems

Object data is considered unstructured because stored files are grouped into flat buckets
and can be of any type. In object storage, the handle for an object is a URL, an object
ID, or both. An object store locates and retrieves an object using the object's unique ID
(its key). Because Nutanix Objects uses S3-compatible APIs, any application that uses
S3 APIs can access the data, assuming that the application has the necessary access
permissions.
Typically, object storage solutions are eventually consistent, which means that stale
copies of data might exist until all copies are refreshed over time. Eventual consistency
provides the latitude necessary for scaling object storage across geographic regions.
However, within the same datacenter, Nutanix Objects delivers strong consistency, so the
latest copy of data fetched is the most current.
The following table captures the main differences between object, file, and block storage,
and it compares Nutanix Objects and generic object storage solutions.
Table: Differences Between Object, File, and Block Storage
Category Block File Object (Nutanix Object
Objects) (Generic)
Consistency Strong Strong Strong Eventual
Interface iSCSI SMB HTTP or HTTPS HTTP or HTTPS
(S3 REST API) (S3 REST API)

© 2024 Nutanix, Inc. All rights reserved | 7

 Nutanix Objects

Category Block File Object (Nutanix Object

Objects) (Generic)
Protocol Fibre Channel NFS HTTP or HTTPS HTTP or HTTPS
Protocol (S3 REST API) (S3 REST API)
Data format Structured Unstructured Unstructured (flat Unstructured (flat
(tables) (hierarchical namespace) namespace)
directories)
Logical container Volume Share and folders Bucket Bucket

© 2024 Nutanix, Inc. All rights reserved | 8

 Nutanix Objects

2. Nutanix Objects Use Cases

Nutanix Objects is a versatile object storage solution suitable for a wide range of
workloads. This section explains some of the primary use cases.

Big Data
Big data is an umbrella term covering many solution types that all involve analyzing large
volumes of data. Big data applications, even those that are coded to use the Hadoop
Distributed File System (HDFS), can use Nutanix Objects as a primary storage target
because Hadoop's S3A client translates HDFS calls to S3 API calls. For example,
Nutanix Objects can serve as primary storage for the following big data workloads:
• Hadoop MapReduce
• Spark
• Presto/Trino
• Vertica
• Dremio
• Snowflake
Most big data query engines provide ways to reduce the amount of data read during a
query, and you can use most of these methods with Nutanix Objects, including S3 Select
Pushdown, which Objects currently supports with the CSV file format.
Another role Nutanix Objects fills in big data environments is that of a secondary storage
tier (warm, cold, or frozen). Examples of applications that can use Nutanix Objects as a
secondary storage tier include the following:
• Splunk (SmartStore)
• Confluent Kafka (storage tier)
• Elastic (searchable snapshots)

© 2024 Nutanix, Inc. All rights reserved | 9

 Nutanix Objects

Cloud Native
With its strong support for the S3 API, Nutanix Objects works seamlessly with cloud-
native applications as they move from cloud to on-premises. Nutanix also provides a
Container Object Storage Interface (COSI) driver that allows Kubernetes applications to
self-provision and share buckets without an administrator's intervention. For automated
resource management, you can set hard quotas to ensure that the number of buckets
created by a given application doesn't exceed allowances.

Backup and Archive

Nutanix Objects works well with any backup application that can write backups to or
archive data to an S3-compatible endpoint. Nutanix Objects is also available as part of a
turnkey backup solution called Nutanix Mine Integrated Backup with the following backup
partners:
• Commvault
• HYCU
• Veeam
• Arcserve
Other backup applications that can back up directly to Nutanix Objects include Veritas
NetBackup and IBM Spectrum Protect. Nutanix Objects is validated for compliance
WORM support, which is often a key feature in these environments.

© 2024 Nutanix, Inc. All rights reserved | 10

 Nutanix Objects

3. Nutanix Objects Architecture

Nutanix Objects runs as a set of containerized, Kubernetes-orchestrated microservices
inside special VMs, known as workers, that you can scale out across the underlying
physical nodes. These VMs run on top of the Nutanix Cloud Platform, allowing Objects to
take advantage of capabilities such as data resilience, encryption, storage efficiency, and
scale-out I/O.

Figure 1: Special Nutanix VMs and Their Roles in Nutanix Objects

© 2024 Nutanix, Inc. All rights reserved | 11

 Nutanix Objects

ESXi deployments must meet several conditions explained in the ESXi Configuration
section of the Objects User Guide. Although you manage Nutanix Objects from Prism
Central, APIs also exist, allowing you to programmatically perform many operations.

Nutanix Objects Load Balancer VMs

Load balancer VMs are local traffic managers. A number of these lightweight VMs are
automatically deployed for performance and redundancy when you create an object
store. The load balancer VMs are connected to the public network, and you must add
each to the Domain Name System (DNS) under the object store's name so that client
requests submitted to the object store are directed to them.
Note: Nutanix Objects doesn't automatically create DNS entries for the object store, so add the load
balancer IP addresses to the DNS under the object store's fully qualified domain name (FQDN).

Nutanix Objects uses DNS techniques such as round-robin allocation to spread the load.
The load balancers evenly distribute these requests to the worker VMs, where they are
actioned.
Note: To achieve good distribution across the load balancers, we recommend using a TTL of under 10
minutes for the Objects DNS records.

The number of load balancers in an object store is automatically determined based on

the number of workers selected during deployment. The quantity is one less than the
number of workers, up to a maximum of four load balancers. If you deploy one worker
VM, Nutanix Objects deploys one load balancer. You can't scale out load balancers, so if
throughput requirements increase over time, assign additional vCPU and memory to the
load balancer VMs to allow them to handle the increased load. For more information, see
KB 12213. If you need to scale up the load balancers to handle more client connections,
assign them more vCPU and memory per the KB article, then engage Nutanix Support to
make necessary additional changes at the microservices level.
Table: Default Compute Resources Used by Workers and Load Balancers
VM Type vCPU Memory (GB)
Worker 10 32
Load balancer 2 4

© 2024 Nutanix, Inc. All rights reserved | 12

 Nutanix Objects

Engage Nutanix Support to increase worker and load balancer vCPU resources
from default to maximum. We recommend the maximum configurations for all-flash
deployments.
Table: Maximum Compute Resources Used by Workers and Load Balancers
VM Type vCPU Memory (GB)
Worker 16 32
Load balancer 4 8

Nutanix Objects Worker VMs

The Nutanix cluster deploys worker VMs to house the various components that make up
the Objects service. These components run as containerized services in a Kubernetes
cluster on the Nutanix Microservices Platform (MSP). Nutanix Objects follows a modular,
scale-out design where each component focuses on a single core function so that we
can develop and upgrade each component independently, helping reduce the time to
release for new features.
Nutanix Objects has the following core services:
Front-end adapter
S3 API–aware adapter that receives client requests and translates them for the
back-end system; part of the object controller service
Object controller service
Oversees the data management layer that interfaces with AOS and coordinates
with the metadata service; essentially the I/O engine that runs on every worker in
the object store
Note: The object controller service supports individual objects as large as 5 GB for a standard upload or 16
TB for a multipart object upload.

Metadata service
Manages the metadata and serves as a distributed key-value store that also
handles partitioning and region mapping; runs on every worker in the object store
except one

© 2024 Nutanix, Inc. All rights reserved | 13

 Nutanix Objects

Atlas (life cycle management service)

Controls life cycle, audits, and background maintenance activities; runs on every
worker in the object store
Identity and access management (IAM) service
Handles user authentication for accessing buckets; only runs on one worker in the
object store

Figure 2: Core Nutanix Objects Microservices

GET and PUT Request Process

When you make a client GET or PUT request, the following process occurs:
1. The load balancer receives the request and forwards it to a worker.
2. The worker checks the user account against the IAM service.
3. The worker approves the request and forwards it (or rejects it and drops it).
4. The front-end adapter translates the incoming S3 API call.
5. The object controller liaises with the metadata service to locate the data (GET) or
determine where to place the new data (PUT).
6. For PUT requests, the object controller writes the data, and the metadata service
updates the metadata and returns a success code to the client. For GET requests, the
object controller reads the data and sends it to the client through the load balancer.

© 2024 Nutanix, Inc. All rights reserved | 14

 Nutanix Objects

Figure 3: Anatomy of a PUT Request

© 2024 Nutanix, Inc. All rights reserved | 15

 Nutanix Objects

Figure 4: Anatomy of a GET Request

Every object controller can connect directly to any CVM in the AOS cluster or to nonlocal
CVMs if you configure multicluster. Nutanix Objects also uses its own storage allocator,
which removes the need for a filesystem and keeps data off the MSP kernel, improving I/
O performance. If you need additional vDisks, Nutanix Objects issues Remote Procedure
Calls to the CVM to automatically provision the vDisks.
Hybrid disk systems write sequential data (synonymous with large objects) directly to
the HDD tier and use the SSD tier almost exclusively for metadata, which helps ensure
low latency for metadata lookups. Nutanix Objects uses volume groups for metadata

© 2024 Nutanix, Inc. All rights reserved | 16

 Nutanix Objects

because you can easily detach and reattach them to a different worker during a high
availability failover event. Nutanix Objects automatically pins the metadata volume
groups to the hot tier during object store deployment.
Note: To achieve high throughput performance with large objects, we recommend equipping hybrid disk
nodes with as many HDDs as possible.

Object Store Storage Efficiency

Every object store instance maps to a pair of storage containers, one for data (prefixed
with objectsd) and one for metadata (prefixed with objectsm). Using this mapping, you
can manage compression and erasure coding settings individually for each object store
deployed. Both inline compression and, with Nutanix Objects version 3.5.1, inline erasure
coding are enabled by default on the data (but not the metadata) storage container to
save capacity. A postprocess erasure coding setting supplements the inline erasure
coding setting for scenarios where data can't be erasure-coded inline due to ingest rates
exceeding the rate of inline erasure coding. Nutanix Objects erasure-codes data that isn't
erasure-coded inline after three days.
Note: Nutanix doesn't recommend turning on deduplication.

Object Store Scaling

You can select the number of workers in an object store during deployment. While no
upper limit exists technically, in practice, to ensure high availability, an object store can't
have more workers than physical nodes. If the AOS cluster has physical nodes without
workers and the object store has at least three workers, you can add more workers using
a one-click scale-out operation. I/O performance scales linearly with each worker you
add to the object store.
Note: While Nutanix supports one- and two-worker deployments, we recommend only using them in remote
office and branch office (ROBO) deployments due to their lower resilience and the fact that you can't scale
them out later.

For more information on scaling limits for Nutanix Objects, see the configuration
maximums page.

© 2024 Nutanix, Inc. All rights reserved | 17

 Nutanix Objects

Nutanix Objects Multitenancy

You can divide a Nutanix Objects deployment between multiple tenants to meet the need
for object store multitenancy in one of two ways:
• Deploy multiple object stores on the same underlying AOS cluster.
• Deploy a single object store and assign multiple FQDNs to it.
When you deploy multiple object stores, the mapping of tenants to instances is 1:1.
Each instance has its own set of workers and load balancers, so a degree of hardware
resource sharing is likely.
If you deploy a single object store, the mapping of tenants to instances is n:1, so the
store uses one set of workers and load balancers to support multiple tenants. Nutanix
Objects has several features that allow you to divide a single object store among multiple
tenants:
• Use Objects Browser as a self-service portal for tenants. You can manage bucket
policies and user access to buckets from Objects Browser without needing to access
Prism.
• Assign multiple FQDNs to a single object store so that each tenant can access the
object store using their own unique FQDN. You can add the relevant SSL security
certificates for each domain.
• Generate access keys for users who belong to different Active Directories or
OpenLDAP directories.
• Use soft and hard quotas for control over tenant resource usage.
• Use bucket tagging to enable service providers to track ongoing tenant usage by
tenants.

Nutanix Objects Networking

The Nutanix Objects architecture uses two networks—an external (public) network
that clients can reach the load balancers through and an internal (storage) network to
facilitate communication between the load balancers, worker VMs, and CVMs. Both
networks must be managed (IPAM networks) so that the correct subnet, gateway, and

© 2024 Nutanix, Inc. All rights reserved | 18

 Nutanix Objects

DNS server details are available to the underlying MSP. Additionally, the storage network
must have its own DHCP pool.
Calculate the minimum number of IP addresses needed in the storage address pool as
follows: number of workers + number of load balancers + 3. For example, for an object
store with three workers and two load balancers, the storage network's DHCP pool needs
at least eight IP addresses. In addition to the DHCP pool addresses, statically assign
two IP addresses from the storage network and one IP address per load balancer from
the public network. The following table shows the IP address quantities required when
deploying an object store consisting of x workers and y load balancers.
Note: You can have public and storage IP addresses in the same network, but we don't recommend it.

Table: IP Address Calculations for Object Store

Allocation Method Network Number of IP Purpose
Addresses Required
Static Public y Client accessibility
Static Storage 2 MSP DNS and API
servers
DHCP (IPAM pool) Storage x+y+3 Worker, load balancer,
and CVM connectivity;
high availability of
MSP services (MSP A
and MSP B) and IPAM
DHCP server

Each CVM already has an IP address allocated, so we didn't account for these
addresses in the previous table. CVMs must have access to the object store's storage
network. We recommend connecting the CVMs' eth0 NICs directly to the storage network
to minimize I/O latency.

© 2024 Nutanix, Inc. All rights reserved | 19

 Nutanix Objects

Figure 5: Network Addresses Required in a Nutanix Objects Deployment

Give Prism Central, which manages the object store and displays performance and
utilization metrics, access to the public and storage networks. For more information about
network configuration, see the Deployment Checklist section.

Nutanix Objects Client Access

After the load balancer forwards the client request to a worker, the IAM service
authenticates the request. Before Nutanix Objects version 4.2, the IAM service ran
exclusively in the first object store deployed under Prism Central. Nutanix Objects
version 4.2 and later uses the central IAM service associated with the ControlPlane
Microservices Platform (CMSP) in Prism Central, with full copies of the S3 user keys
synchronized to every object store under the Prism Central instance. Each object store
can handle user authentication requests locally, with no dependency on the first object
store. The IAM service contains a database of IAM users, and it associates one or more
pairs of access keys and secret keys with each user's email address.

© 2024 Nutanix, Inc. All rights reserved | 20

 Nutanix Objects

Prism Central admins with the Create_Object_Store permission can populate the IAM
database (described in the Role-Based Access Control section) and generate keys
for both standalone users (based on email address) and users belonging to an Active
Directory or Open LDAP directory. After the Prism admin enters the appropriate directory
details and lookup credentials into Prism, they can generate keys for directory users
individually or for all members of a security group at once.
Sharing permissions are assigned at the bucket level, and you can set permissions
on a per-API basis so that Nutanix Objects evaluates the specific API call contained in
every request. Nutanix Objects version 4.3 introduced support for public bucket sharing,
which allows you to share buckets with anonymous users (users who do not have
authentication keys).

Nutanix Objects Buckets

Buckets are logical containers in which you store objects. A single object store can
contain thousands of buckets. You set access permissions at the bucket level, and you
can share a bucket with as many IAM users as required.

Life Cycle Policies

You can create a life cycle policy for a bucket that affects objects written to the bucket (all
objects or a subset). You can use life cycle policies to tier objects after a specified time
or to expire objects, object parts (if using multipart upload), or past versions of objects.
These life cycle mechanisms help administrators manage storage usage.
Note: The tiering policy option is only available after you define a tiering endpoint for the object store
(discussed in the Tiering section).

To apply a life cycle policy to a specific subset of objects in a bucket, use either or both of
the following filtering mechanisms:
• Filter on a prefix in the object name
• Filter on an object tag or set of tags
The life cycle policy only affects the objects that meet the identification criteria.

© 2024 Nutanix, Inc. All rights reserved | 21

 Nutanix Objects

Versioning and Write Once, Read Many

Nutanix Objects fully supports the S3 versioning API. This feature preserves older
versions of an object when users create a new version of the same object and is used to
recover accidentally deleted or replaced object data.
Write Once, Read Many (WORM) refers to data immutability, where data is locked and
can't be modified or deleted for a defined period. Nutanix Objects is fully validated for
compliance WORM. Once enabled on a bucket, you can't disable WORM or reduce
the WORM period except during the initial 24-hour grace period when you can revert
the WORM option. No one, including admins, can delete or modify any objects created
in WORM-enabled buckets until the retention period elapses. You can extend WORM
retention periods but not reduce them.
Note: If you use WORM locking and a life cycle policy on the same set of objects, ensure that you set the
lock to expire before the object is due to expire.

Nutanix Objects supports WORM for nonversioned and versioned buckets. With
nonversioned buckets, WORM only allows one PUT request for an object and doesn't
allow additional PUT requests or deletions for existing objects. With versioned buckets,
WORM allows PUT requests to existing objects so that the previous version of the object
still exists (and can't be deleted) but the object's key now relates to a new current version
of the object. This feature allows you to condense objects for versioned workflows (such
as document revisions) so that you don't need to create a new key with every new
version.

Multiprotocol Support
Nutanix Objects supports NFSv3 read and write access to buckets. Because object data
is inherently immutable, we recommend using NFS only where the written file data isn't
subsequently updated. NFS suits several use cases, such as consolidating old and new
backups and ingesting legacy application data that needs to be accessed for analysis
over HTTP or HTTPS using the S3 API. For more information, see Use Cases and
Recommendations for NFS on Objects.
Note: You can't enable NFS access on object stores that were originally deployed before Nutanix Objects
version 3.3 and upgraded. You also can't enable other S3 bucket features, such as life cycle policies,
versioning, WORM, replication, static website, cross-origin resource sharing (CORS), and notifications, on
NFS-enabled buckets.

© 2024 Nutanix, Inc. All rights reserved | 22

 Nutanix Objects

Other Nutanix Objects Features

Other bucket-level Nutanix Objects features include the following:
Streaming replication
Protect and distribute object data. See the Streaming Replication section.
Notifications and observability
Export metrics and event data to third-party systems for centralized observability
and event notification handling. You can also forward events to Nutanix Data Lens
for analytics-derived insights. See the Notifications and Observability section.
Static website
Use Nutanix Objects to host a static website. After you configure the bucket for
website hosting, you can upload your website files (such as index documents and
error pages) to the bucket or choose to redirect to a website.
Cross-origin resource sharing
Use CORS to allow a web application loaded in one domain to access restricted
resources requested from another domain, creating rich web applications.
Tags
Assign a tag (consisting of a key and value pair that you define) to a bucket or
an object. Nutanix Objects supports object- and bucket-level tags. You can use
object-level tags to ensure that a life cycle policy applies only to a targeted subset
of objects in a bucket or to denote the state of an object. You can use bucket-level
tags to obtain a more accurate view of tenant usage in a multitenant environment.

Nutanix Objects High Availability

Nutanix Objects comprises many containerized services that are distributed across the
cluster. Every worker runs most of these services; however, the metadata service runs
on all but one of the workers deployed on day one. For example, if you deployed an
object store with 11 workers, 10 of those workers host metadata services. Not scheduling
metadata services on the eleventh worker under normal operations provides n + 1
protection for the metadata service.
If the worker VM hosted on a failed physical node was running a metadata service
instance, Kubernetes reschedules the metadata service to run on the worker that

© 2024 Nutanix, Inc. All rights reserved | 23

 Nutanix Objects

previously didn't host metadata services. The corresponding metadata volume group
connects to that worker VM, and all workloads continue.
Additionally, the IAM service runs on only one worker VM. If that worker VM experiences
an outage, Kubernetes reschedules the service to run on one of the surviving
worker VMs. The corresponding IAM volume group connects to that worker VM and
authentication to the object store can continue.

Figure 6: Nutanix Objects High Availability

Because all deployments have one fewer load balancer VM than worker VMs, when the
host of a load-balancer VM fails, the affected load balancer VM restarts on a node that
isn't already hosting a load balancer.

© 2024 Nutanix, Inc. All rights reserved | 24

 Nutanix Objects

Note: If a load balancer experiences issues, we recommend removing its IP address from the object store's
record in the DNS and restoring it after you address the issues.

When a node fails with client I/O operations in flight, the S3 client typically times out and
retries the operation. The Atlas service identifies any partially written data when it does
its next scan and discards it. Load balancers also detect the loss of a worker VM through
periodic polling and stop directing requests to that worker VM.
Data availability for both metadata and object data comes from Nutanix storage's data
resilience mechanisms, such as erasure coding (EC-X) and replication factor for any data
that isn't erasure-coded.

Nutanix Objects Management and Monitoring

Manage Nutanix Objects from Prism Central. Prism Central provides several charts to
show usage and performance over an adjustable time range. Performance statistics are
available at both the object store and individual bucket levels.

Notifications and Observability

With notifications for Nutanix Objects, you can send event logs to configured endpoints,
which helps with centralized event log management, enabling you to use a third-party
app to monitor and analyze the logs and identify issues with an Objects deployment.
Nutanix Object supports syslog, Neural Autonomic Transport System streaming, and
Kafka for event notifications. You can send both bucket management–event and data-
event logs to the configured endpoints using the chosen mechanism. Transmission
Control Protocol is the supported protocol for Nutanix Objects notifications.
Starting with version 4.2, Nutanix Objects is compatible with Nutanix Data Lens, an
optional data analytics as a service offering from Nutanix. If you enable Nutanix Data
Lens, Objects forwards events to the Data Lens service on an ongoing basis. Nutanix
Data Lens profiles the data and metadata that it collects based on attributes such as
object age, size, and type, shows consumption trends over time, and, by tracking all
S3 client operations, provides a fully searchable audit trail. Nutanix Data Lens flags
suboptimal policy configurations and can generate advanced reports.
For observability purposes, Nutanix Objects also supports a statistics exporter function
that exposes an endpoint from which a Prometheus instance can scrape Objects
performance and utilization statistics along with other metrics, such as resource metrics

© 2024 Nutanix, Inc. All rights reserved | 25

 Nutanix Objects

for the Kubernetes pod. After the Prometheus instance fetches the metrics, it can
relay them to a Grafana deployment that visualizes the data in monitoring dashboards.
Exportable metrics are listed in Objects Prometheus Exporter.

Quotas
You can set up hard and soft quotas for IAM users, restricting the capacity that they can
consume and the number of buckets that they can create. A soft quota generates an alert
when the user exceeds the quota, and a hard quota actively prevents users from making
more PUT requests or creating more buckets. Hard capacity quotas are based on usage
across all buckets owned by the individual user. If the user shares their buckets with
other users, no one can write more data to those buckets when the quota is reached,
regardless of who writes the data to the buckets.

Alerts
Nutanix Objects alerts appear on the Alert Manager page in Prism Central. You can
configure the Alert Manager functionality to email Objects alerts to specific recipients.
Alerts exist for exceeded quotas, low capacity, and so on.

Objects Browser
Objects Browser is a web browser–based S3 client that comes with every Nutanix
Objects deployment. Objects Browser allows IAM users to perform bucket- and object-
level operations, including bucket management. Users can self-provision buckets (within
the limitations imposed by quotas), upload or download objects, and share access with
other users. In addition, users can manage bucket life cycle policies and access past
versions of objects. Access Objects Browser by entering the following URL format into
your browser: https://<object store name or IP address>/objectsbrowser.
Nutanix Objects version 4.3 introduced support for object visualization, which extends
Objects Browser’s functionality so that certain file formats can be viewed (videos,
images, PDFs), listened to (audio files), edited (text files), or queried (CSV files) directly
from a web browser.
Objects Browser supports multipart upload, which breaks objects larger than 1 GB
into parts and uploads each part sequentially. This process minimizes the impact of an
upload failure because you only need to retry to upload the affected part rather than
the entire object so that you can upload very large objects. Nutanix Objects (though not

© 2024 Nutanix, Inc. All rights reserved | 26

 Nutanix Objects

Objects Browser) also supports parallel multipart uploads, allowing large object uploads
to finish faster. Both parallel and sequential multipart uploads result in more reliable
uploads.
Note: You can retry each part of an upload request a maximum of three times.

Objects Browser provides a Recycle Bin feature for versioned buckets that allows
you to view all past versions of an object after the live version of the object is deleted.
The Recycle Bin feature makes it easy to restore a past version of any object. Nutanix
Objects version 5.0 introduced accessibility enhancements to Objects Browser, including
support for screen readers on both Mac and Windows and keyboard support for every
Objects Browser workflow.
Objects Browser is also federation-aware, meaning that you can use it to access
geographically distributed object data from a single UI. We describe federations in the
Nutanix Objects Global Namespace section.

Object Store Features

Object stores in Nutanix Objects have the features discussed in the following sections.

Deployment Checklist
When you begin the process of deploying a new object store, Nutanix Objects provides
a checklist with all the prerequisites you must meet, including diagrams of the network
ports you must open. Nutanix Objects guides you through the required inputs for the
deployment details. After you enter all the parameters, Prism Central automatically
performs a check. Any issues detected prevent the deployment from starting and are
flagged so that the admin can address them before resubmitting the deployment request.
This process minimizes the chances of a failed deployment.

FQDN Assignment
An object store can have multiple associated FQDNs that allow it to respond to S3
requests targeted at object stores other than itself. This assignment facilitates seamless
disaster recovery failovers between two inter-replicating Nutanix object stores (described
in more detail in the Streaming Replication section). Additionally, in multitenant
environments, where a single object store is shared across multiple tenants, multi-FQDN
allows each tenant to address the object store as if it were in their own domain.

© 2024 Nutanix, Inc. All rights reserved | 27

 Nutanix Objects

Nutanix Objects automatically assigns new object stores a default FQDN in the format
<your_objectstore_name>.prism-central.cluster.local, which is a subdomain of the
Prism Central domain. The control plane for MSP requires this format, and you can't
change it. However, you can assign your own FQDN to the object store by navigating
to the Manage FQDNs & SSL Certificates page in Nutanix Objects and using the multi-
FQDN feature.
You can import SSL certificates for each FQDN to establish secure connections to all
named endpoints. We describe SSL certificates more in the Nutanix Objects Security
section.

Object Store Capacity

If you need additional capacity, add physical nodes to the physical (AOS) cluster. The
new nodes' capacity is instantly available to any object stores running on the cluster.
By default, object stores are thin-provisioned so that they can access all the capacity
available in the AOS cluster. However, you can set a maximum capacity for the object
store. In this case, when you add a new node to the AOS cluster, you must explicitly
increase the object store's capacity limit.
If you set a maximum capacity, when capacity usage hits 90 percent, an alert that the
cluster is close to exceeding the assigned capacity appears in Prism. When the cluster
reaches 100 percent capacity usage, an alert that the cluster exceeds the maximum
capacity appears in Prism, but you can still make PUT requests.
If required, an object store can use the multicluster capability to access the capacity of
up to four other physical Nutanix clusters in the same datacenter. No workers or load
balancers are deployed on the secondary clusters; instead, the workers on the initial
cluster make direct API calls to the CVMs on the secondary clusters. The secondary
clusters might run other application workloads, so you can limit the capacity that the
object store can consume from these secondary clusters. In a multicluster setup, writes
occur in a round-robin fashion across all the participating clusters. When a cluster fills
85 percent of its allocated storage, it doesn't receive any more writes until all the other
clusters reach 85 percent storage usage.

© 2024 Nutanix, Inc. All rights reserved | 28

 Nutanix Objects

Figure 7: Multicluster Cross-Cluster Consumption

Note: All secondary clusters must be managed by the same Prism Central instance that manages the initial
cluster.

Tiering
Life cycle policies that drive object data tiering to the public cloud or another object store
provide capacity management and can prevent or delay the need to increase an object
store's capacity.
Before you can apply a tiering policy to any bucket, you must establish connectivity to
the tiering endpoint at the object store level by entering information about the endpoint,
access keys, and SSL certificate. Nutanix Objects supports tiering to Amazon Web
Services S3, Azure Blob, Google Cloud Platform, or any S3-compatible endpoint.

© 2024 Nutanix, Inc. All rights reserved | 29

 Nutanix Objects

Figure 8: Object Tiering

Before tiering data, Nutanix Objects automatically concatenates multiple objects into
larger objects (regions), which can reduce the API-related costs—in this case, the
number of PutObject operations—chargeable by the endpoint provider. Nutanix Objects
then uses multipart upload to break these regions into parts just before the data is tiered
and uploads the parts in parallel. This process helps the tiering operation finish faster.
Note: Tiering involves sending data directly from the object store workers to the tiering endpoint. Ensure that
you have the appropriate routing from the storage network.

When an application needs to read data tiered to a public cloud provider (or other
S3 storage), Nutanix Objects can use range reads to avoid reading more data than
is necessary, helping minimize egress charges. Nutanix Objects only retrieves the
necessary data from the endpoint. The local object store retains the metadata for tiered
objects, which helps make tiered object access more transparent.

Nutanix Objects Upgrades

Nutanix Objects uses Prism Central's Life Cycle Manager (LCM) for upgrades, which
ensures that the deployment fulfills all dependencies before the upgrade starts. For
Nutanix Objects, LCM checks compatibility between the versions of Objects Manager,

© 2024 Nutanix, Inc. All rights reserved | 30

 Nutanix Objects

Objects Service, MSP, AOS, and Prism Central. You must also upgrade Nutanix Cluster
Check so that health checks relevant to new features are available.
To upgrade any component before deploying the latest Nutanix Objects version, LCM
creates a sequenced plan and runs it after the admin approves it. The LCM upgrade
process downloads the new Nutanix Objects binaries from secure web servers managed
by Nutanix. However, if the upgrade is at a dark site, follow the procedure in Deploying
Object Store at a Dark Site (Offline Deployment).

Nutanix Objects Security

Nutanix Objects supports several security-related features, such as data immutability and
encryption.

Object Lock
When you enable WORM on a Nutanix Objects bucket, the same retention period
(specified in the bucket WORM policy) is applied to all objects written to the bucket.
However, Nutanix Objects also supports retention period locking at the individual object
level. The application can use the PutObjectRetention API to set a lock period on a per-
object basis when objects are written to a WORM-enabled bucket.
Note: When locking at the object level, we recommend setting the bucket lock policy to 0 days to avoid any
unexpected behavior or confusion over when the lock is released.

Legal Hold
Nutanix Objects supports legal hold, a variant of the WORM locking that allows an
authorized user to set an indefinite WORM lock on an object. You can use a legal hold
if you need digital records to remain accessible for the duration of a legal case or an
external audit. The lock remains in place until the admin explicitly removes it.

Data-at-Rest Encryption
Nutanix Objects provides FIPS 140-2–compliant server-side encryption by using self-
encrypting drives or software-defined data-at-rest encryption (DaRE) through AOS.
AHV-based clusters set encryption at the cluster level, ensuring that all data is always
encrypted.

© 2024 Nutanix, Inc. All rights reserved | 31

 Nutanix Objects

For ESXi, you can enable software encryption at the cluster level or the storage container
level. To enable storage container–level software encryption with Nutanix Objects, run
the following command in the Nutanix CLI (nCLI):
<ncli> storage-container edit enable-software-encryption=true
name=<Objects_Container_Name> force=true

The performance overhead incurred by encryption is extremely small (5 percent or less).

Prism Central provides native key management, but you can also use a third-party key
management system.

In-Flight Encryption
To ensure that clients can establish secure encrypted HTTPS sessions to the Nutanix
object store, you need SSL certificates. Nutanix Objects supports self-signed certificates
(using an RSA 2,048-bit private key), or, if you have a certificate authority–signed
certificate, you can import your own private key and certificate. Nutanix Objects
generates self-signed certificates by default.
Note: If using a certificate signed by your own certificate authority, install the root certificate on all client
machines as a trusted root certificate authority.

Nutanix Objects supports Transport Layer Security (TLS) 1.2 for encrypting data
transmissions.

HTTP Strict Transport Security

Nutanix Objects supports HTTP Strict Transport Security (HSTS) in a manner compliant
with RFC 6797. HSTS is an HTTP header designed to help mitigate HTTP redirection
attacks: When an HTTP request is issued to a website (in our case, Nutanix Objects), an
attacker can intercept the unencrypted request and redirect it to a malicious site, to which
the encrypted connection is then established. HSTS avoids this scenario by forcing the
browser to issue only HTTPS requests to the target website right from the start, providing
no opportunity for malicious redirection.
HSTS is set at the object store level, so the policy affects all buckets. It isn't enabled by
default. Consider the following when using HSTS:
• The HSTS policy is only effective after the site is visited initially.
• You can only apply the policy over an HTTPS connection and if the object store has a
valid certificate.

© 2024 Nutanix, Inc. All rights reserved | 32

 Nutanix Objects

• HSTS only works when you use FQDN for access.

• If the object store's SSL certificate expires, all users are locked out.

Role-Based Access Control

Role-based access control (RBAC) restricts the actions Nutanix Prism administrators
can perform against Objects-related resources. Nutanix Objects–specific roles exist in
Prism Central, and you can customize them granularly. To use RBAC, you must enable
Microservices Infrastructure (CMSP) in Prism Central. CMSP is enabled by default as of
Prism Central version 2023.1.0.2.
Note: The administrator RBAC described here is different from S3 user access permissions.

NFS Client Allowlist

You can restrict NFS client access to a specific set of client machines using an allowlist
that you fill with IP address ranges that represent permitted clients. Nutanix Objects
works on a principle of least privilege, so the allowlist is empty by default and no clients
have access. The NFS client allowlist operates at the object store level.

Flow Network Security

If the Nutanix Objects deployment runs on AHV, you can use Flow Network Security
microsegmentation to protect the object store from network-based attacks. For an
example of how to implement Flow Network Security microsegmentation, see Network
Microsegmentation in Hybrid Cloud: AOS 6.5 with AHV Unified Storage Design.
For more information on Flow Network Security, see Flow Network Security.

© 2024 Nutanix, Inc. All rights reserved | 33

 Nutanix Objects

4. Nutanix Objects Data Protection and

Recovery
Nutanix Objects provides an inherent backup capability through support for the S3
versioning API (see the Versioning and Write Once, Read Many section). With versioning
enabled on a bucket, whenever a user updates any object in the bucket, versioning
creates a new object and preserves the older version of the object. Even if a user deletes
the live object, you can still recover past versions of it, so versions essentially serve as
backups.
Nutanix Objects doesn't use AOS-level protection domain replication.

Streaming Replication
You can replicate objects between different Nutanix object stores using streaming
replication, a native data protection capability of Nutanix Objects. Streaming replication
copies objects (and object parts) written to a protected bucket to a corresponding remote
bucket (or buckets) as soon as the PUT request finishes. Before Nutanix Objects version
5.0, this process applied to all objects written to the bucket. With Nutanix Objects version
5.0 and later, you can apply replication to a subset of objects based on object name
prefix, tag, or a combination of both. Streaming replication copies object metadata and
WORM lock timers along with the data. Streaming replication generally provides a low
recovery-point objective (RPO), although factors such as write rate, object size, and
intersite bandwidth all influence the RPO.
Nutanix Objects supports up to five source buckets replicating to a single destination
bucket (fan-in replication) and one source bucket replicating to up to three destination
buckets (fan-out replication).
With Nutanix Objects versions 3.5 and later, you can skip replicating delete markers
to the destination bucket so that deletions on the source bucket aren't mirrored on the
destination bucket. The life cycle policies on the source and destination buckets are
also entirely separate so that the destination bucket can have a much longer retention
period than the source bucket. Starting with Nutanix Objects version 4.3, you can pause

© 2024 Nutanix, Inc. All rights reserved | 34

 Nutanix Objects

replication activity for a period. When you resume replication, objects created during the
pause are copied to the destination through a catch-up process.
Note: Replication traffic travels directly from the source object store's workers (storage network) to the
destination object store's load balancers (public network). Ensure that you have the appropriate routing in
place.

The source and destination buckets are both writable, meaning that a Nutanix Objects
disaster recovery setup is inherently active-active. This setup helps speed the failover
and failback processes between object stores.
Multi-FQDN support allows an object store to respond to S3 requests targeted at FQDNs
other than its own. Multi-FQDN facilitates rapid disaster recovery failovers between two
Nutanix object stores with streaming replication set up between them. If the source and
destination buckets have the same name and you configure each object store with the
other's FQDN in addition to its own, either object store can service S3 client requests
without modifications. If you add a global server load balancing (GSLB) solution, you can
create a zero-touch failover and failback design.
As of Nutanix Objects version 5.0, you can replicate from Nutanix Objects to native AWS
S3 buckets, with the replication managed entirely from the Objects interface in Prism
Central.
Note: Currently, objects that existed in a bucket before you configured streaming replication on that bucket
aren't automatically replicated. However, you can use Baseline Replicator to scan the bucket for unreplicated
objects and mark them for replication to ensure that they're replicated to the destination bucket. For more
information, see Baseline Replicator Tool.

Pairing Prism Central Availability Zones

If separate Prism Central instances manage the source and destination object stores,
pair the Availability Zones represented by the two Prism Central instances, replicate the
IAM keys between the two Prism Central instances so that the same user accounts can
access the data at the source and the destination.
Note: We recommend creating and managing the IAM keys in one Prism Central instance and replicating
them from that instance to one or more separate Prism Central instances.

Replicated Attributes
Bucket-sharing permissions don't automatically propagate to the destination bucket; you
must set them manually. Streaming replication replicates the following attributes:

© 2024 Nutanix, Inc. All rights reserved | 35

 Nutanix Objects

• Metadata
• Tags
• WORM timer
• Versions
Streaming replication doesn't replicate the following attributes:
• Bucket permissions
• Life cycle policies

Note: Nutanix Objects supports the AWS policy document format. You can export permissions to JSON and
import them on the destination buckets. Additionally, you can import and export bucket life cycle policies as
XML.

If the source bucket has WORM or versioning enabled, the destination bucket must also
have the same feature enabled or you can't create the replication rule. If you haven't
already enabled versioning on the source and destination buckets, you can't enable it on
either bucket unless you remove the replication rule; the same is true for WORM.

© 2024 Nutanix, Inc. All rights reserved | 36

 Nutanix Objects

5. Nutanix Objects Global Namespace

With Nutanix Objects Federation, you can create a global namespace, or federation,
across multiple Nutanix object stores, even if they are in entirely different geographic
locations. Buckets hosted by these different object stores then appear to exist in a
single object store, offering a consolidated view of the data. If you have object data
spread across multiple edge and core sites across the world and need to simplify and
consolidate access to your distributed data sets, you can benefit from this feature. While
much of Federation's value lies in how it dramatically simplifies data access in geo-
distributed environments, it's also well suited to scenarios where you need extreme
scalability: A single federation can store hundreds of petabytes of data across scores of
object stores, all under one S3 namespace.

Federation Architecture
During the process of creating a federation, you add members to it. Members are
existing object stores that contribute resources to the federated namespace, adding to
its overall capacity and performance capabilities. Select a subset of these members to
serve as core members, which are just regular members with extra responsibilities for
maintaining and managing the federated namespace. The following services run inside
the core members:
Federation metadata service
The metadata service tracks which buckets are hosted on which object stores
(members) and maintains consensus in the federation.
Federation controller
The controller routes client requests to the correct member. This service
communicates with the Federation metadata service to understand which buckets
are hosted on which members. It also directly handles create, update, and delete
operations for buckets in the federated namespace, eliminating the possibility of
conflicts. For example, you can't have two identically named buckets in the same
namespace.

© 2024 Nutanix, Inc. All rights reserved | 37

 Nutanix Objects

You can have one core member, although we recommend three or more core members
for fault tolerance. You can have at most five core members, which provides the highest
level of fault tolerance for the federation (the namespace remains available even if two
core members become unavailable).

Figure 9: Prism View of Federation Members and Core Members

As shown in the previous figure, members can belong to different Prism Central
instances. In fact, because Nutanix designed Prism Central instances to manage Nutanix
resources only in their own regions, in a geo-distributed environment, different Prism
Central instances manage members in different regions. In these scenarios, you must
establish trust between the Prism Central instances, which, as described in the Pairing
Prism Central Availability Zones section, you achieve by pairing the Availability Zones
that the Prism Central instances represent. After you pair the Availability Zones, you can
replicate the IAM user access keys between the Prism Central instances.

Viewing Utilization and Scaling

Prism Central has a Federated Namespaces page that shows usage across all members
of the federation in terms of the number of buckets deployed and the capacity consumed

© 2024 Nutanix, Inc. All rights reserved | 38

 Nutanix Objects

in those buckets. The numbers reported are specific to the federation in question, so
usage metrics pertaining to members' local namespaces and any other federations they
might belong to are filtered out.
From the Federated Namespaces page, you can add more members to or remove
existing members from a federation. Before removing a member, you must remove any
federation buckets that the member hosts.

Routing Client Requests

When an S3 client issues a GET, PUT, or HEAD (metadata) request against a federation
bucket, the client remains connected to the same federation member throughout the
entire process. The following list describes the process of routing a client request:
1. The client connects to a member (the client-connected member) and issues a GET,
PUT, or HEAD (metadata) request.
2. The client-connected member checks its cache to see if it has handled a request for
this bucket before.
If it has, it bypasses Steps 3 and 4.
3. If the client-connected member hasn't handled a request for the bucket before, the
client-connected member asks a core member which member hosts the bucket.
4. In the core member, the Federation controller service asks the Federation metadata
service which member hosts the bucket.
The Federation controller returns this information to the client-connected
member.
5. The client-connected member relays the client request directly to the requesting
member.
6. The requested operation is performed by the member hosting the bucket and the
data (or the success code in the case of a PUT) is returned to the client-connected
member.
7. The client-connected member relays the data or success code to the client.
The process is slightly different when a client issues a bucket create, update, or delete
request. For example, the following list describes the process for a create bucket
request:

© 2024 Nutanix, Inc. All rights reserved | 39

 Nutanix Objects

1. The client connects to a member (the client-connected member) and issues a request
to create a bucket named bucket01.
2. The client-connected member asks the core member to create the bucket in the
federated namespace and place it on the client-connected member (itself).
3. In the core member, the Federation controller service asks the Federation metadata
service if a bucket with the name bucket01 already exists in the namespace.
a. If a bucket with this name already exists, the request fails and the failure code is
passed to the client through the client-connected member.
b. If a bucket with this name doesn't already exist, the following process occurs:
i. The Federation controller instructs the client-connected member to create
bucket01.
ii. The Federation controller instructs the Federation metadata service to
update the Federation metadata with the name and location of bucket01.
iii. The core member's Federation controller informs the client-connected
member that it successfully created bucket01 and updated the Federation
metadata.
iv. The client-connected member informs the client that bucket01 was
successfully created in the federated namespace.

Managing Namespaces
The following figure shows five object stores participating in a federation together. An S3
client looking into the namespace sees six buckets listed. The six buckets are spread out
across five different object stores in five different locations, but to the client it looks like all
those buckets belong to one object store.

© 2024 Nutanix, Inc. All rights reserved | 40

 Nutanix Objects

Figure 10: Consolidated View of Distributed Buckets

When an object store joins a federation, any existing data remains unaffected. The
federated namespace that the object store is now a member of sits beside its own local
namespace (and any other federated namespaces the object store might be a member
of), and all namespaces are managed independently. This distinction is clear in Prism
Central, as shown in the following figure. In this example, the bucket listings for the
individual object store contain three tabs. The first tab represents the object store's local
namespace, followed by a tab for each federation the object store is a member of. An
object store can be a member of up to 32 federations at the same time.
The local namespace tab shows a list of all the buckets in the local namespace. When
you select a Federation Namespace tab, you see a list of which buckets in that federation
the particular object store hosts. Other buckets in the same federation that are hosted on
other object stores are not shown.

© 2024 Nutanix, Inc. All rights reserved | 41

 Nutanix Objects

Figure 11: Nutanix Objects Buckets Tab View for Example Federation

You can use an S3 client, such as Objects Browser, to get the consolidated view. You
can launch Objects Browser by using the link provided in Prism Central or by entering a
special URL into your browser. As of the Nutanix Objects 4.0 release, Objects Browser
is federation-aware, meaning that it can provide a complete listing of all buckets in a
federated namespace, assuming that the authenticated IAM user has the appropriate
access permissions to all the buckets. With the correct permissions, the authenticated
user can perform PUTs and GETs to any federation bucket, create or delete buckets in
the global namespace, and manage life cycle policies associated with federation buckets.
The URL format for accessing Objects Browser returns a list of buckets in the object
store's local namespace. However, you can use the following URL format to access a
federation namespace through a specific member: https://<object store name or IP
address>/objectsbrowser?namespace=<federation name>. Any buckets that you create
using Objects Browser are hosted on the member specified at the beginning of the URL.
Any S3 client can access all buckets in a federated namespace regardless of the
individual bucket locations.

© 2024 Nutanix, Inc. All rights reserved | 42

 Nutanix Objects

Fault Tolerance and Migration

In many Federation environments, you can use streaming replication to provide
fault tolerance for buckets in the federated namespace or to migrate data between
namespaces. You can set up replication in either direction between any of the following:
• Buckets in the same federated namespace (fault tolerance use case)
• Buckets in different federated namespaces (data migration use case)
• A bucket in a local namespace and a bucket in a federated namespace (data
migration use case)

Figure 12: Replication Use Cases for Federated Namespaces

The source and destination buckets can be on the same object store, which is useful for
migration.

© 2024 Nutanix, Inc. All rights reserved | 43

 Nutanix Objects

When using replication to achieve data fault tolerance in the namespace, we recommend
using a global server load balancing (GSLB) service to detect the loss of an object store
and seamlessly redirect client requests to the replication destination bucket (the failover).

Client Access Locality

A GSLB service can also enforce client access locality in a geo-distributed federation,
ensuring that clients connect to the nearest member. Access locality helps provide the
best possible client experience when it comes to performance.
To make Federation work, you must add all participating object stores' public IP
addresses to the DNS under the federation's name. However, if participating object
stores are geo-distributed, clients might connect to a remote object store. Clients have
access, but they might be routed across long distances, even for the buckets that are
hosted locally, which adds unwelcome latency. A GSLB service uses techniques such as
network latency detection to ensure that clients are directed to their nearest object store
when they connect to the federated namespace.
The GSLB services referred to here are different from Nutanix Objects' own integrated
load balancers, which are local traffic managers that are discrete to a single object store.
A GSLB service works across multiple object stores.
Another way to achieve access locality is to use multisite DNS. If you configure multisite
DNS in the environment, you can set up local object store IP addresses for local clients,
ensuring that clients' DNS lookups of the federation namespace always resolve to a
member local to them.
With both solutions, all clients can still access all data in the federation (remotely hosted
bucket accesses are internally routed by the local object store), but accessing local data
doesn't require a long-distance trip.
You can also control the placement of new buckets by creating the bucket in Prism
Central (admins only) or Objects Browser. When creating buckets with any other S3
client, you need a GSLB service or multi-DNS to ensure that you create the bucket
locally.

© 2024 Nutanix, Inc. All rights reserved | 44

 Nutanix Objects

Multicluster vs. Federation

Federation and the Nutanix Objects multicluster feature both allow a single Objects
namespace to span multiple physical clusters. However, they have the following key
differences:
• The resources being scaled are different. Multicluster adds physical AOS clusters to
the namespace, whereas Federation adds object store instances.
• Multicluster only scales storage capacity, while Federation scales storage capacity and
performance (the Nutanix Objects services are also scaled).
• Multicluster supports up to five AOS clusters in the same namespace, while
Federation supports up to 128 object stores in a single global namespace.
• With multicluster, all clusters must be in the same datacenter. With Federation, the
object store members can be in different locations.
• With multicluster, each bucket spans all the participating clusters. With Federation,
members host buckets discretely (a bucket doesn't spill across another member).
When using Federation in a geo-distributed environment, performance is consistent in
each bucket.
• Nutanix designed multicluster for simple capacity scaling. Federation simplifies
access to data distributed across multiple different locations and provides scaling on a
different level altogether.

© 2024 Nutanix, Inc. All rights reserved | 45

 Nutanix Objects

6. Conclusion
As part of the Nutanix Unified Storage (NUS) suite, Nutanix Objects provides scalable,
secure, and simple-to-manage S3-compliant object storage. Nutanix Objects' modern
containerized and highly distributed implementation means deployments range from
very small to extremely large. Because it's software-defined, Nutanix Objects can run
on a variety of infrastructure types, and Objects deployments can be dedicated or share
infrastructure with applications to reduce the overall amount of infrastructure needed.
Nutanix Objects runs on AOS and benefits from the scalability and resilience capabilities
of the core Nutanix platform and its data efficiency features such as inline compression
and erasure coding. With support for features such as streaming replication, tiering,
multiprotocol access, and compliance WORM locking coupled with great performance for
large and small objects, Nutanix Objects is a highly versatile S3 storage solution.

© 2024 Nutanix, Inc. All rights reserved | 46

 Nutanix Objects

About Nutanix
Nutanix offers a single platform to run all your apps and data across multiple clouds
while simplifying operations and reducing complexity. Trusted by companies worldwide,
Nutanix powers hybrid multicloud environments efficiently and cost effectively. This
enables companies to focus on successful business outcomes and new innovations.
Learn more at Nutanix.com.

© 2024 Nutanix, Inc. All rights reserved | 47

List of Figures
Figure 1: Special Nutanix VMs and Their Roles in Nutanix Objects........................................................... 11

Figure 2: Core Nutanix Objects Microservices............................................................................................ 14

Figure 3: Anatomy of a PUT Request......................................................................................................... 15

Figure 4: Anatomy of a GET Request......................................................................................................... 16

Figure 5: Network Addresses Required in a Nutanix Objects Deployment................................................. 20

Figure 6: Nutanix Objects High Availability..................................................................................................24

Figure 7: Multicluster Cross-Cluster Consumption.......................................................................................29

Figure 8: Object Tiering................................................................................................................................30

Figure 9: Prism View of Federation Members and Core Members............................................................. 38

Figure 10: Consolidated View of Distributed Buckets..................................................................................41

Figure 11: Nutanix Objects Buckets Tab View for Example Federation.......................................................42

Figure 12: Replication Use Cases for Federated Namespaces...................................................................43